Add API to load a unpacked project to developerPrivate.

This adds 3 api's to developerPrivate.

1) Load an unpacked project (app / extension).
2) Get the list of projects.
3) Skeleton of copying a project from local filesystem to syncfs.



Add API to load a unpacked project to developerPrivate.

Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=195116

[miket_OOO, 2013-04-18 03:11:59]

lgtm

https://codereview.chromium.org/13885007/diff/1/chrome/browser/extensions/api...
File chrome/browser/extensions/api/developer_private/developer_private_api.cc
(right):

https://codereview.chromium.org/13885007/diff/1/chrome/browser/extensions/api...
chrome/browser/extensions/api/developer_private/developer_private_api.cc:936:
path = path.Append(project_name);
Please add a TODO to sanitize this value, and further to assert that the end
result is within kUnpackedAppsFolder. I know that it's a private API, but it's
still taking user-generated, external values, and this would be the appropriate
place to assert that it's not a value like "../../../My Documents/taxes."

https://codereview.chromium.org/13885007/diff/1/chrome/common/extensions/api/...
File chrome/common/extensions/api/developer_private.idl (right):

https://codereview.chromium.org/13885007/diff/1/chrome/common/extensions/api/...
chrome/common/extensions/api/developer_private.idl:164: // Gets the list of
projects with metadata.
This description is confusing. It sounds like a filter (get a list of the
projects that have metadata, excluding those that don't).

[miket_OOO, 2013-04-18 03:15:04]

Also: PLEASE PUT A TODO to add unit tests. We can't go much farther than this
without tests.

[Gaurav, 2013-04-18 17:24:06]

https://codereview.chromium.org/13885007/diff/1/chrome/browser/extensions/api...
File chrome/browser/extensions/api/developer_private/developer_private_api.cc
(right):

https://codereview.chromium.org/13885007/diff/1/chrome/browser/extensions/api...
chrome/browser/extensions/api/developer_private/developer_private_api.cc:936:
path = path.Append(project_name);
On 2013/04/18 03:12:00, miket wrote:
> Please add a TODO to sanitize this value, and further to assert that the end
> result is within kUnpackedAppsFolder. I know that it's a private API, but it's
> still taking user-generated, external values, and this would be the
appropriate
> place to assert that it's not a value like "../../../My Documents/taxes." 

Done.

https://codereview.chromium.org/13885007/diff/1/chrome/common/extensions/api/...
File chrome/common/extensions/api/developer_private.idl (right):

https://codereview.chromium.org/13885007/diff/1/chrome/common/extensions/api/...
chrome/common/extensions/api/developer_private.idl:164: // Gets the list of
projects with metadata.
On 2013/04/18 03:12:00, miket wrote:
> This description is confusing. It sounds like a filter (get a list of the
> projects that have metadata, excluding those that don't).

Done.

[commit-bot: I haz the power, 2013-04-18 17:24:37]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/grv@chromium.org/13885007/7001

[commit-bot: I haz the power, 2013-04-18 17:24:43]

Presubmit check for 13885007-7001 failed and returned exit status 1.

INFO:root:Found 4 file(s).

Traceback (most recent call last):
  File "docs/server2/integration_test.py", line 6, in <module>
    from handler import Handler
  File
"/b/commit-queue/workdir/chromium/chrome/common/extensions/docs/server2/handler.py",
line 13, in <module>
    from server_instance import ServerInstance
  File
"/b/commit-queue/workdir/chromium/chrome/common/extensions/docs/server2/server_instance.py",
line 9, in <module>
    from api_data_source import APIDataSource
  File
"/b/commit-queue/workdir/chromium/chrome/common/extensions/docs/server2/api_data_source.py",
line 10, in <module>
    import third_party.json_schema_compiler.json_parse as json_parse
ImportError: No module named third_party.json_schema_compiler.json_parse
Traceback (most recent call last):
  File "docs/server2/link_converter.py", line 15, in <module>
    import third_party.json_schema_compiler.model as model
ImportError: No module named third_party.json_schema_compiler.model
Running presubmit commit checks ...
Running /b/commit-queue/workdir/chromium/PRESUBMIT.py
Running /b/commit-queue/workdir/chromium/chrome/PRESUBMIT.py
Running /b/commit-queue/workdir/chromium/chrome/browser/extensions/PRESUBMIT.py
Running /b/commit-queue/workdir/chromium/chrome/common/extensions/PRESUBMIT.py
Traceback (most recent call last):
  File "/b/commit-queue/verification/presubmit_shim.py", line 33, in <module>
    sys.exit(presubmit_support.Main(argv))
  File "/b/depot_tools/presubmit_support.py", line 1305, in Main
    rietveld_obj)
  File "/b/depot_tools/presubmit_support.py", line 1129, in DoPresubmitChecks
    results += executer.ExecPresubmitScript(presubmit_script, filename)
  File "/b/depot_tools/presubmit_support.py", line 1046, in ExecPresubmitScript
    result = eval(function_name + '(*__args)', context)
  File "<string>", line 1, in <module>
  File "<string>", line 173, in CheckChangeOnCommit
  File "<string>", line 166, in _CheckChange
  File "<string>", line 134, in _CheckLinks
  File "/b/depot_tools/subprocess2.py", line 449, in check_output
    return check_call_out(args, stdout=PIPE, **kwargs)[0]
  File "/b/depot_tools/subprocess2.py", line 412, in check_call_out
    returncode, args, kwargs.get('cwd'), out[0], out[1])
subprocess2.CalledProcessError: Command docs/server2/link_converter.py -o -f
/b/commit-queue/workdir/chromium/chrome/common/extensions/api/developer_private.idl
returned non-zero exit status 1 in
/b/commit-queue/workdir/chromium/chrome/common/extensions

[commit-bot: I haz the power, 2013-04-18 17:28:14]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/grv@chromium.org/13885007/13001

[commit-bot: I haz the power, 2013-04-18 17:28:20]

Presubmit check for 13885007-13001 failed and returned exit status 1.

INFO:root:Found 4 file(s).

Traceback (most recent call last):
  File "docs/server2/integration_test.py", line 6, in <module>
    from handler import Handler
  File
"/b/commit-queue/workdir/chromium/chrome/common/extensions/docs/server2/handler.py",
line 13, in <module>
    from server_instance import ServerInstance
  File
"/b/commit-queue/workdir/chromium/chrome/common/extensions/docs/server2/server_instance.py",
line 9, in <module>
    from api_data_source import APIDataSource
  File
"/b/commit-queue/workdir/chromium/chrome/common/extensions/docs/server2/api_data_source.py",
line 10, in <module>
    import third_party.json_schema_compiler.json_parse as json_parse
ImportError: No module named third_party.json_schema_compiler.json_parse
Traceback (most recent call last):
  File "docs/server2/link_converter.py", line 15, in <module>
    import third_party.json_schema_compiler.model as model
ImportError: No module named third_party.json_schema_compiler.model
Running presubmit commit checks ...
Running /b/commit-queue/workdir/chromium/PRESUBMIT.py
Running /b/commit-queue/workdir/chromium/chrome/PRESUBMIT.py
Running /b/commit-queue/workdir/chromium/chrome/browser/extensions/PRESUBMIT.py
Running /b/commit-queue/workdir/chromium/chrome/common/extensions/PRESUBMIT.py
Traceback (most recent call last):
  File "/b/commit-queue/verification/presubmit_shim.py", line 33, in <module>
    sys.exit(presubmit_support.Main(argv))
  File "/b/depot_tools/presubmit_support.py", line 1305, in Main
    rietveld_obj)
  File "/b/depot_tools/presubmit_support.py", line 1129, in DoPresubmitChecks
    results += executer.ExecPresubmitScript(presubmit_script, filename)
  File "/b/depot_tools/presubmit_support.py", line 1046, in ExecPresubmitScript
    result = eval(function_name + '(*__args)', context)
  File "<string>", line 1, in <module>
  File "<string>", line 173, in CheckChangeOnCommit
  File "<string>", line 166, in _CheckChange
  File "<string>", line 134, in _CheckLinks
  File "/b/depot_tools/subprocess2.py", line 449, in check_output
    return check_call_out(args, stdout=PIPE, **kwargs)[0]
  File "/b/depot_tools/subprocess2.py", line 412, in check_call_out
    returncode, args, kwargs.get('cwd'), out[0], out[1])
subprocess2.CalledProcessError: Command docs/server2/link_converter.py -o -f
/b/commit-queue/workdir/chromium/chrome/common/extensions/api/developer_private.idl
returned non-zero exit status 1 in
/b/commit-queue/workdir/chromium/chrome/common/extensions

[commit-bot: I haz the power, 2013-04-18 17:46:41]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/grv@chromium.org/13885007/16001

[commit-bot: I haz the power, 2013-04-18 17:46:44]

Presubmit check for 13885007-16001 failed and returned exit status 1.

INFO:root:Found 4 file(s).

Traceback (most recent call last):
  File "docs/server2/integration_test.py", line 6, in <module>
    from handler import Handler
  File
"/b/commit-queue/workdir/chromium/chrome/common/extensions/docs/server2/handler.py",
line 13, in <module>
    from server_instance import ServerInstance
  File
"/b/commit-queue/workdir/chromium/chrome/common/extensions/docs/server2/server_instance.py",
line 9, in <module>
    from api_data_source import APIDataSource
  File
"/b/commit-queue/workdir/chromium/chrome/common/extensions/docs/server2/api_data_source.py",
line 10, in <module>
    import third_party.json_schema_compiler.json_parse as json_parse
ImportError: No module named third_party.json_schema_compiler.json_parse
Traceback (most recent call last):
  File "docs/server2/link_converter.py", line 15, in <module>
    import third_party.json_schema_compiler.model as model
ImportError: No module named third_party.json_schema_compiler.model
Running presubmit commit checks ...
Running /b/commit-queue/workdir/chromium/PRESUBMIT.py
Running /b/commit-queue/workdir/chromium/chrome/PRESUBMIT.py
Running /b/commit-queue/workdir/chromium/chrome/browser/extensions/PRESUBMIT.py
Running /b/commit-queue/workdir/chromium/chrome/common/extensions/PRESUBMIT.py
Traceback (most recent call last):
  File "/b/commit-queue/verification/presubmit_shim.py", line 33, in <module>
    sys.exit(presubmit_support.Main(argv))
  File "/b/depot_tools/presubmit_support.py", line 1305, in Main
    rietveld_obj)
  File "/b/depot_tools/presubmit_support.py", line 1129, in DoPresubmitChecks
    results += executer.ExecPresubmitScript(presubmit_script, filename)
  File "/b/depot_tools/presubmit_support.py", line 1046, in ExecPresubmitScript
    result = eval(function_name + '(*__args)', context)
  File "<string>", line 1, in <module>
  File "<string>", line 173, in CheckChangeOnCommit
  File "<string>", line 166, in _CheckChange
  File "<string>", line 134, in _CheckLinks
  File "/b/depot_tools/subprocess2.py", line 449, in check_output
    return check_call_out(args, stdout=PIPE, **kwargs)[0]
  File "/b/depot_tools/subprocess2.py", line 412, in check_call_out
    returncode, args, kwargs.get('cwd'), out[0], out[1])
subprocess2.CalledProcessError: Command docs/server2/link_converter.py -o -f
/b/commit-queue/workdir/chromium/chrome/common/extensions/api/developer_private.idl
returned non-zero exit status 1 in
/b/commit-queue/workdir/chromium/chrome/common/extensions

[Gaurav, 2013-04-18 18:15:06]

Hi,

The PRESUBMIT checks seems to fail to unable to import modules.

 "third_party.json_schema_compiler.model
 third_party.json_schema_compiler.json_parse"

Running presubmit locally works fine.

Is it possible that the python paths are not correctly setup on buildbots ?

Thanks for the help,
- - Gaurav


On Thu, Apr 18, 2013 at 10:46 AM, <commit-bot@chromium.org> wrote:

> Presubmit check for 13885007-16001 failed and returned exit status 1.
>
>
> INFO:root:Found 4 file(s).
>
> Traceback (most recent call last):
>   File "docs/server2/integration_**test.py", line 6, in <module>
>     from handler import Handler
>   File
> "/b/commit-queue/workdir/**chromium/chrome/common/**
> extensions/docs/server2/**handler.py",
> line 13, in <module>
>     from server_instance import ServerInstance
>   File
> "/b/commit-queue/workdir/**chromium/chrome/common/**
> extensions/docs/server2/**server_instance.py",
> line 9, in <module>
>     from api_data_source import APIDataSource
>   File
> "/b/commit-queue/workdir/**chromium/chrome/common/**
> extensions/docs/server2/api_**data_source.py",
> line 10, in <module>
>     import third_party.json_schema_**compiler.json_parse as json_parse
> ImportError: No module named third_party.json_schema_**compiler.json_parse
> Traceback (most recent call last):
>   File "docs/server2/link_converter.**py", line 15, in <module>
>     import third_party.json_schema_**compiler.model as model
> ImportError: No module named third_party.json_schema_**compiler.model
> Running presubmit commit checks ...
> Running /b/commit-queue/workdir/**chromium/PRESUBMIT.py
> Running /b/commit-queue/workdir/**chromium/chrome/PRESUBMIT.py
> Running /b/commit-queue/workdir/**chromium/chrome/browser/**
> extensions/PRESUBMIT.py
> Running /b/commit-queue/workdir/**chromium/chrome/common/**
> extensions/PRESUBMIT.py
> Traceback (most recent call last):
>   File "/b/commit-queue/verification/**presubmit_shim.py", line 33, in
> <module>
>     sys.exit(presubmit_support.**Main(argv))
>   File "/b/depot_tools/presubmit_**support.py", line 1305, in Main
>     rietveld_obj)
>   File "/b/depot_tools/presubmit_**support.py", line 1129, in
> DoPresubmitChecks
>     results += executer.ExecPresubmitScript(**presubmit_script, filename)
>   File "/b/depot_tools/presubmit_**support.py", line 1046, in
> ExecPresubmitScript
>     result = eval(function_name + '(*__args)', context)
>   File "<string>", line 1, in <module>
>   File "<string>", line 173, in CheckChangeOnCommit
>   File "<string>", line 166, in _CheckChange
>   File "<string>", line 134, in _CheckLinks
>   File "/b/depot_tools/subprocess2.**py", line 449, in check_output
>     return check_call_out(args, stdout=PIPE, **kwargs)[0]
>   File "/b/depot_tools/subprocess2.**py", line 412, in check_call_out
>     returncode, args, kwargs.get('cwd'), out[0], out[1])
> subprocess2.**CalledProcessError: Command docs/server2/link_converter.py
> -o -f
> /b/commit-queue/workdir/**chromium/chrome/common/**
> extensions/api/developer_**private.idl
> returned non-zero exit status 1 in
> /b/commit-queue/workdir/**chromium/chrome/common/**extensions
>
>
>
https://chromiumcodereview.**appspot.com/13885007/<https://chromiumcodereview...
>

[commit-bot: I haz the power, 2013-04-18 18:42:00]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/grv@chromium.org/13885007/16001

[commit-bot: I haz the power, 2013-04-18 18:42:03]

Presubmit check for 13885007-16001 failed and returned exit status 1.

INFO:root:Found 4 file(s).

Traceback (most recent call last):
  File "docs/server2/integration_test.py", line 6, in <module>
    from handler import Handler
  File
"/b/commit-queue/workdir/chromium/chrome/common/extensions/docs/server2/handler.py",
line 13, in <module>
    from server_instance import ServerInstance
  File
"/b/commit-queue/workdir/chromium/chrome/common/extensions/docs/server2/server_instance.py",
line 9, in <module>
    from api_data_source import APIDataSource
  File
"/b/commit-queue/workdir/chromium/chrome/common/extensions/docs/server2/api_data_source.py",
line 10, in <module>
    import third_party.json_schema_compiler.json_parse as json_parse
ImportError: No module named third_party.json_schema_compiler.json_parse
Traceback (most recent call last):
  File "docs/server2/link_converter.py", line 15, in <module>
    import third_party.json_schema_compiler.model as model
ImportError: No module named third_party.json_schema_compiler.model
Running presubmit commit checks ...
Running /b/commit-queue/workdir/chromium/PRESUBMIT.py
Running /b/commit-queue/workdir/chromium/chrome/PRESUBMIT.py
Running /b/commit-queue/workdir/chromium/chrome/browser/extensions/PRESUBMIT.py
Running /b/commit-queue/workdir/chromium/chrome/common/extensions/PRESUBMIT.py
Traceback (most recent call last):
  File "/b/commit-queue/verification/presubmit_shim.py", line 33, in <module>
    sys.exit(presubmit_support.Main(argv))
  File "/b/depot_tools/presubmit_support.py", line 1305, in Main
    rietveld_obj)
  File "/b/depot_tools/presubmit_support.py", line 1129, in DoPresubmitChecks
    results += executer.ExecPresubmitScript(presubmit_script, filename)
  File "/b/depot_tools/presubmit_support.py", line 1046, in ExecPresubmitScript
    result = eval(function_name + '(*__args)', context)
  File "<string>", line 1, in <module>
  File "<string>", line 173, in CheckChangeOnCommit
  File "<string>", line 166, in _CheckChange
  File "<string>", line 134, in _CheckLinks
  File "/b/depot_tools/subprocess2.py", line 449, in check_output
    return check_call_out(args, stdout=PIPE, **kwargs)[0]
  File "/b/depot_tools/subprocess2.py", line 412, in check_call_out
    returncode, args, kwargs.get('cwd'), out[0], out[1])
subprocess2.CalledProcessError: Command docs/server2/link_converter.py -o -f
/b/commit-queue/workdir/chromium/chrome/common/extensions/api/developer_private.idl
returned non-zero exit status 1 in
/b/commit-queue/workdir/chromium/chrome/common/extensions

[commit-bot: I haz the power, 2013-04-18 23:22:26]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/grv@chromium.org/13885007/16001

[commit-bot: I haz the power, 2013-04-19 00:19:49]

Sorry for I got bad news for ya.
Compile failed with a clobber build on win.
http://build.chromium.org/p/tryserver.chromium/buildstatus?builder=win&number...
Your code is likely broken or HEAD is junk. Please ensure your
code is not broken then alert the build sheriffs.
Look at the try server FAQ for more details.

[commit-bot: I haz the power, 2013-04-19 01:51:07]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/grv@chromium.org/13885007/28003

[commit-bot: I haz the power, 2013-04-19 01:51:09]

Failed to apply patch for
chrome/browser/extensions/extension_function_histogram_value.h:
While running patch -p1 --forward --force --no-backup-if-mismatch;
  patching file chrome/browser/extensions/extension_function_histogram_value.h
  Hunk #1 FAILED at 503.
  1 out of 1 hunk FAILED -- saving rejects to file
chrome/browser/extensions/extension_function_histogram_value.h.rej

Patch:       chrome/browser/extensions/extension_function_histogram_value.h
Index: chrome/browser/extensions/extension_function_histogram_value.h
diff --git a/chrome/browser/extensions/extension_function_histogram_value.h
b/chrome/browser/extensions/extension_function_histogram_value.h
index
91c244ea00e12f28f7bf7e97d0ec6d891aee441b..175081c7acc736088ed6cdad686a824dc7b9c138
100644
--- a/chrome/browser/extensions/extension_function_histogram_value.h
+++ b/chrome/browser/extensions/extension_function_histogram_value.h
@@ -503,6 +503,9 @@ enum HistogramValue {
   APP_CURRENTWINDOWINTERNAL_FULLSCREEN,
   DEVELOPERPRIVATE_LOADUNPACKEDCROS,
   NETWORKINGPRIVATE_REQUESTNETWORKSCAN,
+  DEVELOPERPRIVATE_LOADPROJECTTOSYNCFS,
+  DEVELOPERPRIVATE_GETPROJECTSINFO,
+  DEVELOPERPRIVATE_LOADPROJECT,
   ENUM_BOUNDARY // Last entry: Add new entries above.
 };

[commit-bot: I haz the power, 2013-04-19 02:07:41]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/grv@chromium.org/13885007/23002

[commit-bot: I haz the power, 2013-04-19 09:00:23]

Change committed as 195116

[miket_OOO, 2013-04-23 17:02:47]

https://codereview.chromium.org/13885007/diff/1/chrome/browser/extensions/api...
File chrome/browser/extensions/api/developer_private/developer_private_api.cc
(right):

https://codereview.chromium.org/13885007/diff/1/chrome/browser/extensions/api...
chrome/browser/extensions/api/developer_private/developer_private_api.cc:936:
path = path.Append(project_name);
On 2013/04/18 17:24:06, Gaurav wrote:
> On 2013/04/18 03:12:00, miket wrote:
> > Please add a TODO to sanitize this value, and further to assert that the end
> > result is within kUnpackedAppsFolder. I know that it's a private API, but
it's
> > still taking user-generated, external values, and this would be the
> appropriate
> > place to assert that it's not a value like "../../../My Documents/taxes." 
> 
> Done.

This didn't happen. What's going on?

[Gaurav, 2013-04-23 18:08:33]

https://codereview.chromium.org/13885007/diff/1/chrome/browser/extensions/api...
File chrome/browser/extensions/api/developer_private/developer_private_api.cc
(right):

https://codereview.chromium.org/13885007/diff/1/chrome/browser/extensions/api...
chrome/browser/extensions/api/developer_private/developer_private_api.cc:936:
path = path.Append(project_name);
On 2013/04/23 17:02:47, miket wrote:
> On 2013/04/18 17:24:06, Gaurav wrote:
> > On 2013/04/18 03:12:00, miket wrote:
> > > Please add a TODO to sanitize this value, and further to assert that the
end
> > > result is within kUnpackedAppsFolder. I know that it's a private API, but
> it's
> > > still taking user-generated, external values, and this would be the
> > appropriate
> > > place to assert that it's not a value like "../../../My Documents/taxes." 
> > 
> > Done.
> 
> This didn't happen. What's going on?

Added the checks for folder name validation (was thinking that will ensure, that
the path will be within apps_target).

Will use the function as suggested by you in other diff.

Sorry about that.