DevTools: add "npm test" to run tests by fetching content shells

DevTools: add "npm test" to run tests by fetching content shells

Running "npm test" from /devtools fetches an uploaded content shell
(which is cached) and runs devtools layout tests.

This allows developers, especially external contributors, to run our tests
without compiling.

API:
* "npm test" - builds devtools and run all inspector layout tests (primary command)
* "npm test -- -f --child-processes=16" - pass in additional flags to the test harness
* "npm test -- inspector/sources inspector/console" - run specific tests
* "npm test -- inspector/cookie-resource-match.html --debug-devtools" - debug a specific test (non-bundled & minified)
* "npm run test:build" - only builds devtools (in debug mode)
* "npm run test:run" - only runs devtools tests

This currently works w/ a full-checkout, but I plan on working
to enable running devtools layout tests w/ only a frontend checkout.

BUG=654486
Committed: https://crrev.com/605fd97ead89705d29fa86c7b9885db159a35f8a
Cr-Commit-Position: refs/heads/master@{#426024}

[chenwilliam, 2016-10-14 17:44:29]

Description was changed from

==========
DevTools: add "npm test" to run tests by fetching content shells

Running "npm test" from /devtools fetches an uploaded content shell
(which is cached) and runs devtools layout tests.

This allows developers, especially external contributors, to run our tests
without compiling.

BUG=654486
==========

to

==========
DevTools: add "npm test" to run tests by fetching content shells

Running "npm test" from /devtools fetches an uploaded content shell
(which is cached) and runs devtools layout tests.

This allows developers, especially external contributors, to run our tests
without compiling.

API:
* "npm test" - run all inspector layout tests
* "npm test -- -f --child-processes=16" - pass in additional flags to the test
harness
* "npm test -- inspector/sources inspector/console" - run specific tests
* "npm test -- inspector/cookie-resource-match.html --debug-devtools" - debug a
specific test (non-bundled & minified)

This currently works w/ a full-checkout, but I plan on working
to enable running devtools layout tests w/ only a frontend checkout.

BUG=654486
==========

[chenwilliam, 2016-10-14 20:08:26]

chenwilliam@chromium.org changed reviewers:
+ dgozman@chromium.org

[chenwilliam, 2016-10-14 20:08:26]

PTAL. I tested it on Linux, Mac, and Windows and it seems to work OK.

[dgozman, 2016-10-14 21:40:31]

Let's have separate "npm build" and "npm runtests", with a combined "npm test"
which does both.

https://codereview.chromium.org/2413563002/diff/100001/third_party/WebKit/Sou...
File third_party/WebKit/Source/devtools/scripts/run_tests_without_compile.js
(right):

https://codereview.chromium.org/2413563002/diff/100001/third_party/WebKit/Sou...
third_party/WebKit/Source/devtools/scripts/run_tests_without_compile.js:304:
"inspector",
We can use inspector*, http/tests/inspector* :-)

[chenwilliam, 2016-10-17 18:55:46]

Description was changed from

==========
DevTools: add "npm test" to run tests by fetching content shells

Running "npm test" from /devtools fetches an uploaded content shell
(which is cached) and runs devtools layout tests.

This allows developers, especially external contributors, to run our tests
without compiling.

API:
* "npm test" - run all inspector layout tests
* "npm test -- -f --child-processes=16" - pass in additional flags to the test
harness
* "npm test -- inspector/sources inspector/console" - run specific tests
* "npm test -- inspector/cookie-resource-match.html --debug-devtools" - debug a
specific test (non-bundled & minified)

This currently works w/ a full-checkout, but I plan on working
to enable running devtools layout tests w/ only a frontend checkout.

BUG=654486
==========

to

==========
DevTools: add "npm test" to run tests by fetching content shells

Running "npm test" from /devtools fetches an uploaded content shell
(which is cached) and runs devtools layout tests.

This allows developers, especially external contributors, to run our tests
without compiling.

API:
* "npm test" - builds devtools and run all inspector layout tests (primary
command)
* "npm test -- -f --child-processes=16" - pass in additional flags to the test
harness
* "npm test -- inspector/sources inspector/console" - run specific tests
* "npm test -- inspector/cookie-resource-match.html --debug-devtools" - debug a
specific test (non-bundled & minified)
* "npm run test:build" - only builds devtools (in debug mode)
* "npm run test:run" - only runs devtools tests

This currently works w/ a full-checkout, but I plan on working
to enable running devtools layout tests w/ only a frontend checkout.

BUG=654486
==========

[chenwilliam, 2016-10-17 19:01:28]

PTAL. Added commands to allow users to separately build devtools and run tests
(updated description).

- npm run test:build
- npm run test:run

I also took out the release mode code to simplify this because we can just use
the generated files packed with the content shell (i.e.
InspectorBackendCommands.js)

https://codereview.chromium.org/2413563002/diff/100001/third_party/WebKit/Sou...
File third_party/WebKit/Source/devtools/scripts/run_tests_without_compile.js
(right):

https://codereview.chromium.org/2413563002/diff/100001/third_party/WebKit/Sou...
third_party/WebKit/Source/devtools/scripts/run_tests_without_compile.js:304:
"inspector",
On 2016/10/14 21:40:31, dgozman wrote:
> We can use inspector*, http/tests/inspector* :-)

Done.

[dgozman, 2016-10-17 19:13:51]

lgtm

https://codereview.chromium.org/2413563002/diff/140001/third_party/WebKit/Sou...
File third_party/WebKit/Source/devtools/scripts/npm_test.js (right):

https://codereview.chromium.org/2413563002/diff/140001/third_party/WebKit/Sou...
third_party/WebKit/Source/devtools/scripts/npm_test.js:107: var positionsListURL
=
`http://commondatastorage.googleapis.com/chromium-browser-snapshots/?delimiter=/&prefix=${PLATFORM}/&marker=${PLATFORM}/${previousPosition}/`;
Does one need any permissions for this url?

https://codereview.chromium.org/2413563002/diff/140001/third_party/WebKit/Sou...
File third_party/WebKit/Source/devtools/scripts/unzip.py (right):

https://codereview.chromium.org/2413563002/diff/140001/third_party/WebKit/Sou...
third_party/WebKit/Source/devtools/scripts/unzip.py:1: #!/usr/bin/env python
Is there a node.js equivalent?

[chenwilliam, 2016-10-18 18:13:18]

Responses to the questions.

https://chromiumcodereview.appspot.com/2413563002/diff/140001/third_party/Web...
File third_party/WebKit/Source/devtools/scripts/npm_test.js (right):

https://chromiumcodereview.appspot.com/2413563002/diff/140001/third_party/Web...
third_party/WebKit/Source/devtools/scripts/npm_test.js:107: var positionsListURL
=
`http://commondatastorage.googleapis.com/chromium-browser-snapshots/?delimiter=/&prefix=${PLATFORM}/&marker=${PLATFORM}/${previousPosition}/`;
On 2016/10/17 19:13:51, dgozman wrote:
> Does one need any permissions for this url?

Nope, I was able to download it at home w/ my personal computer.

https://chromiumcodereview.appspot.com/2413563002/diff/140001/third_party/Web...
File third_party/WebKit/Source/devtools/scripts/unzip.py (right):

https://chromiumcodereview.appspot.com/2413563002/diff/140001/third_party/Web...
third_party/WebKit/Source/devtools/scripts/unzip.py:1: #!/usr/bin/env python
On 2016/10/17 19:13:51, dgozman wrote:
> Is there a node.js equivalent?

From what I researched, there's no built-in node.js module that handles
unzipping a .zip file. While we could use a third-party npm package, we can
assume Python exists for users running tests since they need it for the layout
test harness. If we do pure JS/node.js unit tests that don't require Python, we
can try eliminating this use of Python.

[chenwilliam, 2016-10-18 18:14:28]

The CQ bit was checked by chenwilliam@chromium.org

[commit-bot: I haz the power, 2016-10-18 18:47:14]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-10-18 19:37:03]

Description was changed from

==========
DevTools: add "npm test" to run tests by fetching content shells

Running "npm test" from /devtools fetches an uploaded content shell
(which is cached) and runs devtools layout tests.

This allows developers, especially external contributors, to run our tests
without compiling.

API:
* "npm test" - builds devtools and run all inspector layout tests (primary
command)
* "npm test -- -f --child-processes=16" - pass in additional flags to the test
harness
* "npm test -- inspector/sources inspector/console" - run specific tests
* "npm test -- inspector/cookie-resource-match.html --debug-devtools" - debug a
specific test (non-bundled & minified)
* "npm run test:build" - only builds devtools (in debug mode)
* "npm run test:run" - only runs devtools tests

This currently works w/ a full-checkout, but I plan on working
to enable running devtools layout tests w/ only a frontend checkout.

BUG=654486
==========

to

==========
DevTools: add "npm test" to run tests by fetching content shells

Running "npm test" from /devtools fetches an uploaded content shell
(which is cached) and runs devtools layout tests.

This allows developers, especially external contributors, to run our tests
without compiling.

API:
* "npm test" - builds devtools and run all inspector layout tests (primary
command)
* "npm test -- -f --child-processes=16" - pass in additional flags to the test
harness
* "npm test -- inspector/sources inspector/console" - run specific tests
* "npm test -- inspector/cookie-resource-match.html --debug-devtools" - debug a
specific test (non-bundled & minified)
* "npm run test:build" - only builds devtools (in debug mode)
* "npm run test:run" - only runs devtools tests

This currently works w/ a full-checkout, but I plan on working
to enable running devtools layout tests w/ only a frontend checkout.

BUG=654486
==========

[commit-bot: I haz the power, 2016-10-18 19:37:04]

Committed patchset #8 (id:140001)

[commit-bot: I haz the power, 2016-10-21 13:02:20]

Description was changed from

==========
DevTools: add "npm test" to run tests by fetching content shells

Running "npm test" from /devtools fetches an uploaded content shell
(which is cached) and runs devtools layout tests.

This allows developers, especially external contributors, to run our tests
without compiling.

API:
* "npm test" - builds devtools and run all inspector layout tests (primary
command)
* "npm test -- -f --child-processes=16" - pass in additional flags to the test
harness
* "npm test -- inspector/sources inspector/console" - run specific tests
* "npm test -- inspector/cookie-resource-match.html --debug-devtools" - debug a
specific test (non-bundled & minified)
* "npm run test:build" - only builds devtools (in debug mode)
* "npm run test:run" - only runs devtools tests

This currently works w/ a full-checkout, but I plan on working
to enable running devtools layout tests w/ only a frontend checkout.

BUG=654486
==========

to

==========
DevTools: add "npm test" to run tests by fetching content shells

Running "npm test" from /devtools fetches an uploaded content shell
(which is cached) and runs devtools layout tests.

This allows developers, especially external contributors, to run our tests
without compiling.

API:
* "npm test" - builds devtools and run all inspector layout tests (primary
command)
* "npm test -- -f --child-processes=16" - pass in additional flags to the test
harness
* "npm test -- inspector/sources inspector/console" - run specific tests
* "npm test -- inspector/cookie-resource-match.html --debug-devtools" - debug a
specific test (non-bundled & minified)
* "npm run test:build" - only builds devtools (in debug mode)
* "npm run test:run" - only runs devtools tests

This currently works w/ a full-checkout, but I plan on working
to enable running devtools layout tests w/ only a frontend checkout.

BUG=654486

Committed: https://crrev.com/605fd97ead89705d29fa86c7b9885db159a35f8a
Cr-Commit-Position: refs/heads/master@{#426024}
==========

[commit-bot: I haz the power, 2016-10-21 13:02:20]

Patchset 8 (id:??) landed as
https://crrev.com/605fd97ead89705d29fa86c7b9885db159a35f8a
Cr-Commit-Position: refs/heads/master@{#426024}