[test] Correctly deduce test-driver arguments on bots using GN

[test] Correctly deduce test-driver arguments on bots using GN

This makes auto-detect-build-configs work on bots (i.e.
if --buildbot was passed).

BUG=v8:5533
CQ_INCLUDE_TRYBOTS=master.tryserver.v8:v8_linux64_msan_rel,v8_linux64_tsan_rel,v8_linux_arm64_dbg,v8_linux_gc_stress_dbg,v8_linux_noi18n_rel_ng,v8_linux_nosnap_rel

Committed: https://crrev.com/9acec9f0a5923a72bbb32783d75a8cc97fb9027f
Cr-Commit-Position: refs/heads/master@{#40424}

[Michael Achenbach, 2016-10-18 12:49:12]

The CQ bit was checked by machenbach@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-10-18 12:49:24]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-10-18 12:52:52]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-10-18 12:52:53]

Dry run: Try jobs failed on following builders:
  v8_linux_dbg_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_dbg_ng/builds/14813)
  v8_linux_dbg_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_dbg_ng_triggered/b...)

[Michael Achenbach, 2016-10-18 13:20:40]

The CQ bit was checked by machenbach@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-10-18 13:20:47]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-10-18 13:24:36]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-10-18 13:24:37]

Dry run: Try jobs failed on following builders:
  v8_linux64_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux64_rel_ng/builds/14802)
  v8_linux64_rel_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux64_rel_ng_triggered...)
  v8_linux_dbg_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_dbg_ng/builds/14818)
  v8_linux_dbg_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_dbg_ng_triggered/b...)
  v8_linux_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_rel_ng/builds/14760)
  v8_linux_rel_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_linux_rel_ng_triggered/b...)

[Michael Achenbach, 2016-10-18 13:32:11]

The CQ bit was checked by machenbach@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-10-18 13:32:20]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[Michael Achenbach, 2016-10-18 13:43:08]

Description was changed from

==========
[test] Correctly deduce test-driver arguments on bots using GN

BUG=
==========

to

==========
[test] Correctly deduce test-driver arguments on bots using GN

This makes auto-detect-build-configs work on bots (i.e.
if --buildbot was passed).

BUG=v8:5533
==========

[commit-bot: I haz the power, 2016-10-18 13:53:49]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-10-18 13:53:50]

Dry run: Try jobs failed on following builders:
  v8_win64_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_win64_rel_ng/builds/16322)
  v8_win64_rel_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_win64_rel_ng_triggered/b...)

[Michael Achenbach, 2016-10-18 14:05:36]

machenbach@chromium.org changed reviewers:
+ jkummerow@chromium.org, jochen@chromium.org, yangguo@chromium.org

[Michael Achenbach, 2016-10-18 14:05:37]

PTAL. This is still a band-aid and not a proper fix to the associated bug. The
proper fix will require a bit redesign.

Looks like the inspector tests broke in the mean time on windows. They should
get fixed before this can land.

https://codereview.chromium.org/2431653002/diff/40001/tools/run-tests.py
File tools/run-tests.py (right):

https://codereview.chromium.org/2431653002/diff/40001/tools/run-tests.py#newc...
tools/run-tests.py:457: BASE_DIR, options.outdir, options.mode,
"v8_build_config.json")
It is a convention on buildbot that the build products are in outdir + mode,
e.g. out/Release.

https://codereview.chromium.org/2431653002/diff/40001/tools/run-tests.py#newc...
tools/run-tests.py:727: elif options.buildbot:
This remains for gyp bots that don't support auto-detect.

[Jakob Kummerow, 2016-10-18 16:45:58]

LGTM if the bot failures are unrelated.

[Michael Achenbach, 2016-10-18 20:26:03]

On 2016/10/18 16:45:58, Jakob Kummerow wrote:
> LGTM if the bot failures are unrelated.

The bot failures are expected in the sense that this CL fixes the test runner
and makes the inspector tests run (as they should).

They should be fixed in a separate CL before landing this. CC'ed dgozman and
kozyatinskiy for that task.

[kozy, 2016-10-18 20:27:26]

On 2016/10/18 20:26:03, machenbach (slow) wrote:
> On 2016/10/18 16:45:58, Jakob Kummerow wrote:
> > LGTM if the bot failures are unrelated.
> 
> The bot failures are expected in the sense that this CL fixes the test runner
> and makes the inspector tests run (as they should).
> 
> They should be fixed in a separate CL before landing this. CC'ed dgozman and
> kozyatinskiy for that task.

I'll fix it - Windows doesn't like call to exit(0) on non main thread.

[Michael Achenbach, 2016-10-18 20:32:07]

On 2016/10/18 20:27:26, kozyatinskiy wrote:
> On 2016/10/18 20:26:03, machenbach (slow) wrote:
> > On 2016/10/18 16:45:58, Jakob Kummerow wrote:
> > > LGTM if the bot failures are unrelated.
> > 
> > The bot failures are expected in the sense that this CL fixes the test
runner
> > and makes the inspector tests run (as they should).
> > 
> > They should be fixed in a separate CL before landing this. CC'ed dgozman and
> > kozyatinskiy for that task.
> 
> I'll fix it - Windows doesn't like call to exit(0) on non main thread.

Thanks, please schedule a dry run of this CL after your fix lands (or land it if
you monitor the waterfall afterwards for other possibly related failures).

[Michael Achenbach, 2016-10-18 20:34:59]

Description was changed from

==========
[test] Correctly deduce test-driver arguments on bots using GN

This makes auto-detect-build-configs work on bots (i.e.
if --buildbot was passed).

BUG=v8:5533
==========

to

==========
[test] Correctly deduce test-driver arguments on bots using GN

This makes auto-detect-build-configs work on bots (i.e.
if --buildbot was passed).

BUG=v8:5533
CQ_INCLUDE_TRYBOTS=master.tryserver.v8:v8_linux64_msan_rel,v8_linux64_tsan_rel,v8_linux_arm64_dbg,v8_linux_gc_stress_dbg,v8_linux_noi18n_rel_ng,v8_linux_nosnap_rel
==========

[Michael Achenbach, 2016-10-18 20:35:16]

The CQ bit was checked by machenbach@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-10-18 20:35:24]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[Michael Achenbach, 2016-10-18 20:36:31]

On 2016/10/18 20:32:07, machenbach (slow) wrote:
> On 2016/10/18 20:27:26, kozyatinskiy wrote:
> > On 2016/10/18 20:26:03, machenbach (slow) wrote:
> > > On 2016/10/18 16:45:58, Jakob Kummerow wrote:
> > > > LGTM if the bot failures are unrelated.
> > > 
> > > The bot failures are expected in the sense that this CL fixes the test
> runner
> > > and makes the inspector tests run (as they should).
> > > 
> > > They should be fixed in a separate CL before landing this. CC'ed dgozman
and
> > > kozyatinskiy for that task.
> > 
> > I'll fix it - Windows doesn't like call to exit(0) on non main thread.
> 
> Thanks, please schedule a dry run of this CL after your fix lands (or land it
if
> you monitor the waterfall afterwards for other possibly related failures).

I've also added a bunch of extra trybots to catch more possible problems early.

[commit-bot: I haz the power, 2016-10-18 20:59:54]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-10-18 20:59:55]

Dry run: Try jobs failed on following builders:
  v8_win_nosnap_shared_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_win_nosnap_shared_rel_ng...)
  v8_win_nosnap_shared_rel_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_win_nosnap_shared_rel_ng...)

[kozy, 2016-10-19 02:30:07]

The CQ bit was checked by kozyatinskiy@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-10-19 02:30:18]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-10-19 02:55:57]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-10-19 02:55:58]

Dry run: Try jobs failed on following builders:
  v8_win64_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_win64_rel_ng/builds/16351)
  v8_win64_rel_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_win64_rel_ng_triggered/b...)

[Michael Achenbach, 2016-10-19 06:33:05]

The CQ bit was checked by machenbach@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-10-19 06:33:11]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-10-19 06:57:59]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2016-10-19 06:58:00]

Dry run: Try jobs failed on following builders:
  v8_win_nosnap_shared_rel_ng on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_win_nosnap_shared_rel_ng...)
  v8_win_nosnap_shared_rel_ng_triggered on master.tryserver.v8 (JOB_FAILED,
http://build.chromium.org/p/tryserver.v8/builders/v8_win_nosnap_shared_rel_ng...)

[Michael Achenbach, 2016-10-19 07:01:34]

On 2016/10/18 20:36:31, machenbach (slow) wrote:
> On 2016/10/18 20:32:07, machenbach (slow) wrote:
> > On 2016/10/18 20:27:26, kozyatinskiy wrote:
> > > On 2016/10/18 20:26:03, machenbach (slow) wrote:
> > > > On 2016/10/18 16:45:58, Jakob Kummerow wrote:
> > > > > LGTM if the bot failures are unrelated.
> > > > 
> > > > The bot failures are expected in the sense that this CL fixes the test
> > runner
> > > > and makes the inspector tests run (as they should).
> > > > 
> > > > They should be fixed in a separate CL before landing this. CC'ed dgozman
> and
> > > > kozyatinskiy for that task.
> > > 
> > > I'll fix it - Windows doesn't like call to exit(0) on non main thread.
> > 
> > Thanks, please schedule a dry run of this CL after your fix lands (or land
it
> if
> > you monitor the waterfall afterwards for other possibly related failures).
> 
> I've also added a bunch of extra trybots to catch more possible problems
early.

I saw a fix CL, but the bots are still failing.

I'll use another strategy and revert the CL that uses inspector by default (as
this is the real blocker here). The failures will then show up on the reland of
that CL.

[Michael Achenbach, 2016-10-19 08:57:58]

The CQ bit was checked by machenbach@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2016-10-19 08:58:01]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[Michael Achenbach, 2016-10-19 09:25:33]

The CQ bit was unchecked by machenbach@chromium.org

[Michael Achenbach, 2016-10-19 09:25:39]

The CQ bit was checked by machenbach@chromium.org

[commit-bot: I haz the power, 2016-10-19 09:25:53]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2016-10-19 10:23:33]

Description was changed from

==========
[test] Correctly deduce test-driver arguments on bots using GN

This makes auto-detect-build-configs work on bots (i.e.
if --buildbot was passed).

BUG=v8:5533
CQ_INCLUDE_TRYBOTS=master.tryserver.v8:v8_linux64_msan_rel,v8_linux64_tsan_rel,v8_linux_arm64_dbg,v8_linux_gc_stress_dbg,v8_linux_noi18n_rel_ng,v8_linux_nosnap_rel
==========

to

==========
[test] Correctly deduce test-driver arguments on bots using GN

This makes auto-detect-build-configs work on bots (i.e.
if --buildbot was passed).

BUG=v8:5533
CQ_INCLUDE_TRYBOTS=master.tryserver.v8:v8_linux64_msan_rel,v8_linux64_tsan_rel,v8_linux_arm64_dbg,v8_linux_gc_stress_dbg,v8_linux_noi18n_rel_ng,v8_linux_nosnap_rel
==========

[commit-bot: I haz the power, 2016-10-19 10:23:34]

Committed patchset #3 (id:40001)

[commit-bot: I haz the power, 2016-11-17 22:05:53]

Description was changed from

==========
[test] Correctly deduce test-driver arguments on bots using GN

This makes auto-detect-build-configs work on bots (i.e.
if --buildbot was passed).

BUG=v8:5533
CQ_INCLUDE_TRYBOTS=master.tryserver.v8:v8_linux64_msan_rel,v8_linux64_tsan_rel,v8_linux_arm64_dbg,v8_linux_gc_stress_dbg,v8_linux_noi18n_rel_ng,v8_linux_nosnap_rel
==========

to

==========
[test] Correctly deduce test-driver arguments on bots using GN

This makes auto-detect-build-configs work on bots (i.e.
if --buildbot was passed).

BUG=v8:5533
CQ_INCLUDE_TRYBOTS=master.tryserver.v8:v8_linux64_msan_rel,v8_linux64_tsan_rel,v8_linux_arm64_dbg,v8_linux_gc_stress_dbg,v8_linux_noi18n_rel_ng,v8_linux_nosnap_rel

Committed: https://crrev.com/9acec9f0a5923a72bbb32783d75a8cc97fb9027f
Cr-Commit-Position: refs/heads/master@{#40424}
==========

[commit-bot: I haz the power, 2016-11-17 22:05:55]

Patchset 3 (id:??) landed as
https://crrev.com/9acec9f0a5923a72bbb32783d75a8cc97fb9027f
Cr-Commit-Position: refs/heads/master@{#40424}