[telemetry] Split blink_perf up on Android.

[telemetry] Split blink_perf up on Android.

With the current sharding strategy (random), having smaller suites will reduce cycle time.
The Android bots will automatically pick up this config change. If we want to do the desktop bots too, that'd require a master restart.

BUG=396001
TEST=None.
R=tonyg

Committed: https://crrev.com/b44e27dda6f994d18d5ad4113b6fd2bccecd3cc3
Cr-Commit-Position: refs/heads/master@{#294915}

[tonyg, 2014-09-12 01:09:10]

https://codereview.chromium.org/566773003/diff/1/tools/perf/benchmarks/blink_...
File tools/perf/benchmarks/blink_perf.py (right):

https://codereview.chromium.org/566773003/diff/1/tools/perf/benchmarks/blink_...
tools/perf/benchmarks/blink_perf.py:43: skipped_file = os.path.join(path,
'Skipped')
Does this still work?

https://codereview.chromium.org/566773003/diff/1/tools/perf/benchmarks/blink_...
tools/perf/benchmarks/blink_perf.py:96: @benchmark.Disabled('android')
Please add a TODO indicating that once the desktop bots are running on recipes
we should remove this benchmark and enable the below on all platforms.

https://codereview.chromium.org/566773003/diff/1/tools/perf/benchmarks/blink_...
tools/perf/benchmarks/blink_perf.py:125: 'third_party', 'WebKit',
'PerformanceTests', 'Bindings')
I'm on the fence about whether these dir names should be dynamic or static. Just
curious what the pros/cons are.

[dtu, 2014-09-12 03:18:52]

https://chromiumcodereview.appspot.com/566773003/diff/1/tools/perf/benchmarks...
File tools/perf/benchmarks/blink_perf.py (right):

https://chromiumcodereview.appspot.com/566773003/diff/1/tools/perf/benchmarks...
tools/perf/benchmarks/blink_perf.py:96: @benchmark.Disabled('android')
On 2014/09/12 01:09:09, tonyg wrote:
> Please add a TODO indicating that once the desktop bots are running on recipes
> we should remove this benchmark and enable the below on all platforms.

Done.

https://chromiumcodereview.appspot.com/566773003/diff/1/tools/perf/benchmarks...
tools/perf/benchmarks/blink_perf.py:125: 'third_party', 'WebKit',
'PerformanceTests', 'Bindings')
On 2014/09/12 01:09:09, tonyg wrote:
> I'm on the fence about whether these dir names should be dynamic or static.
Just
> curious what the pros/cons are.

Pros:
- Much shorter code. Easy to glance at list of tests.
- Adding or removing dirs will automatically add or remove tests.

Cons:
- More complex code, possibly involving inspect.
- Hard to @Disable individual dirs.
- I excluded some folders that will always be skipped. (Dromaeo, Replay,
SunSpider, XSSAuditor, inspector) We'd need a blacklist here to avoid extra test
steps on the waterfall.

[tonyg, 2014-09-12 14:12:50]

https://codereview.chromium.org/566773003/diff/1/tools/perf/benchmarks/blink_...
File tools/perf/benchmarks/blink_perf.py (right):

https://codereview.chromium.org/566773003/diff/1/tools/perf/benchmarks/blink_...
tools/perf/benchmarks/blink_perf.py:43: skipped_file = os.path.join(path,
'Skipped')
On 2014/09/12 01:09:09, tonyg wrote:
> Does this still work?

Ping on this. If the skipped file is still respected then this lgtm.

https://codereview.chromium.org/566773003/diff/1/tools/perf/benchmarks/blink_...
tools/perf/benchmarks/blink_perf.py:125: 'third_party', 'WebKit',
'PerformanceTests', 'Bindings')
On 2014/09/12 03:18:52, dtu wrote:
> On 2014/09/12 01:09:09, tonyg wrote:
> > I'm on the fence about whether these dir names should be dynamic or static.
> Just
> > curious what the pros/cons are.
> 
> Pros:
> - Much shorter code. Easy to glance at list of tests.
> - Adding or removing dirs will automatically add or remove tests.
> 
> Cons:
> - More complex code, possibly involving inspect.
> - Hard to @Disable individual dirs.
> - I excluded some folders that will always be skipped. (Dromaeo, Replay,
> SunSpider, XSSAuditor, inspector) We'd need a blacklist here to avoid extra
test
> steps on the waterfall.

Makes sense. I agree with your approach.

[dtu, 2014-09-15 21:01:13]

https://chromiumcodereview.appspot.com/566773003/diff/1/tools/perf/benchmarks...
File tools/perf/benchmarks/blink_perf.py (right):

https://chromiumcodereview.appspot.com/566773003/diff/1/tools/perf/benchmarks...
tools/perf/benchmarks/blink_perf.py:43: skipped_file = os.path.join(path,
'Skipped')
On 2014/09/12 14:12:50, tonyg wrote:
> On 2014/09/12 01:09:09, tonyg wrote:
> > Does this still work?
> 
> Ping on this. If the skipped file is still respected then this lgtm.

You're right, this logic is flawed. Done.

[dtu, 2014-09-15 21:01:40]

The CQ bit was checked by dtu@chromium.org

[commit-bot: I haz the power, 2014-09-15 21:03:36]

CQ is trying da patch. Follow status at
 https://chromium-cq-status.appspot.com/patchset/566773003/40001

[commit-bot: I haz the power, 2014-09-15 23:19:47]

Committed patchset #3 (id:40001) as cb25c5950aae8cdfeb5cf1d705cd1d807cf1dc4e

[tonyg, 2014-09-15 23:24:11]

Thanks! And please follow up w/ Quinten to adjust alerting rules.

On Mon, Sep 15, 2014 at 4:19 PM, <commit-bot@chromium.org> wrote:

> Committed patchset #3 (id:40001) as cb25c5950aae8cdfeb5cf1d705cd1d
> 807cf1dc4e
>
> https://codereview.chromium.org/566773003/
>

To unsubscribe from this group and stop receiving emails from it, send an email
to chromium-reviews+unsubscribe@chromium.org.

[commit-bot: I haz the power, 2014-09-15 23:30:36]

Patchset 3 (id:??) landed as
https://crrev.com/b44e27dda6f994d18d5ad4113b6fd2bccecd3cc3
Cr-Commit-Position: refs/heads/master@{#294915}