benchmark_health_report: Query anomalies by benchmark/master.

benchmark_health_report: Query anomalies by benchmark/master.

BUG=catapult:#3238

Review-Url: https://codereview.chromium.org/2707263008
Committed: https://chromium.googlesource.com/external/github.com/catapult-project/catapult/+/79c542a5fa5d1610ec2ccfba3c4f75cd07b4039a

[sullivan, 2017-02-23 03:12:35]

The CQ bit was checked by sullivan@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2017-02-23 03:12:41]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[sullivan, 2017-02-23 03:16:20]

The CQ bit was checked by sullivan@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2017-02-23 03:16:21]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-02-23 03:21:31]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-02-23 03:21:32]

Dry run: Try jobs failed on following builders:
  Catapult Linux Tryserver on master.tryserver.client.catapult (JOB_FAILED,
https://build.chromium.org/p/tryserver.client.catapult/builders/Catapult%20Li...)

[sullivan, 2017-02-23 03:27:00]

The CQ bit was checked by sullivan@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2017-02-23 03:27:11]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-02-23 03:32:21]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-02-23 03:32:24]

Dry run: Try jobs failed on following builders:
  Catapult Linux Tryserver on master.tryserver.client.catapult (JOB_FAILED,
https://build.chromium.org/p/tryserver.client.catapult/builders/Catapult%20Li...)

[sullivan, 2017-02-23 03:32:24]

sullivan@chromium.org changed reviewers:
+ eakuefner@chromium.org, simonhatch@chromium.org

[sullivan, 2017-02-23 03:32:25]

Now that I added benchmark_name and master_name properties and indexed them, the
query can be both simplified and sped up.

https://codereview.chromium.org/2707263008/diff/40001/dashboard/dashboard/ben...
File dashboard/dashboard/benchmark_health_report_test.py (right):

https://codereview.chromium.org/2707263008/diff/40001/dashboard/dashboard/ben...
dashboard/dashboard/benchmark_health_report_test.py:129: 'benchmark':
'speedometer',
The previous code was able to figure out that since the first monitored
page_cycler timeseries was monitored just for CrOS, it's not monitored on the
ChromiumPerf master. But this didn't work correctly in all cases, it just
checked the first set of monitored series. The new code is more consistent.

[sullivan, 2017-02-23 03:48:11]

Fixed the tests for real, ptal.

https://codereview.chromium.org/2707263008/diff/60001/dashboard/dashboard/ben...
File dashboard/dashboard/benchmark_health_report_test.py (right):

https://codereview.chromium.org/2707263008/diff/60001/dashboard/dashboard/ben...
dashboard/dashboard/benchmark_health_report_test.py:119: 'benchmark':
'sunspider',
Uh, not sure why this worked before, sunspider was monitored, even though it had
no alerts.

[sullivan, 2017-02-23 03:48:16]

The CQ bit was checked by sullivan@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2017-02-23 03:48:27]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-02-23 04:10:15]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-02-23 04:10:16]

Dry run: This issue passed the CQ dry run.

[sullivan, 2017-02-23 14:58:53]

The CQ bit was checked by sullivan@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2017-02-23 14:59:03]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-02-23 15:21:21]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-02-23 15:21:22]

Dry run: This issue passed the CQ dry run.

[shatch, 2017-02-23 15:58:30]

lgtm

https://codereview.chromium.org/2707263008/diff/60001/dashboard/dashboard/ben...
File dashboard/dashboard/benchmark_health_report_test.py (right):

https://codereview.chromium.org/2707263008/diff/60001/dashboard/dashboard/ben...
dashboard/dashboard/benchmark_health_report_test.py:119: 'benchmark':
'sunspider',
On 2017/02/23 03:48:11, sullivan wrote:
> Uh, not sure why this worked before, sunspider was monitored, even though it
had
> no alerts.

Looks like no sheriff?

https://codereview.chromium.org/2707263008/diff/80001/dashboard/dashboard/ben...
File dashboard/dashboard/benchmark_health_report.py (right):

https://codereview.chromium.org/2707263008/diff/80001/dashboard/dashboard/ben...
dashboard/dashboard/benchmark_health_report.py:62: if
benchmarks[benchmark].get('mon'):
nit: Is it possible to have alerts and not be monitored? Then we can just query
on monitored.

[sullivan, 2017-02-23 16:11:42]

https://codereview.chromium.org/2707263008/diff/80001/dashboard/dashboard/ben...
File dashboard/dashboard/benchmark_health_report.py (right):

https://codereview.chromium.org/2707263008/diff/80001/dashboard/dashboard/ben...
dashboard/dashboard/benchmark_health_report.py:62: if
benchmarks[benchmark].get('mon'):
On 2017/02/23 15:58:30, shatch wrote:
> nit: Is it possible to have alerts and not be monitored? Then we can just
query
> on monitored.

It is possible to have alerts and not be monitored.

The thing we are actually poking at here is the monitored property for all of
master/*/benchmark. Would you prefer to eliminate the dependency on cached test
suites by querying that directly?

[sullivan, 2017-02-23 16:27:23]

The CQ bit was checked by sullivan@chromium.org

[commit-bot: I haz the power, 2017-02-23 16:27:31]

CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[sullivan, 2017-02-23 16:27:39]

On 2017/02/23 16:11:42, sullivan wrote:
>
https://codereview.chromium.org/2707263008/diff/80001/dashboard/dashboard/ben...
> File dashboard/dashboard/benchmark_health_report.py (right):
> 
>
https://codereview.chromium.org/2707263008/diff/80001/dashboard/dashboard/ben...
> dashboard/dashboard/benchmark_health_report.py:62: if
> benchmarks[benchmark].get('mon'):
> On 2017/02/23 15:58:30, shatch wrote:
> > nit: Is it possible to have alerts and not be monitored? Then we can just
> query
> > on monitored.
> 
> It is possible to have alerts and not be monitored.
> 
> The thing we are actually poking at here is the monitored property for all of
> master/*/benchmark. Would you prefer to eliminate the dependency on cached
test
> suites by querying that directly?

We talked offline: I will do this as a follow-up.

[commit-bot: I haz the power, 2017-02-23 16:29:06]

CQ is committing da patch.

Bot data: {"patchset_id": 80001, "attempt_start_ts": 1487867243184330,
"parent_rev": "29f431a7e0c86d01a5dde1f6ebe40f1728f768e6", "commit_rev":
"79c542a5fa5d1610ec2ccfba3c4f75cd07b4039a"}

[commit-bot: I haz the power, 2017-02-23 16:29:09]

Description was changed from

==========
benchmark_health_report: Query anomalies by benchmark/master.

BUG=catapult:#3238
==========

to

==========
benchmark_health_report: Query anomalies by benchmark/master.

BUG=catapult:#3238

Review-Url: https://codereview.chromium.org/2707263008
Committed:
https://chromium.googlesource.com/external/github.com/catapult-project/catapu...
==========

[commit-bot: I haz the power, 2017-02-23 16:29:10]

Committed patchset #5 (id:80001) as
https://chromium.googlesource.com/external/github.com/catapult-project/catapu...