Add a MotionMark smoothness benchmark.

Add a MotionMark smoothness benchmark.

This benchmark includes all MotionMark test suites except
for the 22 tests in the "Basic canvas path suite".

BUG=619680

[vmiura, 2017-03-25 02:20:40]

The CQ bit was checked by vmiura@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2017-03-25 02:20:56]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[vmiura, 2017-03-25 02:38:32]

The CQ bit was checked by vmiura@chromium.org to run a CQ dry run

[commit-bot: I haz the power, 2017-03-25 02:38:50]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[vmiura, 2017-03-25 02:42:18]

The CQ bit was checked by vmiura@chromium.org to run a CQ dry run

[vmiura, 2017-03-25 02:42:30]

vmiura@chromium.org changed reviewers:
+ nednguyen@google.com

[vmiura, 2017-03-25 02:42:30]

nednguyen@ ptal

[commit-bot: I haz the power, 2017-03-25 02:42:32]

Dry run: CQ is trying da patch. Follow status at
 
https://chromium-cq-status.appspot.com/v2/patch-status/codereview.chromium.or...

[commit-bot: I haz the power, 2017-03-25 03:47:41]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2017-03-25 03:47:41]

Dry run: This issue passed the CQ dry run.

[nednguyen, 2017-03-27 13:29:43]

On 2017/03/25 02:42:30, vmiura wrote:
> nednguyen@ ptal

A couple of questions:
1) How long does this takes to run?
2) Are you plan to add other press benchmarks for smoothness testing?
3) Who should be owner of this benchmark?

[vmiura, 2017-03-27 18:27:06]

On 2017/03/27 13:29:43, nednguyen wrote:
> On 2017/03/25 02:42:30, vmiura wrote:
> > nednguyen@ ptal
> 
> A couple of questions:
> 1) How long does this takes to run?

It takes about 9 minutes to run.

With this page set we can remove some tests that would become redundant and save
~ 3 minutes.

tough_filters_cases.py (Focus test) 
tough_webgl_cases.py (WebGL test)
tough_canvas_cases.py
'file://tough_canvas_cases/rendering_throughput/canvas_arcs.html' 
tough_canvas_cases.py
'file://tough_canvas_cases/rendering_throughput/canvas_lines.html'
tough_canvas_cases.py
'file://tough_canvas_cases/rendering_throughput/put_get_image_data.html'
tough_canvas_cases.py
'file://tough_canvas_cases/rendering_throughput/fill_shapes.html'
tough_canvas_cases.py
'file://tough_canvas_cases/rendering_throughput/stroke_shapes.html'
tough_canvas_cases.py
'file://tough_canvas_cases/rendering_throughput/bouncing_clipped_rectangles.html'
tough_canvas_cases.py
'file://tough_canvas_cases/rendering_throughput/bouncing_gradient_circles.html'
tough_canvas_cases.py
'file://tough_canvas_cases/rendering_throughput/bouncing_svg_images.html'
tough_canvas_cases.py
'file://tough_canvas_cases/rendering_throughput/bouncing_png_images.html'

> 2) Are you plan to add other press benchmarks for smoothness testing?

Not at this time.

> 3) Who should be owner of this benchmark?

chrome-gpu

[vmiura, 2017-03-29 17:43:19]

I'm trying to land a change in WebKit to fix an issue with MotionMark and
web-page-replay https://bugs.webkit.org/show_bug.cgi?id=170198.  I'll re-roll
the wpr once that's solved.

Does the patch look OK?  Thanks.

[nednguyen, 2017-03-29 19:22:24]

On 2017/03/29 17:43:19, vmiura wrote:
> I'm trying to land a change in WebKit to fix an issue with MotionMark and
> web-page-replay https://bugs.webkit.org/show_bug.cgi?id=170198.  I'll re-roll
> the wpr once that's solved.
> 
> Does the patch look OK?  Thanks.

Hi Victor, there are a couple of things going on:
1) Our perf bots are overloaded, and we have a P1 bugs about a bunch of timed
out bots (crbug.com/691582)
2) We're in the middle of reconsidering how to reorganize our benchmarks so that
it's more easier for us to manage. One potential direction is probably we should
merge all the smoothness benchmarks into at most 2 benchmark: smoothness.desktop
& smoothness.mobile. This will make it a lot easier for sheriffs to deal with
benchmark issues, for us to track benchmarks' cost & for team to extend their
test suite.
3) Brian Anderson is planning to collaborate with Tim Dresser to modernize the
smoothness metrics using TBMv2

Due to those 3 different efforts, I think we should carefully consider the cost
of adding more smoothness benchmarks at the moment. Next week I will be in MTV,
can we sync up to discuss more about smoothness benchmarking space & gpu team's
overall benchmark needs?

[vmiura, 2017-03-31 17:31:42]

On 2017/03/29 19:22:24, nednguyen wrote:
> On 2017/03/29 17:43:19, vmiura wrote:
> > I'm trying to land a change in WebKit to fix an issue with MotionMark and
> > web-page-replay https://bugs.webkit.org/show_bug.cgi?id=170198.  I'll
re-roll
> > the wpr once that's solved.
> > 
> > Does the patch look OK?  Thanks.
> 
> Hi Victor, there are a couple of things going on:
> 1) Our perf bots are overloaded, and we have a P1 bugs about a bunch of timed
> out bots (crbug.com/691582)
> 2) We're in the middle of reconsidering how to reorganize our benchmarks so
that
> it's more easier for us to manage. One potential direction is probably we
should
> merge all the smoothness benchmarks into at most 2 benchmark:
smoothness.desktop
> & smoothness.mobile. This will make it a lot easier for sheriffs to deal with
> benchmark issues, for us to track benchmarks' cost & for team to extend their
> test suite.
> 3) Brian Anderson is planning to collaborate with Tim Dresser to modernize the
> smoothness metrics using TBMv2
> 
> Due to those 3 different efforts, I think we should carefully consider the
cost
> of adding more smoothness benchmarks at the moment. Next week I will be in
MTV,
> can we sync up to discuss more about smoothness benchmarking space & gpu
team's
> overall benchmark needs?

Sure thing, let's discuss.  I understand there's a lot to figure out with our
benchmarks right now.

Quick thoughts on the points you mentioned

1) Cycle time from new benchmarks and timeouts seem somewhat orthogonal.  I'm
curious to hear more.
3) We don't necessarily need to use smoothness here if that's a concern.  I
could use thread_times, or do a custom metric if needed.
2) This sounds like the crux, making our benchmarks manageable and owned.