Roll Skia DEPS to 4172 & add files to skia.gyp

Add two files + deps roll + 2 new expectations

control group: http://codereview.chromium.org/10541032/

Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=141202

[robertphillips, 2012-06-06 20:48:16]

win_layout
        odd svn failure (client too old to work 
        w/ working copy) on both
win_gpu
        odd svn failure on both
mac_asan
        compile failed on both
android
        both good
linux_layout_rel
        both failed on same four
linix_shared
        both good
linux_gpu
        roll failed on Canvas2DRedBoxSD, CSS3DBlueBox + 2 others
        control did not fail
mac_gpu
        both good
mac_layout_rel
        control failed on 3:
          float-containing-block-with-margin.html
          selection-gaps-at-fractional-offsets.html
          video-buffered-range-contains-currentTime.html
                                        
        roll failed on 5:
          ** 2d.gradient.radial.cone.cylinder.html
          ** canvas-currentColor.html
          float-containing-block-with-margin.html
          selection-gaps-at-fractional-offsets.html
          video-buffered-range-contains-currentTime.html
linux_heapcheck
        both: 39 disabled, crashed or hung
linux_layout
        control failed 6:
          optgroup-rendering.html
          float-containing-block-with-margin.html
          video-buffered-range-contains-currentTime.html
          ** stop-animation-on-suspend.html
          ** dispatch-message-string-data.html
          mouse-event.html
        
        roll failed 5 + 2 flaky
          optgroup-rendering.html
          float-containing-block-with-margin.html
          video-buffered-range-contains-currentTime.html
          ** loadInProgress.html
          mouse-event.html
        
          video-media-source-seek.html
          video-play-progress.html

win_layout_rel
        both failed same 2
        control had "unexpected flaky 18":
          multiple-captions-display.xhtml
          3d-corners.html
          backface-visibility-transformed.html
          child-layer-3d-sorting.html
          huge-layer-rotated.html
          img-layer-grow.html
          layout-width-change.html
          lost-compositor-context-permanently.html
          lost-compositor-context-twice.html
          lost-compositor-context-with-rendersurface.html
          ...and more
linux
        both good
mac
        both good
linux_asan
        both got "6 flaky failed 1" on browser_tests:
             control: OnChangedNotificationsBetweenBackgroundPages
             roll: VideoBearTheora
linux_chomeos_asan
        both good
mac_layout
        control failed 3 + 8 "unexpected flaky"
          float-containing-block-with-margin.html
          selection-gaps-at-fractional-offsets.html
          video-buffered-range-contains-currentTime.html

          overflow-scroll-overlap.html
          ** flex-hang.html
          ** video-played-ranges-1.html
          ** file-writer-write-overlapped.html
          ** select-set-length-with-mutation-remove.html
          terminate-during-sync-operation.html
          ** SVGFECompositeElement-dom-operator-attr.html
          move-by-line-001.html

        roll failed 5 + 5 "unexpected flaky"
          **2d.gradient.radial.cone.cylinder.html
          **canvas-currentColor.html
          float-containing-block-with-margin.html
          selection-gaps-at-fractional-offsets.html
          video-buffered-range-contains-currentTime.html

          overflow-scroll-overlap.html
          ** inline-plaintext-relayout-with-leading-neutrals.html
          terminate-during-sync-operation.html
          ** audio-data-url.html
          move-by-line-001.html
win
        both good
linux_chromeos
        still running
        both all green but seem to be taking a long 
        time on the browser tests

[robertphillips, 2012-06-06 21:11:46]

Final two bots finally came in:

linux_chromeos
        both got 7 flaky and failed one

        control: Audio
        roll: ArgumentValidation

[epoger, 2012-06-07 14:24:40]

Summary: the only failures of real concern are these failures on mac_layout and
mac_layout_rel:
2d.gradient.radial.cone.cylinder.html
canvas-currentColor.html

We should discuss those further.  But first, here's how I went through all your
trybot results.  AFAIK this triage strategy is not actually documented
anywhere... UNTIL NOW.


On 2012/06/06 20:48:16, robertphillips wrote:
> win_layout

I find it easier to validate the result summary if you list the various trybots
in some obvious order.  I certainly wish the codereview page did that when
displaying them. :-)  Below, I have rearranged them into alphabetical order.

For one thing, it is often handy to look at the *_layout results next to their
corresponding *_layout_rel results (often their results will be quite similar).

> android
>         both good
>
> linux
>         both good
> linux_asan
>         both got "6 flaky failed 1" on browser_tests:
>              control: OnChangedNotificationsBetweenBackgroundPages
>              roll: VideoBearTheora

First of all, you need to know that there are various meanings of "flaky".

Webkit layout tests (*_layout and *_layout_rel trybots) have their own concept
of "flaky", which I will describe later on.  In the NON-layout bots, "6 flaky"
means that 6 of the tests contain "FLAKY" in the name and thus failures of those
tests will be completely ignored.  This "FLAKY" designation has been manually
added by somebody; it's not done automatically.

Looking at the bottom of
http://build.chromium.org/p/tryserver.chromium/builders/linux_asan/builds/343...
, I see:
1722 tests run
2 tests failed (1 ignored)
Failing tests:
SUIDSandboxUITest.FLAKY_testSUIDSandboxEnabled (ignored)
MediaTest.VideoBearTheora

So failure of the test with "FLAKY" in the name has been ignored, leaving 1
"real" failure (and thus the "failed 1" part of the result summary).

As far as the MediaTest.VideoBearTheora failure goes, it's a judgment call.  You
could look at the test to see exactly what it does, but given its name and that
this is the only failure, it seems highly unlikely that your change would have
caused this particular test to fail.  So in this case I judge that test to be
flaky (it fails some of the time, for no apparent reason) and ignore it.  I
guess I could go in and add "FLAKY" to its name, and then everyone else would
ignore it too... I don't know what the pros and cons would be.

The fact that one other non-"FLAKY" test failed in the control group indicates
that yes, indeed, there are plenty of flaky tests out there that are not labeled
as such.

> linux_chromeos
>         still running
>         both all green but seem to be taking a long 
>         time on the browser tests
> linux_chomeos_asan
>         both good
> linux_gpu
>         roll failed on Canvas2DRedBoxSD, CSS3DBlueBox + 2 others
>         control did not fail

Those 4 failing tests are the "fab 4" Brian and I were talking about.  They are
flakes that tend to fail as a group.  You can ignore them.
GpuPixelBrowserTest.WebGLGreenTriangle
GpuPixelBrowserTest.CSS3DBlueBox
GpuPixelBrowserTest.Canvas2DRedBoxHD
Canvas2DPixelTestSD.Canvas2DRedBoxSD

> linux_heapcheck
>         both: 39 disabled, crashed or hung

I don't understand the linux_heapcheck trybot very well (we only recently
started running it).  If it had lots more failures in the roll than the control
I would pay attention to it; in this case, I ignore.

> linux_layout
>         control failed 6:
>           optgroup-rendering.html
>           float-containing-block-with-margin.html
>           video-buffered-range-contains-currentTime.html
>           ** stop-animation-on-suspend.html
>           ** dispatch-message-string-data.html
>           mouse-event.html
>         
>         roll failed 5 + 2 flaky
>           optgroup-rendering.html
>           float-containing-block-with-margin.html
>           video-buffered-range-contains-currentTime.html
>           ** loadInProgress.html
>           mouse-event.html

In terms of the tests that "failed": you can completely ignore those that failed
in both control and roll, or only in the control.  That leaves one new failure
in the roll, loadInProgress.html.  I saw this one in my trybots yesterday too
and asked Mike about it... for some reason, we often see this fail in our roll
trybots but not our control trybots; I don't understand why/how that can be.  At
any rate, we've committed plenty of changes that failed this test in the trybot
and it's never caused trouble.  Safe to ignore.

>         
>           video-media-source-seek.html
>           video-play-progress.html

In terms of the tests that are "flaky": you can completely ignore them.  But
it's important for you to know the two meanings of "flaky" in the world of
layout tests (*_layout and *_layout_rel trybots):

1. Tests that are reported as "flaky": The bots will automatically re-run any
failed tests right away.  If a test fails the second time too, the test is
reported as "failing".  If it succeeded the second time, it is reported as
"flaky".  

If you search for "video-media-source-seek" in the stdio, you will see that the
test failed the first time but passed the second time.  Thus it was reported as
"flaky".

You should ignore these, because everyone else does.  

2. Tests that are reported as "failing" but are really flaky, in the larger
sense: loadInProgress.html is a good example of this.  If you click on the names
of any of the failing tests in
http://build.chromium.org/p/tryserver.chromium/buildstatus?builder=linux_layo...
, you will be taken to the "flakiness dashboard".  This shows you the results of
a given layout test across all waterfall bots (NOT trybots!) over time.  This
helps you determine whether a test is "flaky" in the larger sense.

> linux_layout_rel
>         both failed on same four
> linix_shared
>         both good
> mac
>         both good
> mac_asan
>         compile failed on both

We have noticed that this is often the case on mac_asan... ignore.  (I guess it
would be a good idea to report this consistent failure.)

> mac_gpu
>         both good
> mac_layout
>         control failed 3 + 8 "unexpected flaky"
>           float-containing-block-with-margin.html
>           selection-gaps-at-fractional-offsets.html
>           video-buffered-range-contains-currentTime.html
> 
>           overflow-scroll-overlap.html
>           ** flex-hang.html
>           ** video-played-ranges-1.html
>           ** file-writer-write-overlapped.html
>           ** select-set-length-with-mutation-remove.html
>           terminate-during-sync-operation.html
>           ** SVGFECompositeElement-dom-operator-attr.html
>           move-by-line-001.html
> 
>         roll failed 5 + 5 "unexpected flaky"
>           **2d.gradient.radial.cone.cylinder.html
>           **canvas-currentColor.html
>           float-containing-block-with-margin.html
>           selection-gaps-at-fractional-offsets.html
>           video-buffered-range-contains-currentTime.html
> 
>           overflow-scroll-overlap.html
>           ** inline-plaintext-relayout-with-leading-neutrals.html
>           terminate-during-sync-operation.html
>           ** audio-data-url.html
>           move-by-line-001.html

Ignore all flakes.
The 2 new failures are cause for concern... let's look at mac_layout_rel and see
what it reported...

> mac_layout_rel
>         control failed on 3:
>           float-containing-block-with-margin.html
>           selection-gaps-at-fractional-offsets.html
>           video-buffered-range-contains-currentTime.html
>                                         
>         roll failed on 5:
>           ** 2d.gradient.radial.cone.cylinder.html
>           ** canvas-currentColor.html
>           float-containing-block-with-margin.html
>           selection-gaps-at-fractional-offsets.html
>           video-buffered-range-contains-currentTime.html

Same 2 new failures as mac_layout!  It looks to me like this is a real
regression (or at least, an actual output difference) caused by your CL.

> 
> win
>         both good
> win_gpu
>         odd svn failure on both
> win_layout
>         odd svn failure (client too old to work 
>         w/ working copy) on both

Given that win_layout_rel didn't have this problem, ignore.

> win_layout_rel
>         both failed same 2
>         control had "unexpected flaky 18":

Ignore all flakes.

[reed1, 2012-06-07 17:35:11]

These are going into webkit now (I hope). They need to be added to chrome's
text_expectations.txt as part of this CL.


BUGCR131187 :
platform/chromium/virtual/gpu/canvas/philip/tests/2d.gradient.radial.cone.cylinder.html
= TEXT
BUGCR131187 : platform/chromium/virtual/gpu/fast/canvas/canvas-currentColor.html
= TEXT
BUGCR131187 : canvas/philip/tests/2d.gradient.radial.cone.cylinder.html = TEXT
BUGCR131187 : fast/canvas/canvas-currentColor.html = TEXT

[robertphillips, 2012-06-07 18:17:48]

Now with expectations changes

[reed1, 2012-06-07 19:17:16]

lgtm

[commit-bot: I haz the power, 2012-06-08 10:31:09]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/robertphillips@google.com/10546027/12005

[commit-bot: I haz the power, 2012-06-08 11:53:16]

Commit queue rejected this change because the description was changed
between the time the change entered the commit queue and the time it
was ready to commit. You can safely check the commit box again.

[commit-bot: I haz the power, 2012-06-08 12:01:38]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/robertphillips@google.com/10546027/12005

[commit-bot: I haz the power, 2012-06-08 13:21:24]

Change committed as 141202