Implement --browser=<perf trybot>

Implement --browser=<perf trybot>

BUG=401180
Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=291246

[sullivan, 2014-08-12 19:27:38]

This isn't ready for full review yet. I still need to write tests. But first I
wanted to do a first pass review to make sure the general approach is okay, and
to see if anyone knows the answers to the questions i had below.

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
File tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py
(right):

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:51: #
TODO(sullivan): figure out if there are any options trybots don't support
Can anyone think of options to check for?

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:81:
subprocess.check_output(['git', 'checkout', '-b', 'telemetry-tryjob'])
This prints out "Switched to branch 'telemetry-tryjob'". Anyone know if we want
to suppress that and how?

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:135: #
user? logging.info is hidden by default.
What's the best way to print non-error output?

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:168: #
TODO(sullivan): create right platform for target_os?
Should the platform here be for the target_os (trybot)? Or for the platform the
user is running on?

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:186: #
TODO(sullivan): cache results!
Where is a good place to put a cached copy of this data?

[tonyg, 2014-08-12 20:10:46]

This is awesome!

Just some ntis.

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
File tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py
(right):

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:17:
CONFIG = """config = {
Let's deal with this as a dict rather than a string. And just use
write('config=%s' % json.dumps(...)) at the right point.

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:34:
CONFIG_FILENAME = 'tools/run-bisect-perf-regression.cfg'
There's also a third_party/WebKit/Tools/run-perf-test.cfg that we should edit in
the same way if there're blink changes in the patch. That could be deferred for
a follow-up patch.

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:51: #
TODO(sullivan): figure out if there are any options trybots don't support
On 2014/08/12 19:27:37, sullivan wrote:
> Can anyone think of options to check for?

--chrome-root
--device
--remote
--identity
--interactive
--profile-dir

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:65:
['git', 'rev-parse', '--abbrev-ref', 'HEAD']).strip()
Would be nice to make a friendly error message if they're not in a git repo.

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:71:
except subprocess.CalledProcessError:
Can we check the exit status rather than for an exception?

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:73: if
dirty:
We should probably also check the opposite: that they do have a local commit.
Just to avoid confusion about how this should be used.

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:81:
subprocess.check_output(['git', 'checkout', '-b', 'telemetry-tryjob'])
On 2014/08/12 19:27:38, sullivan wrote:
> This prints out "Switched to branch 'telemetry-tryjob'". Anyone know if we
want
> to suppress that and how?

Let's suppress. There are some examples here:
https://code.google.com/p/chromium/codesearch#search/&q=telemetry%20stdout=de...

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:89: if
self._target_os == 'win':
We should double check, but I *think* the trybots know how to convert the
command for windows. if so, we should just rely on that.

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:109:
config_file.close()
Let's replace all close()s with with-blocks.

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:110:
contents = contents.replace(REPLACE_CONFIG, config)
Rather than read the old one and replace something, we should be able to just
stomp on the whole thing.

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:135: #
user? logging.info is hidden by default.
On 2014/08/12 19:27:38, sullivan wrote:
> What's the best way to print non-error output?

I think we usually simply `print`.

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:168: #
TODO(sullivan): create right platform for target_os?
On 2014/08/12 19:27:38, sullivan wrote:
> Should the platform here be for the target_os (trybot)? Or for the platform
the
> user is running on?

I think it has to be the host.

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:176:
return None  # TODO(sullivan): Implement me.
I think this implementation is correct. We'd never use a trybot as the default
browser.

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:186: #
TODO(sullivan): cache results!
On 2014/08/12 19:27:38, sullivan wrote:
> Where is a good place to put a cached copy of this data?

Does @decorators.Cache help?

[sullivan, 2014-08-12 21:52:47]

I still haven't written the unit tests--wanted to double-check that the way I'm
calling git is okay before I write a lot of code to mock it out :)

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
File tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py
(right):

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:17:
CONFIG = """config = {
On 2014/08/12 20:10:46, tonyg wrote:
> Let's deal with this as a dict rather than a string. And just use
> write('config=%s' % json.dumps(...)) at the right point.

Done.

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:34:
CONFIG_FILENAME = 'tools/run-bisect-perf-regression.cfg'
On 2014/08/12 20:10:44, tonyg wrote:
> There's also a third_party/WebKit/Tools/run-perf-test.cfg that we should edit
in
> the same way if there're blink changes in the patch. That could be deferred
for
> a follow-up patch.

Added a TODO for now. But is there a good way to get a list of modified files?
The only way I can think of is searching through local commits one by one.

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:51: #
TODO(sullivan): figure out if there are any options trybots don't support
On 2014/08/12 20:10:45, tonyg wrote:
> On 2014/08/12 19:27:37, sullivan wrote:
> > Can anyone think of options to check for?
> 
> --chrome-root
> --device
> --remote
> --identity
> --interactive
> --profile-dir

Added all those except --identity, which maps to cros_ssh_identity and is
automatically set.

Also added extensions_to_load

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:65:
['git', 'rev-parse', '--abbrev-ref', 'HEAD']).strip()
On 2014/08/12 20:10:45, tonyg wrote:
> Would be nice to make a friendly error message if they're not in a git repo.

Done.

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:71:
except subprocess.CalledProcessError:
On 2014/08/12 20:10:45, tonyg wrote:
> Can we check the exit status rather than for an exception?

I wrote a wrapper around subprocess.Popen() that gets rid of the exceptions. Let
me know if I missed something simpler.

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:73: if
dirty:
On 2014/08/12 20:10:46, tonyg wrote:
> We should probably also check the opposite: that they do have a local commit.
> Just to avoid confusion about how this should be used.

Done.

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:81:
subprocess.check_output(['git', 'checkout', '-b', 'telemetry-tryjob'])
On 2014/08/12 20:10:45, tonyg wrote:
> On 2014/08/12 19:27:38, sullivan wrote:
> > This prints out "Switched to branch 'telemetry-tryjob'". Anyone know if we
> want
> > to suppress that and how?
> 
> Let's suppress. There are some examples here:
>
https://code.google.com/p/chromium/codesearch#search/&q=telemetry%20stdout=de...

Done.

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:89: if
self._target_os == 'win':
On 2014/08/12 20:10:46, tonyg wrote:
> We should double check, but I *think* the trybots know how to convert the
> command for windows. if so, we should just rely on that.

I can't find that in the bisect scripts, is there somewhere in the buildbot code
I should look?

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:109:
config_file.close()
On 2014/08/12 20:10:46, tonyg wrote:
> Let's replace all close()s with with-blocks.

Is this still needed after simplifying to just overwrite?

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:110:
contents = contents.replace(REPLACE_CONFIG, config)
On 2014/08/12 20:10:45, tonyg wrote:
> Rather than read the old one and replace something, we should be able to just
> stomp on the whole thing.

Done.

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:135: #
user? logging.info is hidden by default.
On 2014/08/12 20:10:46, tonyg wrote:
> On 2014/08/12 19:27:38, sullivan wrote:
> > What's the best way to print non-error output?
> 
> I think we usually simply `print`.

Done.

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:168: #
TODO(sullivan): create right platform for target_os?
On 2014/08/12 20:10:45, tonyg wrote:
> On 2014/08/12 19:27:38, sullivan wrote:
> > Should the platform here be for the target_os (trybot)? Or for the platform
> the
> > user is running on?
> 
> I think it has to be the host. 

Done.

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:176:
return None  # TODO(sullivan): Implement me.
On 2014/08/12 20:10:46, tonyg wrote:
> I think this implementation is correct. We'd never use a trybot as the default
> browser.

Done.

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:186: #
TODO(sullivan): cache results!
On 2014/08/12 20:10:45, tonyg wrote:
> On 2014/08/12 19:27:38, sullivan wrote:
> > Where is a good place to put a cached copy of this data?
> 
> Does @decorators.Cache help?

The doc says, "cached functions maintain their cache for the lifetime of the
/module/". The code looks like an in-memory cache. So that means it would reload
the URL every time you ran the command, right?

[sullivan, 2014-08-13 19:05:56]

I started writing unit tests, posted WIP for feedback. Still have a few
questions:

1) What's the best way to mock out file open() for unit tests? Google tells me
to use MagicMock, but it looks like we prefer the modules in the
telemetry/unittest directory? i can't find anything in there that would mock out
a file.

2) Still not sure where the best place to put a cache file for the http request
is.

[sullivan, 2014-08-13 19:09:14]

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
File tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py
(right):

https://codereview.chromium.org/461043003/diff/1/tools/telemetry/telemetry/co...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:34:
CONFIG_FILENAME = 'tools/run-bisect-perf-regression.cfg'
On 2014/08/12 21:52:45, sullivan wrote:
> On 2014/08/12 20:10:44, tonyg wrote:
> > There's also a third_party/WebKit/Tools/run-perf-test.cfg that we should
edit
> in
> > the same way if there're blink changes in the patch. That could be deferred
> for
> > a follow-up patch.
> 
> Added a TODO for now. But is there a good way to get a list of modified files?
> The only way I can think of is searching through local commits one by one.

I realized that I'd need to re-run almost the entire codepath in the
third_party/WebKit dir, since it's on a different git branch. Need to think more
about how this would work, definitely going to punt on it for this CL.

[sullivan, 2014-08-14 19:49:37]

This is ready for review. It will require a merge with
https://codereview.chromium.org/472913002/

[sullivan, 2014-08-15 14:58:37]

All the dependent CLs for this are committed and merged, but I have one problem,
see below:

https://codereview.chromium.org/461043003/diff/100001/tools/telemetry/telemet...
File tools/telemetry/telemetry/unittest/run_tests.py (right):

https://codereview.chromium.org/461043003/diff/100001/tools/telemetry/telemet...
tools/telemetry/telemetry/unittest/run_tests.py:222: return
super(RunTestsCommand, cls).main(args)
There's one big problem with this CL: the unit tests end up invoking FindBrowser
which calls FindAllBrowserTypes() which calls
trybot_browser_finder.FindAllBrowserTypes() which pings the buildbot JSON url.

I made this ugly hack to demonstrate that decorators.Cache is working correctly:
urllib.urlopen() gets called exactly once when running all the unit tests. But
mocking out url opens for my trybot_browser_finder_unittest in run_tests is
awful. Is there a better way to do this?

[tonyg, 2014-08-15 20:56:44]

lgtm

Dave, would you like to take a pass too?

https://codereview.chromium.org/461043003/diff/100001/tools/telemetry/telemet...
File tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py
(right):

https://codereview.chromium.org/461043003/diff/100001/tools/telemetry/telemet...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:48: def
_RunProcess(self, cmd):
How is this different than check_output?

[sullivan, 2014-08-15 21:03:20]

I still feel like there should be a better way to do the monkey-patch in
run_tests.py. Dave, any ideas?

https://codereview.chromium.org/461043003/diff/100001/tools/telemetry/telemet...
File tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py
(right):

https://codereview.chromium.org/461043003/diff/100001/tools/telemetry/telemet...
tools/telemetry/telemetry/core/backends/remote/trybot_browser_finder.py:48: def
_RunProcess(self, cmd):
On 2014/08/15 20:56:43, tonyg wrote:
> How is this different than check_output?

It doesn't throw an exception when the process has a non-zero exit code. It also
swallows the output from git checkout, which was printed to the console with
check_output.

[dtu, 2014-08-20 20:47:55]

On 2014/08/15 21:03:20, sullivan wrote:
> I still feel like there should be a better way to do the monkey-patch in
> run_tests.py. Dave, any ideas?

For the purposes of the trybot_browser_finder unit test, we can monkey patch
urllib2 in the unit test's set up, similar to how we stub open().

For eliminating the extra request during run_tests.main(), we should change
browser_finder to only call FindAllBrowserTypes when we can't find the chosen
browser in a higher-priority browser_finder. An in, short-circuit and return
early if we have a matching browser. Then we only make the request if the user
does --browser=trybotsomething, or the browser can't be found, or if the user
does --browser=list.

[sullivan, 2014-08-20 20:52:46]

On 2014/08/20 20:47:55, dtu wrote:
> On 2014/08/15 21:03:20, sullivan wrote:
> > I still feel like there should be a better way to do the monkey-patch in
> > run_tests.py. Dave, any ideas?
> 
> For the purposes of the trybot_browser_finder unit test, we can monkey patch
> urllib2 in the unit test's set up, similar to how we stub open().
> 
> For eliminating the extra request during run_tests.main(), we should change
> browser_finder to only call FindAllBrowserTypes when we can't find the chosen
> browser in a higher-priority browser_finder. An in, short-circuit and return
> early if we have a matching browser. Then we only make the request if the user
> does --browser=trybotsomething, or the browser can't be found, or if the user
> does --browser=list.

I'm working on this now.

How should we handle the case where --browser=any? run_tests.main() sets
browser_type to 'any', which currently makes a list of all available browsers. I
started down the path of splitting into FindAllBrowserTypesLocal/Remote,
FindAllAvailableBrowsersLocal/Remote, but maybe i missed a simpler way to do it?

[chromium-reviews, 2014-08-20 21:06:34]

Have you considered having a --trybot flag instead of shoehorning this into
--browser?


Chris~


On Wed, Aug 20, 2014 at 1:52 PM, <sullivan@chromium.org> wrote:

> On 2014/08/20 20:47:55, dtu wrote:
>
>> On 2014/08/15 21:03:20, sullivan wrote:
>> > I still feel like there should be a better way to do the monkey-patch in
>> > run_tests.py. Dave, any ideas?
>>
>
>  For the purposes of the trybot_browser_finder unit test, we can monkey
>> patch
>> urllib2 in the unit test's set up, similar to how we stub open().
>>
>
>  For eliminating the extra request during run_tests.main(), we should
>> change
>> browser_finder to only call FindAllBrowserTypes when we can't find the
>> chosen
>> browser in a higher-priority browser_finder. An in, short-circuit and
>> return
>> early if we have a matching browser. Then we only make the request if the
>> user
>> does --browser=trybotsomething, or the browser can't be found, or if the
>> user
>> does --browser=list.
>>
>
> I'm working on this now.
>
> How should we handle the case where --browser=any? run_tests.main() sets
> browser_type to 'any', which currently makes a list of all available
> browsers. I
> started down the path of splitting into FindAllBrowserTypesLocal/Remote,
> FindAllAvailableBrowsersLocal/Remote, but maybe i missed a simpler way to
> do it?
>
>
>
> https://chromiumcodereview.appspot.com/461043003/
>
> To unsubscribe from this group and stop receiving emails from it, send an
> email to telemetry+unsubscribe@chromium.org.
>

To unsubscribe from this group and stop receiving emails from it, send an email
to chromium-reviews+unsubscribe@chromium.org.

[sullivan, 2014-08-20 21:18:41]

On 2014/08/20 21:06:34, chromium-reviews wrote:
> Have you considered having a --trybot flag instead of shoehorning this into
> --browser?

The main reason not to do that is that we want to keep the command line as
simple as possible. It might be confusing for users that the --browser flag gets
ignored if the --trybot is specified, etc. And there may be additional remote
hosts in the future.

That said, I implemented a working version of this with --browser and it's
pretty ugly. Tony, Dave, WDYT?

[tonyg, 2014-08-20 21:24:52]

On 2014/08/20 21:18:41, sullivan wrote:
> On 2014/08/20 21:06:34, chromium-reviews wrote:
> > Have you considered having a --trybot flag instead of shoehorning this into
> > --browser?
> 
> The main reason not to do that is that we want to keep the command line as
> simple as possible. It might be confusing for users that the --browser flag
gets
> ignored if the --trybot is specified, etc. And there may be additional remote
> hosts in the future.
> 
> That said, I implemented a working version of this with --browser and it's
> pretty ugly. Tony, Dave, WDYT?

I'm ambivalent on the syntax here, there are pros and cons to each.

[sullivan, 2014-08-21 14:53:47]

On 2014/08/20 21:24:52, tonyg wrote:
> On 2014/08/20 21:18:41, sullivan wrote:
> > On 2014/08/20 21:06:34, chromium-reviews wrote:
> > > Have you considered having a --trybot flag instead of shoehorning this
into
> > > --browser?
> > 
> > The main reason not to do that is that we want to keep the command line as
> > simple as possible. It might be confusing for users that the --browser flag
> gets
> > ignored if the --trybot is specified, etc. And there may be additional
remote
> > hosts in the future.
> > 
> > That said, I implemented a working version of this with --browser and it's
> > pretty ugly. Tony, Dave, WDYT?
> 
> I'm ambivalent on the syntax here, there are pros and cons to each.

I'm going to try to reimplement this with a --trybot flag like the --device
flag.

[sullivan, 2014-08-21 16:32:41]

On 2014/08/21 14:53:47, sullivan wrote:
> On 2014/08/20 21:24:52, tonyg wrote:
> > On 2014/08/20 21:18:41, sullivan wrote:
> > > On 2014/08/20 21:06:34, chromium-reviews wrote:
> > > > Have you considered having a --trybot flag instead of shoehorning this
> into
> > > > --browser?
> > > 
> > > The main reason not to do that is that we want to keep the command line as
> > > simple as possible. It might be confusing for users that the --browser
flag
> > gets
> > > ignored if the --trybot is specified, etc. And there may be additional
> remote
> > > hosts in the future.
> > > 
> > > That said, I implemented a working version of this with --browser and it's
> > > pretty ugly. Tony, Dave, WDYT?
> > 
> > I'm ambivalent on the syntax here, there are pros and cons to each.
> 
> I'm going to try to reimplement this with a --trybot flag like the --device
> flag.

I looked at the code in more detail, and other options like --device and
--remote still require --browser, and the code is really structured around
--browser. But I realized if I pass the options into FindAllBrowserTypes, I can
isolate the code that has to know about whether to make a url request into
trybot_browser_finder.py, and not need to do ugly stuff like monkeypatch
run_tests.py. New patch, PTAL

[tonyg, 2014-08-21 16:37:44]

lgtm

https://codereview.chromium.org/461043003/diff/140001/tools/telemetry/telemet...
File tools/telemetry/telemetry/core/backends/chrome/desktop_browser_finder.py
(right):

https://codereview.chromium.org/461043003/diff/140001/tools/telemetry/telemet...
tools/telemetry/telemetry/core/backends/chrome/desktop_browser_finder.py:91: #
pylint: disable=W0613
Rather than disable the lint warning, just change the arg name to "_".

[sullivan, 2014-08-21 18:01:00]

The CQ bit was checked by sullivan@chromium.org

[commit-bot: I haz the power, 2014-08-21 18:01:41]

CQ is trying da patch. Follow status at
 https://chromium-status.appspot.com/cq/sullivan@chromium.org/461043003/180001

[commit-bot: I haz the power, 2014-08-21 19:58:44]

FYI, CQ is re-trying this CL (attempt #1).
The failing builders are:
  chromium_presubmit on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presub...)
  linux_chromium_gn_rel on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/linux_chromium_...)
  mac_chromium_rel_swarming on tryserver.chromium.mac
(http://build.chromium.org/p/tryserver.chromium.mac/builders/mac_chromium_rel_...)
  win_chromium_rel_swarming on tryserver.chromium.win
(http://build.chromium.org/p/tryserver.chromium.win/builders/win_chromium_rel_...)

[commit-bot: I haz the power, 2014-08-21 20:01:37]

The CQ bit was unchecked by commit-bot@chromium.org

[commit-bot: I haz the power, 2014-08-21 20:01:38]

Try jobs failed on following builders:
  chromium_presubmit on tryserver.chromium.linux
(http://build.chromium.org/p/tryserver.chromium.linux/builders/chromium_presub...)

[sullivan, 2014-08-21 21:50:50]

The CQ bit was checked by sullivan@chromium.org

[commit-bot: I haz the power, 2014-08-21 21:54:36]

CQ is trying da patch. Follow status at
 https://chromium-status.appspot.com/cq/sullivan@chromium.org/461043003/160002

[commit-bot: I haz the power, 2014-08-21 23:38:23]

Committed patchset #11 (160002) as 291246