Check build status by parsing tryserver buildbot JSON.

Check build status by parsing tryserver buildbot JSON.

Instead of waiting for build based on timeout, now we try to check
the build status by parsing the buildbot JSON. This will remove unnecessary
wait when build fails. Otherwise build is treated as Pending, and bisect script waits till max timeout.

BUG=368939
NOTRY=true

Committed: https://src.chromium.org/viewvc/chrome?view=rev&revision=275573

[prasadv, 2014-05-22 23:20:27]

ptal

[ghost stip (do not use), 2014-05-23 00:40:22]

lgtm with a request to use chrome-build-extract.

this is way faster and shouldn't have any load impact on the TS

I believe the build only shows up when it's done, ryan can you comment on that?

https://chromiumcodereview.appspot.com/295133002/diff/20001/tools/post_perf_b...
File tools/post_perf_builder_job.py (right):

https://chromiumcodereview.appspot.com/295133002/diff/20001/tools/post_perf_b...
tools/post_perf_builder_job.py:241: buildbot_url = BUILDER_JSON_URL %
{'server_url': server_url,
it would be ideal to use
http://chrome-build-extract.appspot.com/get_builds?builder=linux_perf_bisect_...
 (fill in builder as appropriate) as it's a good mirror of what is happening on
the tryserver.

[prasadv, 2014-05-23 17:56:19]

https://codereview.chromium.org/295133002/diff/20001/tools/post_perf_builder_...
File tools/post_perf_builder_job.py (right):

https://codereview.chromium.org/295133002/diff/20001/tools/post_perf_builder_...
tools/post_perf_builder_job.py:241: buildbot_url = BUILDER_JSON_URL %
{'server_url': server_url,
Thank you Mike,
 Can bisect bot on master4 access this site?
Where can I get more information about chrome-build-extract.appspot, I would
like to get a status of particular build not all build, is there a way to do
this?

On 2014/05/23 00:40:23, stip wrote:
> it would be ideal to use
>
http://chrome-build-extract.appspot.com/get_builds?builder=linux_perf_bisect_...
>  (fill in builder as appropriate) as it's a good mirror of what is happening
on
> the tryserver.

[qyearsley, 2014-05-23 23:17:16]

https://codereview.chromium.org/295133002/diff/20001/tools/bisect-perf-regres...
File tools/bisect-perf-regression.py (right):

https://codereview.chromium.org/295133002/diff/20001/tools/bisect-perf-regres...
tools/bisect-perf-regression.py:1651: while True:
Suggestion: Would it be clearer if this wait-loop were extracted out to its own
function?

https://codereview.chromium.org/295133002/diff/20001/tools/bisect-perf-regres...
tools/bisect-perf-regression.py:1652: res = condition()
It's not obvious here that this condition function is actually trying to fetch
from cloud storage -- it might be clearer if you renamed this condition argument
and added something in the docstring about what it's expected to do.

[ghost stip (do not use), 2014-05-23 23:30:38]

On 2014/05/23 17:56:19, prasadv wrote:
>
https://codereview.chromium.org/295133002/diff/20001/tools/post_perf_builder_...
> File tools/post_perf_builder_job.py (right):
> 
>
https://codereview.chromium.org/295133002/diff/20001/tools/post_perf_builder_...
> tools/post_perf_builder_job.py:241: buildbot_url = BUILDER_JSON_URL %
> {'server_url': server_url,
> Thank you Mike,
>  Can bisect bot on master4 access this site?
> Where can I get more information about chrome-build-extract.appspot, I would
> like to get a status of particular build not all build, is there a way to do
> this?
> 
> On 2014/05/23 00:40:23, stip wrote:
> > it would be ideal to use
> >
>
http://chrome-build-extract.appspot.com/get_builds?builder=linux_perf_bisect_...
> >  (fill in builder as appropriate) as it's a good mirror of what is happening
> on
> > the tryserver.

you should be able to get to it from the bots. ryan, chrome-build-extract has a
specific build endpoint right?

[chromium-reviews, 2014-05-27 20:10:26]

CBE has a build endpoint using the same scheme as buildbot, eg:
http://chrome-build-extract.appspot.com/p/tryserver.chromium/builders/linux_c...

The functionality exists, but is not widely used, and therefore might be
buggy so let me know if it 500's.  Also CBE has a ~10 min lag on retrieving
build logs (which is fine for our data aggregation purposes, but it means
its not a super mirror)


On Fri, May 23, 2014 at 4:30 PM, <stip@chromium.org> wrote:

> On 2014/05/23 17:56:19, prasadv wrote:
>
> https://codereview.chromium.org/295133002/diff/20001/
> tools/post_perf_builder_job.py
>
>> File tools/post_perf_builder_job.py (right):
>>
>
>
> https://codereview.chromium.org/295133002/diff/20001/
> tools/post_perf_builder_job.py#newcode241
>
>> tools/post_perf_builder_job.py:241: buildbot_url = BUILDER_JSON_URL %
>> {'server_url': server_url,
>> Thank you Mike,
>>   Can bisect bot on master4 access this site?
>> Where can I get more information about chrome-build-extract.appspot, I
>> would
>> like to get a status of particular build not all build, is there a way to
>> do
>> this?
>>
>
>  On 2014/05/23 00:40:23, stip wrote:
>> > it would be ideal to use
>> >
>>
>
> http://chrome-build-extract.appspot.com/get_builds?
> builder=linux_perf_bisect_builder&master=tryserver.chromium
>
>> >  (fill in builder as appropriate) as it's a good mirror of what is
>> happening
>> on
>> > the tryserver.
>>
>
> you should be able to get to it from the bots. ryan, chrome-build-extract
> has a
> specific build endpoint right?
>
> https://chromiumcodereview.appspot.com/295133002/
>

To unsubscribe from this group and stop receiving emails from it, send an email
to chromium-reviews+unsubscribe@chromium.org.

[ghost stip (do not use), 2014-05-28 00:13:41]

ok, if the CBE stuff is possibly unreliable let's use the method here for now.
lgtm pending qyearsley's approval

[qyearsley, 2014-05-30 02:58:01]

On 2014/05/28 00:13:41, stip wrote:
> ok, if the CBE stuff is possibly unreliable let's use the method here for now.
> lgtm pending qyearsley's approval

LGTM although I think that the PostBuildRequestAndWait function could be made
simpler.

[prasadv, 2014-05-30 18:17:57]

On 2014/05/30 02:58:01, qyearsley wrote:
> On 2014/05/28 00:13:41, stip wrote:
> > ok, if the CBE stuff is possibly unreliable let's use the method here for
now.
> > lgtm pending qyearsley's approval
> 
> LGTM although I think that the PostBuildRequestAndWait function could be made
> simpler.

Modified as per Quinten's commnets,  can you please take a quick looks.

[ghost stip (do not use), 2014-05-30 22:36:31]

almost lg with one change

https://chromiumcodereview.appspot.com/295133002/diff/30001/tools/bisect-perf...
File tools/bisect-perf-regression.py (right):

https://chromiumcodereview.appspot.com/295133002/diff/30001/tools/bisect-perf...
tools/bisect-perf-regression.py:1577: condition=fetch_build_func,
you should rename condition= to fetch_build=, right?

https://chromiumcodereview.appspot.com/295133002/diff/30001/tools/bisect-perf...
tools/bisect-perf-regression.py:1636: return (res, None)
nit: it's slightly clearer to the reader that res is always true here if you
make None into 'build successfully found' even if that message is never used

[prasadv, 2014-06-06 22:18:20]

Hi Mike, Can you please take a final look at this one.

thanks,
Prasad

[ghost stip (do not use), 2014-06-06 22:44:02]

lgtm, any further issues we can shake out in production

[prasadv, 2014-06-06 22:44:39]

The CQ bit was checked by prasadv@chromium.org

[commit-bot: I haz the power, 2014-06-06 22:46:39]

CQ is trying da patch. Follow status at
 https://chromium-status.appspot.com/cq/prasadv@chromium.org/295133002/50001

[commit-bot: I haz the power, 2014-06-07 00:27:19]

Change committed as 275573