Add ChromiumWebRTC FYI waterfall

Add ChromiumWebRTC FYI waterfall

This waterfall has the purpose of providing support for:
* updating the WebRTC stable branch (use status for latest trunk)
* updating webrtc_revision in Chromium DEPS (use status for latest 
  stable).

Triggering of builders:
The idea is to trigger on new revisions in WebRTC trunk and stable 
branch. Since these change quite infrequently compared to Chromium, 
I've also added a Periodic scheduler with 4 hour intervals that 
triggers both the trunk and stable builders.
That way we can narrow down the number of Chromium revisions that 
has passed between a failing build and the latest passing build.
Eventually we would like to add more machines and separate 
builders/testers to bring down the interval even further, but this 
is good enough to start with.

This CL also changes the master.chromium.webrtc waterfall to build 
plain Chromium builds instead of using WebRTC ToT, since that's being 
done in this waterfall now. 
It is moved in here since it should be consdiered a FYI build if the 
latest WebRTC trunk would break Chromium.

Support for Periodic schedulers is added to 
scripts/master/master_config.py

BUG=none
TEST=local builds.


Committed: http://src.chromium.org/viewvc/chrome?view=rev&revision=178817

[kjellander_chromium, 2013-01-24 15:09:26]

Hi Mike!
I've been having ilevy@ as reviewer for most of my recent buildbot CLs but I
thought it would be good to spread the load a bit.

This waterfall will make life easier for people that are about to merge from
WebRTC trunk -> stable branch and also provide useful data when about to roll
webrtc_revision in Chromium DEPS.
We have similar bots setup in our own infrastructure today and we'd be grateful
if we can move this into Chrome infra.

[iannucci, 2013-01-24 19:08:45]

On 2013/01/24 15:09:26, Henrik Kjellander wrote:
> Hi Mike!
> I've been having ilevy@ as reviewer for most of my recent buildbot CLs but I
> thought it would be good to spread the load a bit.
> 
> This waterfall will make life easier for people that are about to merge from
> WebRTC trunk -> stable branch and also provide useful data when about to roll
> webrtc_revision in Chromium DEPS.
> We have similar bots setup in our own infrastructure today and we'd be
grateful
> if we can move this into Chrome infra.

+xusydoc, since he's been looking at webrtc stuff

Hey Henrik :)

Mike's username is xusydoc (I'm Robbie). But no worries! I have forwarded this
CL to him.

Cheers,
Rob

[Mike Stip (use stip instead), 2013-01-24 21:35:12]

lgtm

Looks great! Are you planning on hosting this off of build.chromium.org? If so,
we'll have to add a redirect to make sure that's accessible.

[kjellander_chromium, 2013-01-25 11:00:28]

On 2013/01/24 21:35:12, Mike Stipicevic wrote:
> lgtm
> 
> Looks great! Are you planning on hosting this off of build.chromium.org? If
so,
> we'll have to add a redirect to make sure that's accessible.

Yes. I will request that together with the database setup in a chrome infra
ticket.

[kjellander_chromium, 2013-01-25 11:05:42]

I think I avoided breaking some other waterfalls by having a second look today. 
See my comment and question in scripts/master/master_config.py

I'm still committing, since I assume it must be safe now that I only append the
Periodic support without altering the other code.

https://codereview.chromium.org/12033076/diff/6001/scripts/master/master_conf...
File scripts/master/master_config.py (right):

https://codereview.chromium.org/12033076/diff/6001/scripts/master/master_conf...
scripts/master/master_config.py:5: from buildbot.schedulers.basic import
SingleBranchScheduler
I forgot to ask about this part since I was tempted to change the imports to be
Buildbot 0.8.x to get rid of the warning about Scheduler being deprecated during
startup. 

I suspect it might break things if there are masters only running Buildbot
0.7.x. 
I looked around and couldn't find any use of it except some PYTHONPATHS setup in
Makefile.win32 files.
Are there masters in Chrome infra that runs Windows?

Since I suspect 0.7.x still needs to be supported I reverted my changes to that
version. Please have another look to ensure I don't break other waterfalls.

[commit-bot: I haz the power, 2013-01-25 11:05:57]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/kjellander@chromium.org/12033076/12001

[commit-bot: I haz the power, 2013-01-25 11:09:29]

Presubmit check for 12033076-12001 failed and returned exit status 1.

Running presubmit commit checks ...
Running pylint on 293 files.

** Presubmit ERRORS **
test/masters_test.py failed!
Command test/masters_test.py returned non-zero exit status 1 in
/b/commit-queue/workdir/tools/build
1 masters failed:
master.chromium.webrtc
1 masters were not expected:
master.chromium.webrtc.fyi
44 masters succeeded, 1 failed, 12 skipped in 140.9s.
Trying master.chromium
Trying master.chromium.chrome
Trying master.chromium.chromebot
Trying master.chromium.chromiumos
Trying master.chromium.endure
Trying master.chromium.flaky
Trying master.chromium.fyi
Trying master.chromium.git
Trying master.chromium.gpu
Trying master.chromium.gpu.fyi
Trying master.chromium.linux
Trying master.chromium.lkgr
Trying master.chromium.mac
Trying master.chromium.memory
Trying master.chromium.memory.fyi
Trying master.chromium.perf
Trying master.chromium.perf_av
Trying master.chromium.pyauto
Trying master.chromium.swarm
Trying master.chromium.webkit
Trying master.chromium.webrtc
2013-01-25 03:07:31-0800 [-] Creating BuildMaster -- buildbot.version: 0.8.4p1
2013-01-25 03:07:31-0800 [-] loading configuration from
/b/commit-queue/workdir/tools/build/masters/master.chromium.webrtc/master.cfg
2013-01-25 03:07:31-0800 [-] configuration update started
2013-01-25 03:07:31-0800 [-] unable to import dnotify, so Maildir will use
polling instead
2013-01-25 03:07:31-0800 [-] error while parsing config file
2013-01-25 03:07:31-0800 [-] Unhandled Error
	Traceback (most recent call last):
	  File
"/b/commit-queue/workdir/tools/build/third_party/buildbot_8_4p1/buildbot/master.py",
line 245, in loadTheConfigFile
	    d = self.loadConfig(f)
	  File
"/b/commit-queue/workdir/tools/build/third_party/buildbot_8_4p1/buildbot/master.py",
line 597, in loadConfig
	    d.addCallback(do_load)
	  File
"/b/commit-queue/workdir/tools/build/third_party/twisted_10_2/twisted/internet/defer.py",
line 297, in addCallback
	    callbackKeywords=kw)
	  File
"/b/commit-queue/workdir/tools/build/third_party/twisted_10_2/twisted/internet/defer.py",
line 286, in addCallbacks
	    self._runCallbacks()
	--- <exception caught here> ---
	  File
"/b/commit-queue/workdir/tools/build/third_party/twisted_10_2/twisted/internet/defer.py",
line 542, in _runCallbacks
	    current.result = callback(current.result, *args, **kw)
	  File
"/b/commit-queue/workdir/tools/build/third_party/buildbot_8_4p1/buildbot/master.py",
line 274, in do_load
	    exec f in localDict
	  File
"/b/commit-queue/workdir/tools/build/masters/master.chromium.webrtc/master.cfg",
line 24, in <module>
	    import master_cros_cfg
	  File
"/b/commit-queue/workdir/tools/build/masters/master.chromium.webrtc/master_cros_cfg.py",
line 49, in <module>
	    short_name='x86'))
	  File
"/b/commit-queue/workdir/tools/build/masters/master.chromium.webrtc/master_cros_cfg.py",
line 25, in CreateCbuildbotFactory

Trying master.chromium.win
Trying master.chromiumos
Trying master.client.dart
Trying master.client.dart.fyi
Trying master.client.drmemory
Trying master.client.dynamorio
Trying master.client.libjingle
Trying master.client.libyuv
Trying master.client.nacl
Trying master.client.nacl.chrome
Trying master.client.nacl.llvm
Trying master.client.nacl.ports
Trying master.client.nacl.ragel
Trying master.client.nacl.sdk
Trying master.client.nacl.sdk.addin
Trying master.client.nacl.sdk.mono
Trying master.client.nacl.toolchain
Trying master.client.omaha
Trying master.client.pagespeed
Trying master.client.v8
Trying master.client.webrtc
Trying master.devtools
Trying master.tryserver.chromium
Trying master.tryserver.nacl


test/masters_cfg_test.py failed!
Command test/masters_cfg_test.py returned non-zero exit status 1 in
/b/commit-queue/workdir/tools/build
Parsing client.v8
Parsing chromium.endure
Parsing chromium.memory
Skipping chromium.swarm, fix and enable in masters_cfg_test.py!
Skipping client.nacl.chrome, fix and enable in masters_cfg_test.py!
Parsing experimental
Parsing chromium.win
Parsing client.skia
Parsing chromium
Parsing client.sfntly
Parsing chromium.webrtc
error while parsing
/b/commit-queue/workdir/tools/build/masters/master.chromium.webrtc/master.cfg: 
Traceback (most recent call last):
  File "/b/commit-queue/workdir/tools/build/scripts/common/master_cfg_utils.py",
line 76, in LoadConfig
    localdict = ExecuteConfig(canonical_config)
  File "/b/commit-queue/workdir/tools/build/scripts/common/master_cfg_utils.py",
line 50, in ExecuteConfig
    exec f in localDict
  File
"/b/commit-queue/workdir/tools/build/masters/master.chromium.webrtc/master.cfg",
line 24, in <module>
    import master_cros_cfg
  File
"/b/commit-queue/workdir/tools/build/masters/master.chromium.webrtc/master_cros_cfg.py",
line 49, in <module>
    short_name='x86'))
  File
"/b/commit-queue/workdir/tools/build/masters/master.chromium.webrtc/master_cros_cfg.py",
line 25, in CreateCbuildbotFactory
    chrome_factory =
linux().ChromiumWebRTCLatestFactory(slave_type='WebRtcCros')
AttributeError: 'ChromiumFactory' object has no attribute
'ChromiumWebRTCLatestFactory'

build error encountered! aborting build
Skipping client.dart.fyi, fix and enable in masters_cfg_test.py!
Parsing chromium.webrtc.fyi
Parsing client.pagespeed
Parsing chromium.pyauto
Parsing client.libjingle
Parsing tryserver.chromium
perf_id: chromium-dbg-linux-try
perf_subdir: linux-try-debug
url: http://build.chromium.org/buildbot/coverage/linux-try-debug/total_coverage
Skipping chromiumos.tryserver, fix and enable in masters_cfg_test.py!
Parsing client.dynamorio
Skipping client.syzygy, fix and enable in masters_cfg_test.py!
Parsing client.omaha
Parsing client.nacl.llvm
Parsing chromium.perf
Parsing chromium.memory.fyi
Parsing tryserver.nacl
Parsing chromium.webkit
Parsing client.nacl.sdk
Parsing chromium.linux
Parsing chromium.perf_av
Parsing chromium.gpu.fyi
Parsing chromium.chrome
Parsing chromiumos
Parsing client.nacl
Parsing client.nacl.toolchain
Parsing chromium.lkgr
Parsing client.nacl.sdk.mono
Parsing chromium.flaky
Parsing client.webrtc
Parsing client.nacl.ragel
Parsing chromium.fyi
perf_id: chromium-dbg-linux
perf_subdir: linux-debug
url: http://build.chromium.org/buildbot/coverage/linux-debug/total_coverage
Parsing devtools
Parsing client.nacl.sdk.addin
Parsing client.drmemory
Parsing chromium.mac
Parsing client.libyuv
Skipping chromium.chromebot, fix and enable in masters_cfg_test.py!
Parsing chromium.gpu
Skipping client.dart, fix and enable in masters_cfg_test.py!
Parsing chromium.chromiumos
Parsing client.nacl.ports
Parsing chromium.git
The following master.cfgs did not load:
  chromium.webrtc:
/b/commit-queue/workdir/tools/build/masters/master.chromium.webrtc


Presubmit checks took 207.9s to calculate.

[commit-bot: I haz the power, 2013-01-25 12:54:32]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/kjellander@chromium.org/12033076/1037

[commit-bot: I haz the power, 2013-01-25 12:58:09]

Presubmit check for 12033076-1037 failed and returned exit status 1.

Running presubmit commit checks ...
Running pylint on 293 files.

** Presubmit ERRORS **
test/masters_test.py failed!
Command test/masters_test.py returned non-zero exit status 1 in
/b/commit-queue/workdir/tools/build
1 masters were not expected:
master.chromium.webrtc.fyi
45 masters succeeded, 0 failed, 12 skipped in 139.8s.
Trying master.chromium
Trying master.chromium.chrome
Trying master.chromium.chromebot
Trying master.chromium.chromiumos
Trying master.chromium.endure
Trying master.chromium.flaky
Trying master.chromium.fyi
Trying master.chromium.git
Trying master.chromium.gpu
Trying master.chromium.gpu.fyi
Trying master.chromium.linux
Trying master.chromium.lkgr
Trying master.chromium.mac
Trying master.chromium.memory
Trying master.chromium.memory.fyi
Trying master.chromium.perf
Trying master.chromium.perf_av
Trying master.chromium.pyauto
Trying master.chromium.swarm
Trying master.chromium.webkit
Trying master.chromium.webrtc
Trying master.chromium.win
Trying master.chromiumos
Trying master.client.dart
Trying master.client.dart.fyi
Trying master.client.drmemory
Trying master.client.dynamorio
Trying master.client.libjingle
Trying master.client.libyuv
Trying master.client.nacl
Trying master.client.nacl.chrome
Trying master.client.nacl.llvm
Trying master.client.nacl.ports
Trying master.client.nacl.ragel
Trying master.client.nacl.sdk
Trying master.client.nacl.sdk.addin
Trying master.client.nacl.sdk.mono
Trying master.client.nacl.toolchain
Trying master.client.omaha
Trying master.client.pagespeed
Trying master.client.v8
Trying master.client.webrtc
Trying master.devtools
Trying master.tryserver.chromium
Trying master.tryserver.nacl


Presubmit checks took 204.8s to calculate.

[commit-bot: I haz the power, 2013-01-25 13:10:02]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/kjellander@chromium.org/12033076/6024

[commit-bot: I haz the power, 2013-01-25 13:13:56]

Presubmit check for 12033076-6024 failed and returned exit status 1.

Running presubmit commit checks ...
Running pylint on 293 files.

** Presubmit ERRORS **
test/masters_test.py failed!
Command test/masters_test.py returned non-zero exit status 1 in
/b/commit-queue/workdir/tools/build
ERROR:root:Didn't find open port for master.chromium.webkit
1 masters failed:
master.chromium.webkit
45 masters succeeded, 1 failed, 12 skipped in 162.6s.
Trying master.chromium
Trying master.chromium.chrome
Trying master.chromium.chromebot
Trying master.chromium.chromiumos
Trying master.chromium.endure
Trying master.chromium.flaky
Trying master.chromium.fyi
Trying master.chromium.git
Trying master.chromium.gpu
Trying master.chromium.gpu.fyi
Trying master.chromium.linux
Trying master.chromium.lkgr
Trying master.chromium.mac
Trying master.chromium.memory
Trying master.chromium.memory.fyi
Trying master.chromium.perf
Trying master.chromium.perf_av
Trying master.chromium.pyauto
Trying master.chromium.swarm
Trying master.chromium.webkit
Trying master.chromium.webrtc
Trying master.chromium.webrtc.fyi
Trying master.chromium.win
Trying master.chromiumos
Trying master.client.dart
Trying master.client.dart.fyi
Trying master.client.drmemory
Trying master.client.dynamorio
Trying master.client.libjingle
Trying master.client.libyuv
Trying master.client.nacl
Trying master.client.nacl.chrome
Trying master.client.nacl.llvm
Trying master.client.nacl.ports
Trying master.client.nacl.ragel
Trying master.client.nacl.sdk
Trying master.client.nacl.sdk.addin
Trying master.client.nacl.sdk.mono
Trying master.client.nacl.toolchain
Trying master.client.omaha
Trying master.client.pagespeed
Trying master.client.v8
Trying master.client.webrtc
Trying master.devtools
Trying master.tryserver.chromium
Trying master.tryserver.nacl


Presubmit checks took 227.9s to calculate.

[commit-bot: I haz the power, 2013-01-25 13:21:00]

CQ is trying da patch. Follow status at
https://chromium-status.appspot.com/cq/kjellander@chromium.org/12033076/6024

[commit-bot: I haz the power, 2013-01-25 13:24:51]

Change committed as 178817