Chromium Code Reviews
chromiumcodereview-hr@appspot.gserviceaccount.com (chromiumcodereview-hr) | Please choose your nickname with Settings | Help | Chromium Project | Gerrit Changes | Sign out
(60)

Side by Side Diff: third_party/buildbot_7_12/buildbot/NEWS

Issue 12207158: Bye bye buildbot 0.7.12. (Closed) Base URL: svn://svn.chromium.org/chrome/trunk/tools/build
Patch Set: Created 7 years, 10 months ago
Use n/p to move between diff chunks; N/P to move between comments. Draft comments are only viewable by you.
Jump to:
View unified diff | Download patch | Annotate | Revision Log
« no previous file with comments | « third_party/buildbot_7_12/buildbot/CREDITS ('k') | third_party/buildbot_7_12/buildbot/README » ('j') | no next file with comments »
Toggle Intra-line Diffs ('i') | Expand Comments ('e') | Collapse Comments ('c') | Show Comments Hide Comments ('s')
OLDNEW
(Empty)
1 Major User visible changes in Buildbot. -*- outline -*-
2 (see ChangeLog for a detailed list of changes)
3
4 * Release 0.7.12 (January 21, 2010)
5
6 ** New 'console' display
7
8 This is a new web status view combining the best of the (t)grid and waterfall
9 views.
10
11 ** New 'extended' stylesheet
12
13 Buildbot has a new, much nicer stylesheet available. Copy the file
14 buildbot/status/web/extended.css over your existing public_html/buildbot.css to
15 se it.
16
17 ** Builders can be configured with an object
18
19 Instead of a list of dictionaries, builders can now specified using a
20 BuilderConfig object in the configuration file. This will allow for better
21 argument checking and default values, and also makes it easier for users to
22 create subclasses to handle site-specific builder details. The old,
23 dictionary-based method of configuration is still supported.
24
25 ** Check for common mis-configuration in addStep
26
27 When adding a new step to a factory, either of these are acceptable:
28
29 f.addStep(ShellCommand(command="echo hello, world", description="say hi"))
30 f.addStep(ShellCommand, command="echo hello, world", description="say hi")
31
32 but trying to mix these syntaxes is a common misconfiguration:
33
34 f.addStep(ShellCommand(command="echo hello, world"), description="say hi")
35
36 in which case the description argument was silently ignored. This is now an
37 error.
38
39 ** Support for log compression
40
41 Log files can be compressed on the master side using either gzip or bzip2.
42
43 ** Builder.ping no longer accepts timeout argument (bug #664).
44
45 The implementation was not robust enough and could cause the master to
46 unexpectedly disconnect the slave.
47
48 ** MailNotifier's customMesg replaced by messageFormatter
49
50 The customMesg mechanism had the unfortunate side effect of loading all data
51 for a build into memory simultaneously, which for some builds could cause
52 memory exhaustion.
53
54 ** Suppression of selected compiler warnings
55
56 The WarningCountingShellCommand class has been extended with the ability to
57 upload from the slave a file contain warnings to be ignored. See the
58 documentation of the suppressionFile argument to the Compile build step.
59
60 ** New buildstep `MTR'
61
62 A new class buildbot.process.mtrlogobserver.MTR was added. This buildstep is
63 used to run test suites using mysql-test-run. It parses the stdio output for
64 test failures and summarises them on the waterfall page. It also makes server
65 error logs available for debugging failures, and optionally inserts
66 information about test runs and test failures into an external database.
67
68 ** Python API Docs
69
70 The docstrings for buildbot are now available in a web-friendly format:
71 http://djmitche.github.com/buildbot/docs/latest/reference
72
73 ** Many, many bugfixes
74
75 * Release 0.7.11p (July 16, 2009)
76
77 Fixes a few test failures in 0.7.11, and gives a default value for branchType
78 if it is not specified by the master.
79
80 * Release 0.7.11 (July 5, 2009)
81
82 Developers too numerous to mention contributed to this release. Buildbot has
83 truly become a community-maintained application. Much hard work is not
84 mentioned here, so please consult the git logs for the detailed changes in this
85 release.
86
87 ** Better Memory Performance, Disk Cleanup
88
89 Buildbot handles its memory usage a bit better, and can automatically purge old
90 history to keep memory and disk usage low. Look for eventHorizon,
91 buildHorizon, logHorizon, and changeHorizon.
92
93 ** Password Protection for Force Build and Stop actions
94
95 It is now possible to require authentication to force build and stop via the
96 WebStatus interface. To use this, set the 'auth' field of WebStatus to a valid
97 IAuth implementation. Current implementations are:
98 BasicAuth with a list of user/passwords
99 HTPasswdAuth with an .htpasswd file
100 By default, the unauthenticated behavior will occur.
101
102 ** Web Status changes
103
104 The "Graceful Shutdown" feature, as a kind of "force", now obeys allowForce.
105
106 The waterfall and other pages are more deeply interlinked.
107
108 Pending builds can be individually cancelled, or cancelled in bulk.
109
110 ** Fixed Transfer Steps
111
112 Transfer step classes are more reliable; DirectoryUpload and DirectoryDownload
113 use tarfile instead of manually framing files. The DirectoryUpload step also
114 now supports compression.
115
116 ** Conditional Steps
117
118 Steps now take a doStepIf parameter which can be used to implement simple
119 conditional execution of a step.
120
121 ** Colorized Steps
122
123 Steps are now hilighted with a color in the build view to indicate their
124 success or failure.
125
126 ** Improved build prioritization
127
128 Bugfixes and fairer scheduling
129
130 ** Transposed Grid
131
132 Similar to the grid view, but with the axes reversed and showing different
133 info. Located at /tgrid.
134
135 ** Trigger steps improvements
136
137 Trigger now supports copy_properties, to send selected properties to the
138 triggered build.
139
140 * Release 0.7.10 (25 Feb 2009)
141
142 This release is mainly a collection of user-submitted patches since
143 the last release.
144
145 ** New Features
146
147 *** Environment variables in a builder (#100)
148
149 It is useful to be able to pass environment variables to all steps in a
150 builder. This is now possible by adding { .. 'env': { 'var' : 'value' }, ... }
151 to the builder specification.
152
153 *** IRC status plugin improvements (#330, #357, #378, #280, #381, #411, #368)
154
155 *** usePTY specified in master.cfg, defaults to False (#158, #255)
156
157 Using a pty has some benefits in terms of supporting "Stop Build", but causes
158 numerous problems with simpler jobs which can be killed by a SIGHUP when their
159 standard input is closed. With this change, PTYs are not used by default,
160 although you can enable them either on slaves (with the --usepty option to
161 create-slave) or on the master.
162
163 *** More information about buildslaves via the web plugin (#110)
164
165 A new page, rooted at /buildslave/$SLAVENAME, gives extensive information about
166 the buildslave.
167
168 *** More flexible merging of requests (#415)
169
170 The optional c['mergeRequests'] configuration parameter takes a function
171 which can decide whether two requests are mergeable.
172
173 *** Steps can be made to run even if the build has halted (#414)
174
175 Adding alwaysRun=True to a step will cause it to run even if some other step
176 has failed and has haltOnFailure=True.
177
178 *** Compress buildstep logfiles (#26)
179
180 Logs for each buildstep, which can take a lot of space on a busy buildmaster,
181 are automatically compressed after the step has finished.
182
183 *** Support for "latent" buildslaves
184
185 The buildslaves that are started on-demand are called "latent" buildslaves.
186 Buildbot ships with an abstract base class for building latent buildslaves, and
187 a concrete implementation for AWS EC2.
188
189 *** Customized MailNotifier messages (#175)
190
191 MailNotifier now takes an optional function to build the notification message,
192 allowing ultimate site-level control over the format of buildbot's notification
193 emails.
194
195 *** Nightly scheduler support for building only if changes have occurred
196
197 With the addition of onlyIfChanged=True, the Nightly scheduler will not schedule
198 a new build if no changes have been made since its last scheduled build.
199
200 *** Add ATOM/RSS feeds to WebStatus (#372)
201
202 Two new pages, /atom and /rss, provide feeds of build events to any feed
203 reader. These paths take the same "category" and "branch" arguments as the
204 waterfall and grid.
205
206 *** Add categories to Schedulers and Changes (#182)
207
208 This allows a moderate amount of support for multiple projects built in a
209 single buildmaster.
210
211 *** Gracefully shut down a buildslave after its build is complete
212
213 The /buildslaves/$SLAVENAME pages have a "Gracefully Shutdown" button which
214 will cause the corresponding slave to shut itself down when it finishes its
215 current build. This is a good way to do work on a slave without causing a
216 spurious build failure.
217
218 *** SVN source steps can send usernames and passwords (#41)
219
220 Adding username="foo" and/or password="bar" to an SVN step will cause
221 --username and --password arguments to be passed to 'svn' on the slave side.
222 Passwords are suitably obfuscated in logfiles.
223
224 ** New Steps
225
226 *** DirectoryUpload (#393)
227
228 This step uploads an entire directory to the master, and can be useful when a
229 build creates several products (e.g., a client and server package).
230
231 *** MasterShellCommand
232
233 This step runs a shell command on the server, and can be useful for
234 post-processing build products, or performing other maintenance tasks on the
235 master.
236
237 *** PyLint (#259)
238
239 A PyLint step is available to complement the existing PyFlakes step.
240
241 ** Bugs Fixed
242
243 *** Process output from new versions of Test::Harness (#346)
244
245 *** Fixes to the try client and scheduler
246
247 *** Remove redundant loop in MailNotifier (#315)
248
249 *** Display correct $PWD in logfiles (#179)
250
251 *** Do not assume a particular python version on Windows (#401)
252
253 *** Sort files in changes (#402)
254
255 *** Sort buildslaves lexically (#416)
256
257 *** Send properties to all builds initiated by AnyBranchScheduler
258
259 *** Dependent Schedulers are more robust to reconfiguration (#35)
260
261 *** Fix properties handling in triggered buidls (#392)
262
263 *** Use "call" on Windows to avoid errors (#417)
264
265 *** Support setDefaultWorkdir in FileUpload and FileDownload (#209)
266
267 *** Support WithProperties in FileUpload and FileDownload (#210)
268
269 *** Fix a bug where changes could be lost on a master crash (#202)
270
271 *** Remove color settings from non-presentation code (#251)
272
273 *** Fix builders which stopped working after a PING (#349, #85)
274
275 *** Isolate Python exceptions in status plugins (#388)
276
277 *** Notify about slaves missing at master startup (#302)
278
279 *** Fix tracebacks in web display after a reconfig (#176)
280
281 ** Version-Control Changes
282
283 *** Many Mercurial fixes
284
285 - Inrepo branch support finalized (source step + changegroup hook + test case)
286 (#65 #185 #187)
287
288 - Reduced amount of full clones by separating clone with update into
289 clone/pull/update steps (#186, #227) (see #412 for future work here)
290
291 - Fixed mercurial changegroup hook to work with Mercurial 1.1 API (#181, #380)
292
293 *** Many git fixes
294
295 *** Add got_revision to Perforce support (#127)
296
297 *** Use "git foo" everywhere instead of deprecated "git-foo"
298
299 ** Minor Changes
300
301 *** factory.addSteps (#317)
302
303 If you have a common list of steps that are included in multiple factories, you
304 can use f.addSteps(steplist) to add them all at once.
305
306 *** Twisted logfile rotation and cleanup (#108)
307
308 By default, Buildbot now rotates and cleans up the (potentially voluminous)
309 twistd.log files.
310
311 *** Prioritize build requests based on the time they wre submitted (#334)
312
313 Balancing of load is a bit more fair, although not true load balancing.
314
315 * Release 0.7.9 (15 Sep 2008)
316
317 ** New Features
318
319 *** Configurable public_html directory (#162)
320
321 The public_html/ directory, which provides static content for the WebStatus()
322 HTTP server, is now configurable. The default location is still the
323 public_html/ subdirectory of the buildmaster's base directory, but you can
324 change this by passing a suitable argument when creating the WebStatus()
325 instance in your master.cfg file:
326
327 c['status'].append( WebStatus(8080, public_html="/var/www/buildbot") )
328
329 *** Lock access modes (#313)
330
331 Albert Hofkamp added code to provide two distinct access modes to Locks:
332 "counting" and "exclusive". Locks can accept a configurable number of
333 "counting"-mode users, or a single "exclusive"-mode. For example, a Lock is
334 defined with maxCount=3, and then a 'compile' BuildStep uses this lock in
335 counting mode, while a 'cleanup' BuildStep uses this lock in exclusive mode.
336 Then, there can be one, two, or three simultaneous Builds in the compile step
337 (as long as there are no builds in the cleanup step). Only one build can be
338 in the cleanup step at a time, and if there is such a build in the cleanup
339 step, then the compile steps in other builds will wait for it to finish.
340 Please see the "Interlocks" section of the user's manual for more details.
341
342 ** Bugs Fixed
343
344 *** Buildslave missing_timeout= fired too quickly (#211)
345
346 By providing a missing_timeout= argument when creating the BuildSlave
347 instance, you can ask the buildmaster to send email if a buildslave is
348 disconnected for too long. A bug in the previous version caused this
349 notification to be sent too soon, rather than waiting until the timeout
350 period expired. This should be fixed now.
351
352 *** Test command display fixed (#332)
353
354 In the previous version, a steps.shell.Test step would display the parsed
355 test results (in the step's box on the waterfall display) in lieu of any
356 other descriptive text the step might provide. In this release, these two
357 pieces of information are combined.
358
359 ** Minor Changes
360
361 The buildmaster's version is logged to its twistd.log file at startup. The
362 buildslave does the same, to its own logfile.
363
364 Remote commands now record how long each command took. The "elapsedTime="
365 message will appear in the step's main logfile.
366
367 The "buildbot restart" command no longer fails if the buildbot wasn't already
368 running.
369
370 The FileUpload and FileDownload steps now create their target directories
371 (and any missing intermediate directories) before writing to the destination
372 file.
373
374 The per-build and per-step web pages now show the start, finish, and elapsed
375 time of their build or step.
376
377 If a Subversion-based build is started with a mixture of Changes that specify
378 particular numeric revisions and "HEAD" Changes (which indicate that a trunk
379 checkout is desired), the build will use a trunk checkout. Previously this
380 would probably cause an error. It is not clear how this situation might
381 arise.
382
383 ** Compability With Other Tools
384
385 The mercurial commit hook (buildbot.changes.hgbuildbot) in the previous
386 version doesn't work with hg-1.0 or later (it uses an API function that was
387 present in the hg-0.9.5 release, but was removed from hg-1.0). This
388 incompability has been fixed: the new version of buildbot should be
389 compatible with hg-1.0 and newer (and it probably retains compability with
390 hg-0.9.5 and earlier too). (#328)
391
392 The Git tool has traditionally provided two ways to run each command, either
393 as subcommands of /usr/bin/git (like "git checkout"), or as individual tools
394 (like /usr/bin/git-checkout). The latter form is being removed in the
395 upcoming 1.6 Git release. Previous versions of Buildbot have used the
396 git-checkout form, and will break when Git is upgraded to 1.6 or beyond. The
397 new Buildbot release switches to the subcommand form. Note that this is a
398 change on the buildslave side.
399
400 The Git checkout command will now use the default branch (as set in the
401 steps.source.Git() step definition) if the changes that it is building do not
402 specify some other branch to build. (#340)
403
404 ** Deprecation Schedule
405
406 No features have been deprecated in this release, and no deprecated features
407 have been removed. As a reminder, the following deprecated features are
408 scheduled for removal in an upcoming release:
409
410 c['sources'] (plural) was replaced by c['change_source'] (singular) in 0.7.6,
411 and will be removed by 0.8.0.
412
413 c['bots'] was replaced by c['buildslaves'] in 0.7.6, and will be removed by
414 0.8.0 . c['bots'] only accepts BuildSlave instances, not name/passwd tuples.
415
416 The html.Waterfall status target was replaced by html.WebStatus in 0.7.6, and
417 will be removed by 0.8.0.
418
419
420 * Release 0.7.8 (24 Jul 2008)
421
422 ** New features
423
424 The IRC bot will respond to three new commands: 'notify' subscribes the
425 channel (or the sender, if the command is sent as a private "/msg") to hear
426 about build events. 'join' tells the bot to join some new IRC channel.
427 'leave' tells it to leave a channel. See the "IRC Bot" section of the User's
428 Manual for details. (#171)
429
430 Build Steps now have "statistics", in addition to logfiles. These are used to
431 count things like how many tests passed or failed. There are methods to sum
432 these counters across all steps and display the results in the Build status.
433 The Waterfall display now shows the count of failed tests on the top-most box
434 in each column, using this mechanism.
435
436 The new buildbot.steps.shell.PerlModuleTest step was added, to run Perl unit
437 tests. This is a wrapper around the regular ShellCommand that parses the
438 output of the standard perl unit test system and counts how many tests
439 passed/failed/etc. The results are put into the step's summary text, and a
440 count of tests passed/failed/skipped are tracked in the steps's statistics.
441 The factory.CPAN build factory has been updated to use this, so configuring a
442 Buildbot to test a perl module available from CPAN should be as easy as:
443
444 s = source.CVS(cvsroot, cvsmodule)
445 f = factory.CPAN(s)
446
447 Build Properties have been generalized: they remain associated with a single
448 Build, but the properties can be set from a variety of sources. In previous
449 releases, the Build itself would set properties like 'buildername', 'branch',
450 and 'revision' (the latter two indicating which version of the source code it
451 was trying to get), and the source-checkout BuildSteps would set a property
452 named 'got_revision' (to indicate what version of the soruce code it actually
453 got). In this release, the 'scheduler' property is set to indicate which
454 Scheduler caused the build to be started. In addition, the config file can
455 specify properties to be set on all Builds, or on all Builds for a specific
456 Builder. All these properties are available for interpolation into
457 ShellCommands and environment variables by using the WithProperties() marker.
458
459 It may be easier to implement simple build parameterization (e.g. to upload
460 generated binaries to a specific directory, or to only perform long-running
461 tests on a nightly build instead of upon every checkin) by using these Build
462 Properties than to write custom BuildSteps.
463
464 ** Other improvements
465
466 The /buildslaves web page shows which slaves are currently running builds.
467 Offline slaves are displayed in bold.
468
469 Buildbot's setup.py now provides metadata to setuptools (if installed): an
470 entry_points script was added, and a dependency upon twisted-2.4.x or newer
471 was declared. This makes it more likely that 'easy_install buildbot' will
472 work.
473
474 The MailNotifier class acquired a mode="passing" flag: in this mode, the
475 buildbot will only send mail about passing builds (versus only on failing
476 builds, or only on builds which failed when the previous build had passed).
477
478 ** Bugs fixed
479
480 Don't display force/stop build buttons when build control is disabled (#246)
481
482 When a build is waiting on a lock, don't claim that it has started (#107)
483
484 Make SVN mode=copy tolerate symlinks on freebsd, "cp -rp" -> "cp -RPp" (#86)
485
486 The svnpoller changesource now ignores branch deletion (#261)
487
488 The Git unit tests should run even if the user has not told Git about their
489 username/email.
490
491 The WebStatus /xmlrpc server's getStatus() method was renamed to the
492 more-accurate getLastBuildResults().
493
494 The TinderboxMailNotifier status output acquired an useChangeTime= argument.
495
496 The bonsaipoller changesource got some fixes.
497
498 ** Deprecation Schedule
499
500 No features have been deprecated in this release, and no deprecated features
501 have been removed. As a reminder, the following deprecated features are
502 scheduled for removal in an upcoming release:
503
504 c['sources'] (plural) was replaced by c['change_source'] (singular) in 0.7.6,
505 and will be removed by 0.8.0.
506
507 c['bots'] was replaced by c['buildslaves'] in 0.7.6, and will be removed by
508 0.8.0 . c['bots'] only accepts BuildSlave instances, not name/passwd tuples.
509
510 The html.Waterfall status target was replaced by html.WebStatus in 0.7.6, and
511 will be removed by 0.8.0.
512
513
514
515 * Release 0.7.7 (29 Mar 2008)
516
517 ** Things You Need To Know
518
519 *** builder names must not start with an underscore (`_').
520
521 These are now reserved for internal buildbot purposes, such as the magic
522 "_all" pseudo-builder that the web pages use to allow force-build buttons
523 that start builds on all Builders at once.
524
525 ** New Features
526
527 *** "buildbot checkconfig"
528
529 The "buildbot checkconfig" command will look at your master.cfg file and tell
530 you if there are any problems with it. This can be used to test potential
531 changes to your config file before submitting them to the running
532 buildmaster. This is particularly useful to run just before doing "buildbot
533 restart", since the restart will fail if the config file has an error. By
534 running "buildbot checkconfig master.cfg && buildbot restart", you'll only
535 perform the restart if the config file was ok. Many thanks to Ben Hearsum for
536 the patch.
537
538 *** Waterfall "?category=FOO" query-arguments
539
540 The Waterfall page now accepts one or more "category=" query arguments in the
541 URL, to filter the display by categories. These behave a lot like the
542 "builder=" query argument. Thanks to Jermo Davann for the patch.
543
544 ** Bugs Fixed
545
546 Many bugs were fixed, and many minor features were added. Many thanks to
547 Dustin Mitchell who fixed and coordinated many of these. Here is a terse
548 list, for more details, please see the Trac page for the 0.7.7 release, at
549 http://buildbot.net/trac/query?status=closed&milestone=0.7.7 :
550
551 Many of the URLs generated by the buildbot were wrong.
552 Display of last-heard-from timestamps on the buildslaves web page were wrong.
553 Asking an IRC bot about a build waiting on a Lock should no longer crash.
554 Same for the web viewer.
555 Stop treating the encouraged info/ directory as leftover.
556 Add more force/stop build buttons.
557 Timestamps displayed on the waterfall now handle daylight savings properly.
558 p4poller no longer quits after a single failure.
559 Improved Git support, including 'try', branch, and revisions.
560 Buildslaves now use 'git', not 'cogito'.
561 Make older hg client/servers handle specific-revision builds properly.
562 Fix twisted.scripts._twistw problem on twisted-2.5.0 and windows.
563 Fix workdir= and env= on ShellCommands
564 Fix logfile-watching in 'buildbot start' on OS-X.
565 Fix ShellCommand crashes when the program emits >640kB of output per chunk.
566 New WarningCountingShellCommand step.
567 Fix TreeSize step.
568 Fix transfer.FileUpload/FileDownload crashes for large files.
569 Make 'buildbor reconfig' on windows tell you that it doesn't work.
570 Add a To: header to the mail sent by the slave-missing timeout.
571 Disable usePTY= for most unit tests, it makes some debian systems flunk tests.
572 Add 'absolute source stamps'
573 Add 'triggerable schedulers', and a buildstep to trigger them.
574 Remove buildbot.changes.freshcvsmail
575 Add new XMLRPC methods: getAllBuilders, getStatus, getLastBuilds.
576 Accept WithProperties in more places: env=, workdir=, others.
577 Use --no-auth-cache with SVN commands to avoid clobbering shared svn state.
578 Add hours/minutes/seconds in the waterfall's ETA display.
579 Trial: count Doctest lines too.
580 ShellCommand: record more info in the headers: stdin closing, PTY usage.
581 Make it possible to stop builds across reconfig boundaries.
582 SVN revision numbers are now passed as strings, which was breaking MailNotifier
583
584 ** Deprecation Schedule
585
586 The changes.freshcvsmail change source was replaced by
587 changes.mail.FCMaildirSource in 0.7.6, and has been removed in 0.7.7 .
588
589 c['sources'] (plural) was replaced by c['change_source'] (singular) in 0.7.6,
590 and will be removed by 0.8.0.
591
592 c['bots'] was replaced by c['buildslaves'] in 0.7.6, and will be removed by
593 0.8.0 . c['bots'] only accepts BuildSlave instances, not name/passwd tuples.
594
595 The html.Waterfall status target was replaced by html.WebStatus in 0.7.6, and
596 will be removed by 0.8.0.
597
598
599 * Release 0.7.6 (30 Sep 2007)
600
601 ** Things You Need To Know
602
603 *** 'buildbot upgrade-master'
604
605 Each time you install a new version of Buildbot, you should run the new
606 'buildbot upgrade-master' command on each of your pre-existing buildmasters.
607 This will add files and fix (or at least detect) incompatibilities between
608 your old config and the new code.
609
610 *** new WebStatus page
611
612 The Waterfall has been replaced by the more general WebStatus display,
613 described below. WebStatus serves static files from a new public_html/
614 directory that lives in the buildmaster's basedir. Files like index.html,
615 buildbot.css, and robots.txt are served directly from that directory, so any
616 modifications you wish to make should be made to those files. In particular,
617 any custom CSS you've written should be copied into public_html/buildbot.css.
618 The 'upgrade-master' command will populate this directory for you.
619
620 The old Waterfall page is deprecated, but it should continue to work for
621 another few releases. It is now a subclass of WebStatus which just replaces
622 the default root URL with another copy of the /waterfall resource.
623
624 *** Compatibility: Python-2.3 or newer, Twisted-2.0 or newer
625
626 No compatiblity losses here, buildbot-0.7.6 is compatible with the same
627 versions of python and twisted that 0.7.5 was.
628
629 Buildbot is tested on a regular basis (http://buildbot.buildbot.net) against
630 nearly a full matrix of Python-(2.3,2.4,2.5) * Twisted-(2.0,2.1,2.2,2.4,2.5).
631
632 *** New Buildbot Home Page
633
634 Buildbot has moved to a new Trac instance at http://buildbot.net/ , and all
635 new bugs and tickets should be filed there. The old sourceforge bugs at
636 http://buildbot.sf.net/ will slowly be migrated over. Mailing lists are still
637 managed at sourceforge, and downloads are still available there.
638
639 *** Changed/Deprecated master.cfg Keys and Classes
640
641 c['sources'] (plural) has been replaced by c['change_source'] (singular).
642
643 c['bots'] has been replaced by c['buildslaves'], and it expects a list of
644 BuildSlave instances instead of tuples. See below for more details.
645
646 The 'freshcvsmail' change source has been deprecated, and will be removed in
647 the next release.
648
649 The html.Waterfall status target has been deprecated, and replaced by
650 html.WebStatus .
651
652 ** New Features
653
654 *** WebStatus
655
656 The new WebStatus display is a superset of the old Waterfall. It contains a
657 waterfall as a sub-page, but it also contains pages with more compact
658 representations of recent build status. The "one_line_per_build" page
659 contains just that, and "one_box_per_builder" shows just the information from
660 the top of the waterfall page (last-finished-build and current-activity).
661
662 The initial page (when you hit the root of the web site) is served from
663 index.html, and provides links to the Waterfall as well as the other pages.
664
665 Most of these pages can be filtered by adding query arguments to the URL.
666 Adding "?builder=XYZ" will cause the page to only show results for the given
667 builder. Adding "?builder=XYZ&builder=ABC" will show results for either
668 builder. "?branch=trunk" will limit the results to builds that involved code
669 from the trunk.
670
671 The /waterfall page has arguments to hide those annoying "buildslave
672 connected" messages, to start and and at arbitrary times, and to auto-refresh
673 at a chosen interval (with a hardcoded minimum of 15 seconds). It also has a
674 "help" page with forms that will help you add all of these nifty filtering
675 arguments.
676
677 The recommended practice is to modify the index.html file to include links to
678 the filtered pages that you find most useful.
679
680 Note that WebStatus defaults to allowForce=False, meaning that the display
681 will not offer or accept "Force Build" or "Stop Build" controls. (The old
682 Waterfall defaults to allowForce=True).
683
684 The new WebStatus pages try very hard to use only relative links, making life
685 better when the Buildbot sits behind an HTTP reverse proxy.
686
687 In addition, there is a rudimentary XMLRPC server run by the WebStatus
688 object. It only has two methods so far, but it will acquire more in the
689 future. The first customer of this is a project to add a buildbot plugin to
690 Trac.
691
692 *** BuildFactory.addStep(Step(args))
693
694 BuildFactories can be set up either with a complete list of steps, or by
695 calling the .addStep() method repeatedly. The preferred way to provide a step
696 is by instantiating it, rather than giving a class/kwargs pair. This gives
697 the BuildStep class a chance to examine the arguments (and complain about
698 anything it doesn't like) while the config file is being read and problems
699 are being logged. For example, the old-style:
700
701 from buildbot.process.factory import BuildFactory, s
702 steps = [s(CVS, cvsroot="blah", mode="copy"),
703 s(Compile, command=["make", "all"]),
704 s(Test, command=["make", "test"]),
705 ]
706 f = BuildFactory(steps)
707
708 is now:
709
710 f = BuildFactory()
711 f.addStep( CVS(cvsroot="blah", mode="copy") )
712 f.addStep( Compile(command=["make", "all"]) )
713 f.addStep( Test(command=["make", "test"]) )
714
715 Authors of BuildStep subclasses which override __init__ to add new arguments
716 must register them with self.addFactoryArguments(**newargs) to make sure that
717 those classes will work with this new style, otherwise the new arguments will
718 be lost.
719
720 Using class/kwargs pairs is deprecated, and will be removed in a future
721 release.
722
723
724 *** BuildSlave instances, max_builds=, notify_on_missing=
725
726 Buildslave specification has changed a lot in this release. The old config:
727
728 c['bots'] = [ ("bot1name", "bot1passwd"),
729 ("bot2name", "bot2passwd") ]
730
731 is now:
732
733 from buildbot.buildslave import BuildSlave
734 c['slaves'] = [ BuildSlave("bot1name", "bot1passwd"),
735 BuildSlave("bot2name", "bot2passwd") ]
736
737 This new form gives us the ability to add new controls. The first is
738 "max_builds=", which imposes a concurrency limit that is like the usual
739 SlaveLock, but gives the buildmaster the opportunity to find a different
740 slave to run the build. (the buildslave is chosen before the SlaveLock is
741 claimed, so pure SlaveLocks don't let you take full advantage of build
742 farms).
743
744 The other addition is "notify_on_missing=", which accepts an email address
745 (or list of addresses), and sends a message when the buildslave has been
746 disconnected for more than an hour (configurable with missing_timeout=). This
747 may be useful when you expect that the buildslave hosts should be available
748 most of the time, and want to investigate the reasons that it went offline.
749
750
751 ** Other Improvements
752
753 The IRC bot has been refactored to make it easier to add instant-messaging
754 status delivery in the future. The IM plugins are not yet written, though.
755
756 When multiple buildslaves are available for a given build, one of them will
757 be picked at random. In previous releases, the first one on the list was
758 always picked. This helps to add a certain measure of load-balancing. More
759 improvements will be made in the future.
760
761 When the buildslave does a VC checkout step that requires clobbering the
762 build directory (i.e. in all modes except for 'update'), the buildslave will
763 first set the permissions on all build files to allow their deletion, before
764 it attempts to delete them. This should fix some problems in which a build
765 process left non-user-writable files lying around (frequently a result of
766 enthusiastic unit tests).
767
768 The BuildStep's workdir= argument can now accept a WithProperties()
769 specification, allowing greater control over the workdir.
770
771 Support for the 'Bazaar' version control system (/usr/bin/bzr) has been
772 added, using the buildbot.steps.source.Bzr class. This is a replacement for
773 the old 'Arch' (/usr/bin/tla and /usr/bin/baz) systems, which are still
774 supported by Buildbot with the source.Arch and source.Bazaar classes,
775 respectively. Unfortunately the old baz system claimed the 'Bazaar' classname
776 early, so the new system must use source.Bzr instead of the desired
777 source.Bazaar . A future release might change this.
778
779 A rudimentary Gnome Panel applet is provided in contrib/bb_applet.py, which
780 provides 'buildbot statusgui' -like colored status boxes inside the panel.
781 Installing it is a bit tricky, though.
782
783 The 'buildbot try' command now accepts a '--diff=foo.patch' argument, to let
784 you provide a pre-computed patch. This makes it easier to test out patches
785 that you've looked over for safety, without first applying them to your local
786 source tree.
787
788 A new Mercurial change source was added, hg_buildbot.py, which runs as an
789 in-process post-commit hook. This gives us access to much more information
790 about the change, as well as being much faster.
791
792 The email-based changesource have been refactored, to make it easier to write
793 new mail parsers. A parser for the SVN "commit-email.pl" script has been
794 added.
795
796 ** Bugs Fixed
797
798 Far too many to count. Please see
799 http://buildbot.net/trac/query?status=closed&milestone=0.7.6 for a partial
800 list of tickets closed for this release, and the ChangeLog for a complete
801 list of all changes since 0.7.5 .
802
803
804 * Release 0.7.5 (10 Dec 2006)
805
806 ** Things You Need To Know
807
808 *** The Great BuildStep Renaming
809
810 All BuildSteps have moved! They used to be classes in buildbot.process.step,
811 but now they all have separate modules in buildbot.steps.* . They have been
812 split out into separate categories: for example, the source checkout steps
813 are now buildbot.steps.source.CVS, buildbot.steps.source.Darcs, etc. The most
814 commonly used one is probably buildbot.steps.shell.ShellCommand . The
815 python-specific steps are in buildbot.steps.python, and the Twisted-specific
816 steps are in buildbot.steps.python_twisted .
817
818 You will need to update your master.cfg files to use the new names. The old
819 names are deprecated and will be removed altogether in the next release.
820
821 *** Compatibility
822
823 Buildbot now requires python-2.3 or later. Buildbot now requires
824 Twisted-2.0.0 or later. Support for earlier versions of both has finally been
825 removed. If you discover it works with unsupported versions, please return
826 your Buildbot to the factory for repairs :-).
827
828 Buildbot has *not* yet been tested against the recent python-2.5 release. It
829 has been tested against the latest SVN version of Twisted, but only in
830 conjunction with python-2.4 .
831
832 ** new features
833
834 *** reconfiguring a Builder no longer causes a disconnect/reconnect cycle
835
836 This means that sending SIGHUP to the master or running 'buildbot reconfig
837 MASTERDIR' command no longer interrupts any current builds, nor does it lose
838 pending builds like it did before. This involved a fairly substantial
839 refactoring of the various internal BotPerspective/BotMaster/Builder classes.
840 Note that reconfiguring Schedulers still loses any Changes that were waiting
841 for the tree to become stable: hopefully this will be fixed in the next
842 release.
843
844 *** 'buildbot start/restart/reconfig' now show logs until startup is complete
845
846 These commands now have additional code to follow twistd.log and display all
847 the lines that are emitted from the beginning of the start/reconfig action
848 until it has completed. This gives you a chance to see any problems detected
849 in the config file without needing to manually look in twistd.log or use
850 another shell to 'tail -f' it. This also makes it clear which config file is
851 being used. This functionality is not available under windows.
852
853 In addition, if any problems are detected during 'start' or 'restart' (but
854 not reconfig), the buildbot command will terminate with a non-zero exit
855 status, making it easier to use in scripts. Closes SF#1517975.
856
857 *** Locks now take maxCount=N to allow multiple simultaneous owners
858
859 This allows Locks to be non-exclusive but still limit maximum concurrency.
860 Thanks to James Knight for the patch. Closes SF#1434997.
861
862 *** filetransfer steps
863
864 buildbot.steps.transfer.FileUpload is a buildstep that will move files from
865 the slave to the master. Likewise, FileDownload will move files from the
866 master down to the buildslave. Many thanks to Albert Hofkamp for contributing
867 these classes. Closes SF#1504631.
868
869 *** pyflakes step
870
871 buildbot.steps.python.PyFlakes will run the simple 'pyflakes' static analysis
872 tool and parse the results to tell you about undefined names, unused imports,
873 etc. You'll need to tell it how to run pyflakes, usually with something like
874 command=["pyflakes", "src/packagedir"] or the like. The default command is
875 "make pyflakes", which assumes that you have a suitable target in your
876 top-level Makefile.
877
878 *** Monotone support
879
880 Nathaniel Smith has contributed initial support for the Monotone version
881 control system. The code still needs docs and tests, but on the other hand it
882 has been in use by the Monotone buildbot for a long time now, so it is
883 probably fairly stable.
884
885 *** Tinderbox support
886
887 Ben Hearsum and the Mozilla crew have contributed some classes to allow
888 Buildbot to work with Tinderbox clients. One piece is
889 buildbot.changes.bonsaipoller.BonsaiPoller, which is a ChangeSource that
890 polls a Bonsai server (which is a kind of web-vased viewcvs CGI script) to
891 discover source code changes. The other piece is
892 buildbot.status.tinderbox.TinderboxMailNotifier, which is a status plugin
893 that sends email in the same format as Tinderbox does, which allows a number
894 of Tinderbox tools to be driven by Buildbot instead.
895
896 *** SVN Poller
897
898 Niklaus Giger contributed a ChangeSource (buildbot.changes.svnpoller) which
899 polls a remote SVN repository on a periodic basis. This is useful when, for
900 whatever reason, you cannot add a post-commit hook script to the repository.
901 This obsoletes the external contrib/svn_watcher.py script.
902
903 ** notes for plugin developers
904
905 *** IStatusLog.readlines()
906
907 This new method makes it easier for a status plugin (or a
908 BuildStep.createSummary method) to walk through a StatusLog one line at a
909 time. For example, if you wanted to create an extra logfile that just
910 contained all the GCC warnings from the main log, you could use the
911 following:
912
913 def createSummary(self, log):
914 warnings = []
915 for line in log.readlines():
916 if "warning:" in line:
917 warnings.append()
918 self.addCompleteLog('warnings', "".join(warnings))
919
920 The "BuildStep LogFiles" section of the user's manual contains more
921 information. This method is not particularly memory-efficient yet (it reads
922 the whole logfile into memory first, then splits it into lines); this will be
923 improved in a future release.
924
925 ** bug fixes
926
927 *** Update source.SVN to work with the new SVN-1.4.0
928
929 The latest subversion changed the behavior in an unusual situation which
930 caused the unit tests to fail. This was unlikely to cause a problem in actual
931 usage, but the tests have been updated to pass with the new version.
932
933 *** update svn_buildbot.py to avoid mangling filenames
934
935 Older versions of this script were stripping the wrong number of columns from
936 the output of 'svnlook changed', and would sometimes mangle filenames. This
937 has been fixed. Closes SF#1545146.
938
939 *** logfiles= caused subsequent build failures under Windows
940
941 Earlier versions of buildbot didn't explicitly close any logfiles= file
942 handles when the build finished. On windows (where you cannot delete a file
943 that someone else is reading), this could cause the next build to fail as the
944 source checkout step was unable to delete the old working directory. This has
945 been fixed. Closes SF#1568415.
946
947 *** logfiles= didn't work on OS-X
948
949 Macintosh OS-X has a different behavior when reading files that have reached
950 EOF, the result was that logfiles= sometimes didn't work. Thanks to Mark Rowe
951 for the patch.
952
953 ** other changes
954
955 The 'buildbot sighup MASTERDIR' command has been replaced with 'buildbot
956 reconfig MASTERDIR', since that seems to be a slightly more meaningful name.
957 The 'sighup' form will remain as an alias.
958
959
960 * Release 0.7.4 (23 Aug 2006)
961
962 ** Things You Need To Know
963
964 The PBChangeSource's prefix= argument has changed, you probably need to add a
965 slash now. This is mostly used by sites which use Subversion and
966 svn_buildbot.py.
967
968 The subcommands that are used to create a buildmaster or a buildslave have
969 changed. They used to be called 'buildbot master' and 'buildbot slave'. Now
970 they are called 'buildbot create-master' and 'buildbot create-slave'. Zipf's
971 Law suggests that these are more appropriate names for these
972 infrequently-used commands.
973
974 The syntax for the c['manhole'] feature has changed.
975
976 ** new features
977
978 *** full Perforce support
979
980 SF#1473939: large patch from Scott Lamb, with docs and unit tests! This
981 includes both the step.P4 source-checkout BuildStep, and the changes.p4poller
982 ChangeSource you'll want to feed it. P4 is now supported just as well as all
983 the other VC systems. Thanks Scott!
984
985 *** SSH-based Manhole
986
987 The 'manhole' feature allows buildbot developers to get access to a python
988 read/eval/print loop (REPL) inside the buildmaster through a network
989 connection. Previously, this ran over unencrypted telnet, using a simple
990 username/password for access control. The new release defaults to encrypted
991 SSH access, using either username/password or an authorized_keys file (just
992 like sshd). There also exists an unencrypted telnet form, but its use is
993 discouraged. The syntax for setting up a manhole has changed, so master.cfg
994 files that use them must be updated. The "Debug options" section in the
995 user's manual provides a complete description.
996
997 *** Multiple Logfiles
998
999 BuildSteps can watch multiple log files in realtime, not just stdout/stderr.
1000 This works in a similar fashion to 'tail -f': the file is polled once per
1001 second, and any new data is sent to the buildmaster.
1002
1003 This requires a buildslave running 0.7.4 or later, and a warning message is
1004 produced if used against an old buildslave (which will otherwise produce no
1005 data). Use "logfiles={'name': 'filename'}" to take advantage of this feature
1006 from master.cfg, and see the "ShellCommand" section of the user's manual for
1007 full documentation.
1008
1009 The 'Trial' buildstep has been updated to use this, to display
1010 _trial_temp/test.log in realtime. It also knows to fall back to the previous
1011 "cat" command if the buildslave is too old.
1012
1013 *** BuildStep URLs
1014
1015 BuildSteps can now add arbitrary URLs which will be displayed on the
1016 Waterfall page in the same place that Logs are presented. This is intended to
1017 provide a link to generated HTML pages, such as the output of a code coverage
1018 tool. The step is responsible for somehow uploading the HTML to a web server:
1019 this feature merely provides an easy way to present the HREF link to the
1020 user. See the "BuildStep URLs" section of the user's manual for details and
1021 examples.
1022
1023 *** LogObservers
1024
1025 BuildSteps can now attach LogObservers to various logfiles, allowing them to
1026 get real-time log output. They can use this to watch for progress-indicating
1027 events (like counting the number of files compiled, or the number of tests
1028 which have run), and update both ETA/progress-tracking and step text. This
1029 allows for more accurate ETA information, and more information passed to the
1030 user about how much of the process has completed.
1031
1032 The 'Trial' buildstep has been updated to use this for progress tracking, by
1033 counting how many test cases have run.
1034
1035 ** new documentation
1036
1037 What classes are useful in your master.cfg file? A table of them has been
1038 added to the user's manual, in a section called "Index of Useful Classes".
1039
1040 Want a list of all the keys in master.cfg? Look in the "Index of master.cfg
1041 keys" section.
1042
1043 A number of pretty diagrams have been added to the "System Architecture"
1044 portion of the manual, explaining how all the buildbot pieces fit together.
1045
1046 An HTML form of the user's manual is now shipped in the source tarball. This
1047 makes it a bit bigger: sorry about that. The old PyCon-2003 paper has been
1048 removed from the distribution, as it is mostly supplanted by the user's
1049 manual by this point.
1050
1051 ** bugfixes
1052
1053 SF#1217699 + SF#1381867: The prefix= argument to PBChangeSource has been
1054 changed: now it does just a simple string-prefix match and strip. The
1055 previous behavior was buggy and unhelpful. NOTE: if you were using prefix=
1056 before, you probably need to add a slash to the end of it.
1057
1058 SF#1398174: ignore SVN property changes better, fixed by Olivier Bonnet
1059
1060 SF#1452801: don't double-escape the build URL, fixed by Olivier Bonnet
1061
1062 SF#1401121: add support for running py2exe on windows, by Mark Hammond
1063
1064 reloading unchanged config files with WithProperties shouldn't change anything.
1065
1066 All svn commands now include --non-interactive so they won't ask for
1067 passwords. Instead, the command will fail if it cannot be performed without
1068 user input.
1069
1070 Deprecation warnings with newer versions of Twisted have been hushed.
1071
1072 ** compatibility
1073
1074 I haven't actually removed support for Twisted-1.3.0 yet, but I'd like to.
1075
1076 The step_twisted default value for --reporter matches modern Twisteds,
1077 though, and won't work under 1.3.0.
1078
1079 ShellCommand.flunkOnFailure now defaults to True, so any shell command which
1080 fails counts as a build failure. Set this to False if you don't want this
1081 behavior.
1082
1083 ** minor features
1084
1085 contrib/darcs_buildbot.py contains a new script suitable for use in a darcs
1086 commit-hook.
1087
1088 Hovering a cursor over the yellow "Build #123" box in the Waterfall display
1089 will pop up an HTML tooltip to show the reason for the build. Thanks to Zandr
1090 Milewski for the suggestion.
1091
1092 contrib/CSS/*.css now contains several contributed stylesheets to make the
1093 Waterfall display a bit less ugly. Thanks to John O'Duinn for gathering them.
1094
1095 ShellCommand and its derivatives can now accept either a string or a list of
1096 strings in the description= and descriptionDone= arguments. Thanks to Paul
1097 Winkler for the catch.
1098
1099
1100 * Release 0.7.3 (23 May 2006)
1101
1102 ** compatibility
1103
1104 This release is compatible with Twisted-1.3.0, but the next one will not be.
1105 Please upgrade to at least Twisted-2.0.x soon, as the next buildbot release
1106 will require it.
1107
1108 ** new features
1109
1110 *** Mercurial support
1111
1112 Support for Mercurial version control system (http://selenic.com/mercurial)
1113 has been added. This adds a buildbot.process.step.Mercurial BuildStep. A
1114 suitable hook script to deliver changes to the buildmaster is still missing.
1115
1116 *** 'buildbot restart' command
1117
1118 The 'buildbot restart BASEDIR' command will perform a 'buildbot stop' and
1119 'buildbot start', and will attempt to wait for the buildbot process to shut
1120 down in between. This is useful when you need to upgrade the code on your
1121 buildmaster or buildslave and want to take it down for a minimum amount of
1122 time.
1123
1124 *** build properties
1125
1126 Each build now has a set of named "Build Properties", which can be set by
1127 steps and interpolated into ShellCommands. The 'revision' and 'got_revision'
1128 properties are the most interesting ones available at this point, and can be
1129 used e.g. to get the VC revision number into the filename of a generated
1130 tarball. See the user's manual section entited "Build Properties" for more
1131 details.
1132
1133 ** minor features
1134
1135 *** IRC now takes password= argument
1136
1137 Useful for letting your bot claim a persistent identity.
1138
1139 *** svn_buildbot.py is easier to modify to understand branches
1140 *** BuildFactory has a new .addStep method
1141 *** p4poller has new arguments
1142 *** new contrib scripts: viewcvspoll, svnpoller, svn_watcher
1143
1144 These poll an external VC repository to watch for changes, as opposed to
1145 adding a hook script to the repository that pushes changes into the
1146 buildmaster. This means higher latency but may be easier to configure,
1147 especially if you do not have authority on the repository host.
1148
1149 *** VC build property 'got_revision'
1150
1151 The 'got_revision' property reports what revision a VC step actually
1152 acquired, which may be useful to know when building from HEAD.
1153
1154 *** improved CSS in Waterfall
1155
1156 The Waterfall display has a few new class= tags, which may make it easier to
1157 write custom CSS to make it look prettier.
1158
1159 *** robots_txt= argument in Waterfall
1160
1161 You can now pass a filename to the robots_txt= argument, which will be served
1162 as the "robots.txt" file. This can be used to discourage search engine
1163 spiders from crawling through the numerous build-status pages.
1164
1165 ** bugfixes
1166
1167 *** tests more likely to pass on non-English systems
1168
1169 The unit test suite now sets $LANG='C' to make subcommands emit error
1170 messages in english instead of whatever native language is in use on the
1171 host. This improves the chances that the unit tests will pass on such
1172 systems. This affects certain VC-related subcommands too.
1173
1174 test_vc was assuming that the system time was expressed with a numeric
1175 timezone, which is not always the case, especially under windows. This
1176 probably works better now than it did before. This only affects the CVS
1177 tests.
1178
1179 'buildbot try' (for CVS) now uses UTC instead of the local timezone. The
1180 'got_revision' property is also expressed in UTC. Both should help deal with
1181 buggy versions of CVS that don't parse numeric timezones properly.
1182
1183
1184 * Release 0.7.2 (17 Feb 2006)
1185
1186 ** new features
1187
1188 *** all TCP port numbers in config file now accept a strports string
1189
1190 Sometimes it is useful to restrict certain TCP ports that the buildmaster
1191 listens on to use specific network interfaces. In particular, if the
1192 buildmaster and SVN repository live on the same machine, you may want to
1193 restrict the PBChangeSource to only listen on the loopback interface,
1194 insuring that no external entities can inject Changes into the buildbot.
1195 Likewise, if you are using something like Apache's reverse-proxy feature to
1196 provide access to the buildmaster's HTML status page, you might want to hide
1197 the real Waterfall port by having it only bind to the loopback interface.
1198
1199 To accomplish this, use a string like "tcp:12345:interface=127.0.0.1" instead
1200 of a number like 12345. These strings are called "strports specification
1201 strings", and are documented in twisted's twisted.application.strports module
1202 (you can probably type 'pydoc twisted.application.strports' to see this
1203 documentation). Pretty much everywhere the buildbot takes a port number will
1204 now accept a strports spec, and any bare numbers are translated into TCP port
1205 numbers (listening on all network interfaces) for compatibility.
1206
1207 *** buildslave --umask control
1208
1209 Twisted's daemonization utility (/usr/bin/twistd) automatically sets the
1210 umask to 077, which means that all files generated by both the buildmaster
1211 and the buildslave will only be readable by the account under which the
1212 respective daemon is running. This makes it unnecessarily difficult to share
1213 build products (e.g. by symlinking ~/public_html/current_docs/ to a directory
1214 within the slave's build directory where each build puts the results of a
1215 "make docs" step).
1216
1217 The 'buildbot slave <PARAMS>' command now accepts a --umask argument, which
1218 can be used to override the umask set by twistd. If you create the buildslave
1219 with '--umask=022', then all build products will be world-readable, making it
1220 easier for other processes (run under other accounts) to access them.
1221
1222 ** bug fixes
1223
1224 The 0.7.1 release had a bug whereby reloading the config file could break all
1225 configured Schedulers, causing them to raise an exception when new changes
1226 arrived but not actually schedule a new build. This has been fixed.
1227
1228 Fixed a bug which caused the AnyBranchScheduler to explode when branch==None.
1229 Thanks to Kevin Turner for the catch. I also think I fixed a bug whereby the
1230 TryScheduler would explode when it was given a Change (which it is supposed
1231 to simply ignore).
1232
1233 The Waterfall display now does more quoting of names (including Builder
1234 names, BuildStep names, etc), so it is more likely that these names can
1235 contain unusual characters like spaces, quotes, and slashes. There may still
1236 be some problems with these kinds of names, however.. please report any bugs
1237 to the mailing list.
1238
1239
1240 * Release 0.7.1 (26 Nov 2005)
1241
1242 ** new features
1243
1244 *** scheduler.Nightly
1245
1246 Dobes Vandermeer contributed a cron-style 'Nightly' scheduler. Unlike the
1247 more-primitive Periodic class (which only lets you specify the duration
1248 between build attempts), Nightly lets you schedule builds for specific times
1249 of day, week, month, or year. The interface is very much like the crontab(5)
1250 file. See the buildbot.scheduler.Nightly docstring for complete details.
1251
1252 ** minor new features
1253
1254 *** step.Trial can work with Trial from Twisted >2.1.0
1255
1256 The 'Trial' step now accepts the trialMode= argument, which should be a list
1257 of strings to be added to trial's argv array. This defaults to ["-to"], which
1258 is appropriate for the Trial that ships in Twisted-2.1.0 and earlier, and
1259 tells Trial to emit non-colorized verbose output. To use this step with
1260 trials from later versions of Twisted, this should be changed to
1261 ["--reporter=bwverbose"].
1262
1263 In addition, you can now set other Trial command-line parameters through the
1264 trialArgs= argument. This is a list of strings, and defaults to an empty list.
1265
1266 *** Added a 'resubmit this build' button to the web page
1267
1268 *** Make the VC-checkout step's description more useful
1269
1270 Added the word "[branch]" to the VC step's description (used in the Step's
1271 box on the Waterfall page, among others) when we're checking out a
1272 non-default branch. Also add "rNNN" where appropriate to indicate which
1273 revision is being checked out. Thanks to Brad Hards and Nathaniel Smith for
1274 the suggestion.
1275
1276 ** bugs fixed
1277
1278 Several patches from Dobes Vandermeer: Escape the URLs in email, in case they
1279 have spaces and such. Fill otherwise-empty <td> elements, as a workaround for
1280 buggy browsers that might optimize them away. Also use binary mode when
1281 opening status pickle files, to make windows work better. The
1282 AnyBranchScheduler now works even when you don't provide a fileIsImportant=
1283 argument.
1284
1285 Stringify the base revision before stuffing it into a 'try' jobfile, helping
1286 SVN and Arch implement 'try' builds better. Thanks to Steven Walter for the
1287 patch.
1288
1289 Fix the compare_attrs list in PBChangeSource, FreshCVSSource, and Waterfall.
1290 Before this, certain changes to these objects in the master.cfg file were
1291 ignored, such that you would have to stop and re-start the buildmaster to
1292 make them take effect.
1293
1294 The config file is now loaded serially, shutting down old (or replaced)
1295 Status/ChangeSource plugins before starting new ones. This fixes a bug in
1296 which changing an aspect of, say, the Waterfall display would cause an
1297 exception as both old and new instances fight over the same TCP port. This
1298 should also fix a bug whereby new Periodic Schedulers could fire a build
1299 before the Builders have finished being added.
1300
1301 There was a bug in the way Locks were handled when the config file was
1302 reloaded: changing one Builder (but not the others) and reloading master.cfg
1303 would result in multiple instances of the same Lock object, so the Locks
1304 would fail to prevent simultaneous execution of Builds or Steps. This has
1305 been fixed.
1306
1307 ** other changes
1308
1309 For a long time, certain StatusReceiver methods (like buildStarted and
1310 stepStarted) have been able to return another StatusReceiver instance
1311 (usually 'self') to indicate that they wish to subscribe to events within the
1312 new object. For example, if the buildStarted() method returns 'self', the
1313 status receiver will also receive events for the new build, like
1314 stepStarted() and buildETAUpdate(). Returning a 'self' from buildStarted() is
1315 equivalent to calling build.subscribe(self).
1316
1317 Starting with buildbot-0.7.1, this auto-subscribe convenience will also
1318 register to automatically unsubscribe the target when the build or step has
1319 finished, just as if build.unsubscribe(self) had been called. Also, the
1320 unsubscribe() method has been changed to not explode if the same receiver is
1321 unsubscribed multiple times. (note that it will still explode is the same
1322 receiver is *subscribed* multiple times, so please continue to refrain from
1323 doing that).
1324
1325
1326 * Release 0.7.0 (24 Oct 2005)
1327
1328 ** new features
1329
1330 *** new c['schedulers'] config-file element (REQUIRED)
1331
1332 The code which decides exactly *when* a build is performed has been massively
1333 refactored, enabling much more flexible build scheduling. YOU MUST UPDATE
1334 your master.cfg files to match: in general this will merely require you to
1335 add an appropriate c['schedulers'] entry. Any old ".treeStableTime" settings
1336 on the BuildFactory instances will now be ignored. The user's manual has
1337 complete details with examples of how the new Scheduler classes work.
1338
1339 *** c['interlocks'] removed, Locks and Dependencies now separate items
1340
1341 The c['interlocks'] config element has been removed, and its functionality
1342 replaced with two separate objects. Locks are used to tell the buildmaster
1343 that certain Steps or Builds should not run at the same time as other Steps
1344 or Builds (useful for test suites that require exclusive access to some
1345 external resource: of course the real fix is to fix the tests, because
1346 otherwise your developers will be suffering from the same limitations). The
1347 Lock object is created in the config file and then referenced by a Step
1348 specification tuple or by the 'locks' key of the Builder specification
1349 dictionary. Locks come in two flavors: MasterLocks are buildmaster-wide,
1350 while SlaveLocks are specific to a single buildslave.
1351
1352 When you want to have one Build run or not run depending upon whether some
1353 other set of Builds have passed or failed, you use a special kind of
1354 Scheduler defined in the scheduler.Dependent class. This scheduler watches an
1355 upstream Scheduler for builds of a given source version to complete, and only
1356 fires off its own Builders when all of the upstream's Builders have built
1357 that version successfully.
1358
1359 Both features are fully documented in the user's manual.
1360
1361 *** 'buildbot try'
1362
1363 The 'try' feature has finally been added. There is some configuration
1364 involved, both in the buildmaster config and on the developer's side, but
1365 once in place this allows the developer to type 'buildbot try' in their
1366 locally-modified tree and to be given a report of what would happen if their
1367 changes were to be committed. This works by computing a (base revision,
1368 patch) tuple that describes the developer's tree, sending that to the
1369 buildmaster, then running a build with that source on a given set of
1370 Builders. The 'buildbot try' tool then emits status messages until the builds
1371 have finished.
1372
1373 'try' exists to allow developers to run cross-platform tests on their code
1374 before committing it, reducing the chances they will inconvenience other
1375 developers by breaking the build. The UI is still clunky, but expect it to
1376 change and improve over the next few releases.
1377
1378 Instructions for developers who want to use 'try' (and the configuration
1379 changes necessary to enable its use) are in the user's manual.
1380
1381 *** Build-On-Branch
1382
1383 When suitably configured, the buildbot can be used to build trees from a
1384 variety of related branches. You can set up Schedulers to build a tree using
1385 whichever branch was last changed, or users can request builds of specific
1386 branches through IRC, the web page, or (eventually) the CLI 'buildbot force'
1387 subcommand.
1388
1389 The IRC 'force' command now takes --branch and --revision arguments (not that
1390 they always make sense). Likewise the HTML 'force build' button now has an
1391 input field for branch and revision. Your build's source-checkout step must
1392 be suitably configured to support this: for SVN it involves giving both a
1393 base URL and a default branch. Other VC systems are configured differently.
1394 The ChangeSource must also provide branch information: the 'buildbot
1395 sendchange' command now takes a --branch argument to help hook script writers
1396 accomplish this.
1397
1398 *** Multiple slaves per Builder
1399
1400 You can now attach multiple buildslaves to each Builder. This can provide
1401 redundancy or primitive load-balancing among many machines equally capable of
1402 running the build. To use this, define a key in the Builder specification
1403 dictionary named 'slavenames' with a list of buildslave names (instead of the
1404 usual 'slavename' that contains just a single slavename).
1405
1406 *** minor new features
1407
1408 The IRC and email status-reporting facilities now provide more specific URLs
1409 for particular builds, in addition to the generic buildmaster home page. The
1410 HTML per-build page now has more information.
1411
1412 The Twisted-specific test classes have been modified to match the argument
1413 syntax preferred by Trial as of Twisted-2.1.0 and newer. The generic trial
1414 steps are still suitable for the Trial that comes with older versions of
1415 Twisted, but may produce deprecation warnings or errors when used with the
1416 latest Trial.
1417
1418 ** bugs fixed
1419
1420 DNotify, used by the maildir-watching ChangeSources, had problems on some
1421 64-bit systems relating to signed-vs-unsigned constants and the DN_MULTISHOT
1422 flag. A workaround was provided by Brad Hards.
1423
1424 The web status page should now be valid XHTML, thanks to a patch by Brad
1425 Hards. The charset parameter is specified to be UTF-8, so VC comments,
1426 builder names, etc, should probably all be in UTF-8 to be displayed properly.
1427
1428 ** creeping version dependencies
1429
1430 The IRC 'force build' command now requires python2.3 (for the shlex.split
1431 function).
1432
1433
1434 * Release 0.6.6 (23 May 2005)
1435
1436 ** bugs fixed
1437
1438 The 'sendchange', 'stop', and 'sighup' subcommands were broken, simple bugs
1439 that were not caught by the test suite. Sorry.
1440
1441 The 'buildbot master' command now uses "raw" strings to create .tac files
1442 that will still function under windows (since we must put directory names
1443 that contain backslashes into that file).
1444
1445 The keep-on-disk behavior added in 0.6.5 included the ability to upgrade old
1446 in-pickle LogFile instances. This upgrade function was not added to the
1447 HTMLLogFile class, so an exception would be raised when attempting to load or
1448 display any build with one of these logs (which are normally used only for
1449 showing build exceptions). This has been fixed.
1450
1451 Several unnecessary imports were removed, so the Buildbot should function
1452 normally with just Twisted-2.0.0's "Core" module installed. (of course you
1453 will need TwistedWeb, TwistedWords, and/or TwistedMail if you use status
1454 targets that require them). The test suite should skip all tests that cannot
1455 be run because of missing Twisted modules.
1456
1457 The master/slave's basedir is now prepended to sys.path before starting the
1458 daemon. This used to happen implicitly (as a result of twistd's setup
1459 preamble), but 0.6.5 internalized the invocation of twistd and did not copy
1460 this behavior. This change restores the ability to access "private.py"-style
1461 modules in the basedir from the master.cfg file with a simple "import
1462 private" statement. Thanks to Thomas Vander Stichele for the catch.
1463
1464
1465 * Release 0.6.5 (18 May 2005)
1466
1467 ** deprecated config keys removed
1468
1469 The 'webPortnum', 'webPathname', 'irc', and 'manholePort' config-file keys,
1470 which were deprecated in the previous release, have now been removed. In
1471 addition, Builders must now always be configured with dictionaries: the
1472 support for configuring them with tuples has been removed.
1473
1474 ** master/slave creation and startup changed
1475
1476 The buildbot no longer uses .tap files to store serialized representations of
1477 the buildmaster/buildslave applications. Instead, this release now uses .tac
1478 files, which are human-readable scripts that create new instances (rather
1479 than .tap files, which were pickles of pre-created instances). 'mktap
1480 buildbot' is gone.
1481
1482 You will need to update your buildbot directories to handle this. The
1483 procedure is the same as creating a new buildmaster or buildslave: use
1484 'buildbot master BASEDIR' or 'buildbot slave BASEDIR ARGS..'. This will
1485 create a 'buildbot.tac' file in the target directory. The 'buildbot start
1486 BASEDIR' will use twistd to start the application.
1487
1488 The 'buildbot start' command now looks for a Makefile.buildbot, and if it
1489 finds one (and /usr/bin/make exists), it will use it to start the application
1490 instead of calling twistd directly. This allows you to customize startup,
1491 perhaps by adding environment variables. The setup commands create a sample
1492 file in Makefile.sample, but you must copy this to Makefile.buildbot to
1493 actually use it. The previous release looked for a bare 'Makefile', and also
1494 installed a 'Makefile', so you were always using the customized approach,
1495 even if you didn't ask for it. That old Makefile launched the .tap file, so
1496 changing names was also necessary to make sure that the new 'buildbot start'
1497 doesn't try to run the old .tap file.
1498
1499 'buildbot stop' now uses os.kill instead of spawning an external process,
1500 making it more likely to work under windows. It waits up to 5 seconds for the
1501 daemon to go away, so you can now do 'buildbot stop BASEDIR; buildbot start
1502 BASEDIR' with less risk of launching the new daemon before the old one has
1503 fully shut down. Likewise, 'buildbot start' imports twistd's internals
1504 directly instead of spawning an external copy, so it should work better under
1505 windows.
1506
1507 ** new documentation
1508
1509 All of the old Lore-based documents were converted into a new Texinfo-format
1510 manual, and considerable new text was added to describe the installation
1511 process. The docs are not yet complete, but they're slowly shaping up to form
1512 a proper user's manual.
1513
1514 ** new features
1515
1516 Arch checkouts can now use precise revision stamps instead of always using
1517 the latest revision. A separate Source step for using Bazaar (an alternative
1518 Arch client) instead of 'tla' was added. A Source step for Cogito (the new
1519 linux kernel VC system) was contributed by Brandon Philips. All Source steps
1520 now accept a retry= argument to indicate that failing VC checkouts should be
1521 retried a few times (SF#1200395), note that this requires an updated
1522 buildslave.
1523
1524 The 'buildbot sendchange' command was added, to be used in VC hook scripts to
1525 send changes at a pb.PBChangeSource . contrib/arch_buildbot.py was added to
1526 use this tool; it should be installed using the 'Arch meta hook' scheme.
1527
1528 Changes can now accept a branch= parameter, and Builders have an
1529 isBranchImportant() test that acts like isFileImportant(). Thanks to Thomas
1530 Vander Stichele. Note: I renamed his tag= to branch=, in anticipation of an
1531 upcoming feature to build specific branches. "tag" seemed too CVS-centric.
1532
1533 LogFiles have been rewritten to stream the incoming data directly to disk
1534 rather than keeping a copy in memory all the time (SF#1200392). This
1535 drastically reduces the buildmaster's memory requirements and makes 100MB+
1536 log files feasible. The log files are stored next to the serialized Builds,
1537 in files like BASEDIR/builder-dir/12-log-compile-output, so you'll want a
1538 cron job to delete old ones just like you do with old Builds. Old-style
1539 Builds from 0.6.4 and earlier are converted when they are first read, so the
1540 first load of the Waterfall display after updating to this release may take
1541 quite some time.
1542
1543 ** build process updates
1544
1545 BuildSteps can now return a status of EXCEPTION, which terminates the build
1546 right away. This allows exceptions to be caught right away, but still make
1547 sure the build stops quickly.
1548
1549 ** bug fixes
1550
1551 Some more windows incompatibilities were fixed. The test suite now has two
1552 failing tests remaining, both of which appear to be Twisted issues that
1553 should not affect normal operation.
1554
1555 The test suite no longer raises any deprecation warnings when run against
1556 twisted-2.0 (except for the ones which come from Twisted itself).
1557
1558
1559 * Release 0.6.4 (28 Apr 2005)
1560
1561 ** major bugs fixed
1562
1563 The 'buildbot' tool in 0.6.3, when used to create a new buildmaster, failed
1564 unless it found a 'changes.pck' file. As this file is created by a running
1565 buildmaster, this made 0.6.3 completely unusable for first-time
1566 installations. This has been fixed.
1567
1568 ** minor bugs fixed
1569
1570 The IRC bot had a bug wherein asking it to watch a certain builder (the "I'll
1571 give a shout when the build finishes" message) would cause an exception, so
1572 it would not, in fact, shout. The HTML page had an exception in the "change
1573 sources" page (reached by following the "Changes" link at the top of the
1574 column that shows the names of commiters). Re-loading the config file while
1575 builders were already attached would result in a benign error message. The
1576 server side of the PBListener status client had an exception when providing
1577 information about a non-existent Build (e.g., when the client asks for the
1578 Build that is currently running, and the server says "None").
1579
1580 These bugs have all been fixed.
1581
1582 The unit tests now pass under python2.2; they were failing before because of
1583 some 2.3isms that crept in. More unit tests which failed under windows now
1584 pass, only one (test_webPathname_port) is still failing.
1585
1586 ** 'buildbot' tool looks for a .buildbot/options file
1587
1588 The 'statusgui' and the 'debugclient' subcommands can both look for a
1589 .buildbot/ directory, and an 'options' file therein, to extract default
1590 values for the location of the buildmaster. This directory is searched in the
1591 current directory, its parent, etc, all the way up to the filesystem root
1592 (assuming you own the directories in question). It also look in ~/.buildbot/
1593 for this file. This feature allows you to put a .buildbot at the top of your
1594 working tree, telling any 'buildbot' invocations you perform therein how to
1595 get to the buildmaster associated with that tree's project.
1596
1597 Windows users get something similar, using %APPDATA%/buildbot instead of
1598 ~/.buildbot .
1599
1600 ** windows ShellCommands are launched with 'cmd.exe'
1601
1602 The buildslave has been modified to run all list-based ShellCommands by
1603 prepending [os.environ['COMSPEC'], '/c'] to the argv list before execution.
1604 This should allow the buildslave's PATH to be searched for commands,
1605 improving the chances that it can run the same 'trial -o foo' commands as a
1606 unix buildslave. The potential downside is that spaces in argv elements might
1607 be re-parsed, or quotes might be re-interpreted. The consensus on the mailing
1608 list was that this is a useful thing to do, but please report any problems
1609 you encounter with it.
1610
1611 ** minor features
1612
1613 The Waterfall display now shows the buildbot's home timezone at the top of
1614 the timestamp column. The default favicon.ico is now much nicer-looking (it
1615 is generated with Blender.. the icon.blend file is available in CVS in
1616 docs/images/ should you care to play with it).
1617
1618
1619
1620 * Release 0.6.3 (25 Apr 2005)
1621
1622 ** 'buildbot' tool gets more uses
1623
1624 The 'buildbot' executable has acquired three new subcommands. 'buildbot
1625 debugclient' brings up the small remote-control panel that connects to a
1626 buildmaster (via the slave port and the c['debugPassword']). This tool,
1627 formerly in contrib/debugclient.py, lets you reload the config file, force
1628 builds, and simulate inbound commit messages. It requires gtk2, glade, and
1629 the python bindings for both to be installed.
1630
1631 'buildbot statusgui' brings up a live status client, formerly available by
1632 running buildbot/clients/gtkPanes.py as a program. This connects to the PB
1633 status port that you create with:
1634
1635 c['status'].append(client.PBListener(portnum))
1636
1637 and shows two boxes per Builder, one for the last build, one for current
1638 activity. These boxes are updated in realtime. The effect is primitive, but
1639 is intended as an example of what's possible with the PB status interface.
1640
1641 'buildbot statuslog' provides a text-based running log of buildmaster events.
1642
1643 Note: command names are subject to change. These should get much more useful
1644 over time.
1645
1646 ** web page has a favicon
1647
1648 When constructing the html.Waterfall instance, you can provide the filename
1649 of an image that will be provided when the "favicon.ico" resource is
1650 requested. Many web browsers display this as an icon next to the URL or
1651 bookmark. A goofy little default icon is included.
1652
1653 ** web page has CSS
1654
1655 Thanks to Thomas Vander Stichele, the Waterfall page is now themable through
1656 CSS. The default CSS is located in buildbot/status/classic.css, and creates a
1657 page that is mostly identical to the old, non-CSS based table.
1658
1659 You can specify a different CSS file to use by passing it as the css=
1660 argument to html.Waterfall(). See the docstring for Waterfall for some more
1661 details.
1662
1663 ** builder "categories"
1664
1665 Thomas has added code which places each Builder in an optional "category".
1666 The various status targets (Waterfall, IRC, MailNotifier) can accept a list
1667 of categories, and they will ignore any activity in builders outside this
1668 list. This makes it easy to create some Builders which are "experimental" or
1669 otherwise not yet ready for the world to see, or indicate that certain
1670 builders should not harass developers when their tests fail, perhaps because
1671 the build slaves for them are not yet fully functional.
1672
1673 ** Deprecated features
1674
1675 *** defining Builders with tuples is deprecated
1676
1677 For a long time, the preferred way to define builders in the config file has
1678 been with a dictionary. The less-flexible old style of a 4-item tuple (name,
1679 slavename, builddir, factory) is now officially deprecated (i.e., it will
1680 emit a warning if you use it), and will be removed in the next release.
1681 Dictionaries are more flexible: additional keys like periodicBuildTime are
1682 simply unavailable to tuple-defined builders.
1683
1684 Note: it is a good idea to watch the logfile (usually in twistd.log) when you
1685 first start the buildmaster, or whenever you reload the config file. Any
1686 warnings or errors in the config file will be found there.
1687
1688 *** c['webPortnum'], c['webPathname'], c['irc'] are deprecated
1689
1690 All status reporters should be defined in the c['status'] array, using
1691 buildbot.status.html.Waterfall or buildbot.status.words.IRC . These have been
1692 deprecated for a while, but this is fair warning that these keys will be
1693 removed in the next release.
1694
1695 *** c['manholePort'] is deprecated
1696
1697 Again, this has been deprecated for a while, in favor of:
1698
1699 c['manhole'] = master.Manhole(port, username, password)
1700
1701 The preferred syntax will eventually let us use other, better kinds of debug
1702 shells, such as the experimental curses-based ones in the Twisted sandbox
1703 (which would offer command-line editing and history).
1704
1705 ** bug fixes
1706
1707 The waterfall page has been improved a bit. A circular-reference bug in the
1708 web page's TextLog class was fixed, which caused a major memory leak in a
1709 long-running buildmaster with large logfiles that are viewed frequently.
1710 Modifying the config file in a way which only changed a builder's base
1711 directory now works correctly. The 'buildbot' command tries to create
1712 slightly more useful master/slave directories, adding a Makefile entry to
1713 re-create the .tap file, and removing global-read permissions from the files
1714 that may contain buildslave passwords.
1715
1716 ** twisted-2.0.0 compatibility
1717
1718 Both buildmaster and buildslave should run properly under Twisted-2.0 . There
1719 are still some warnings about deprecated functions, some of which could be
1720 fixed, but there are others that would require removing compatibility with
1721 Twisted-1.3, and I don't expect to do that until 2.0 has been out and stable
1722 for at least several months. The unit tests should pass under 2.0, whereas
1723 the previous buildbot release had tests which could hang when run against the
1724 new "trial" framework in 2.0.
1725
1726 The Twisted-specific steps (including Trial) have been updated to match 2.0
1727 functionality.
1728
1729 ** win32 compatibility
1730
1731 Thankt to Nick Trout, more compatibility fixes have been incorporated,
1732 improving the chances that the unit tests will pass on windows systems. There
1733 are still some problems, and a step-by-step "running buildslaves on windows"
1734 document would be greatly appreciated.
1735
1736 ** API docs
1737
1738 Thanks to Thomas Vander Stichele, most of the docstrings have been converted
1739 to epydoc format. There is a utility in docs/gen-reference to turn these into
1740 a tree of cross-referenced HTML pages. Eventually these docs will be
1741 auto-generated and somehow published on the buildbot web page.
1742
1743
1744
1745 * Release 0.6.2 (13 Dec 2004)
1746
1747 ** new features
1748
1749 It is now possible to interrupt a running build. Both the web page and the
1750 IRC bot feature 'stop build' commands, which can be used to interrupt the
1751 current BuildStep and accelerate the termination of the overall Build. The
1752 status reporting for these still leaves something to be desired (an
1753 'interrupt' event is pushed into the column, and the reason for the interrupt
1754 is added to a pseudo-logfile for the step that was stopped, but if you only
1755 look at the top-level status it appears that the build failed on its own).
1756
1757 Builds are also halted if the connection to the buildslave is lost. On the
1758 slave side, any active commands are halted if the connection to the
1759 buildmaster is lost.
1760
1761 ** minor new features
1762
1763 The IRC log bot now reports ETA times in a MMSS format like "2m45s" instead
1764 of the clunky "165 seconds".
1765
1766 ** bug fixes
1767
1768 *** Slave Disconnect
1769
1770 Slave disconnects should be handled better now: the current build should be
1771 abandoned properly. Earlier versions could get into weird states where the
1772 build failed to finish, clogging the builder forever (or at least until the
1773 buildmaster was restarted).
1774
1775 In addition, there are weird network conditions which could cause a
1776 buildslave to attempt to connect twice to the same buildmaster. This can
1777 happen when the slave is sending large logfiles over a slow link, while using
1778 short keepalive timeouts. The buildmaster has been fixed to allow the second
1779 connection attempt to take precedence over the first, so that the older
1780 connection is jettisoned to make way for the newer one.
1781
1782 In addition, the buildslave has been fixed to be less twitchy about timeouts.
1783 There are now two parameters: keepaliveInterval (which is controlled by the
1784 mktap 'keepalive' argument), and keepaliveTimeout (which requires editing the
1785 .py source to change from the default of 30 seconds). The slave expects to
1786 see *something* from the master at least once every keepaliveInterval
1787 seconds, and will try to provoke a response (by sending a keepalive request)
1788 'keepaliveTimeout' seconds before the end of this interval just in case there
1789 was no regular traffic. Any kind of traffic will qualify, including
1790 acknowledgements of normal build-status updates.
1791
1792 The net result is that, as long as any given PB message can be sent over the
1793 wire in less than 'keepaliveTimeout' seconds, the slave should not mistakenly
1794 disconnect because of a timeout. There will be traffic on the wire at least
1795 every 'keepaliveInterval' seconds, which is what you want to pay attention to
1796 if you're trying to keep an intervening NAT box from dropping what it thinks
1797 is an abandoned connection. A quiet loss of connection will be detected
1798 within 'keepaliveInterval' seconds.
1799
1800 *** Large Logfiles
1801
1802 The web page rendering code has been fixed to deliver large logfiles in
1803 pieces, using a producer/consumer apparatus. This avoids the large spike in
1804 memory consumption when the log file body was linearized into a single string
1805 and then buffered in the socket's application-side transmit buffer. This
1806 should also avoid the 640k single-string limit for web.distrib servers that
1807 could be hit by large (>640k) logfiles.
1808
1809
1810
1811 * Release 0.6.1 (23 Nov 2004)
1812
1813 ** win32 improvements/bugfixes
1814
1815 Several changes have gone in to improve portability to non-unix systems. It
1816 should be possible to run a build slave under windows without major issues
1817 (although step-by-step documentation is still greatly desired: check the
1818 mailing list for suggestions from current win32 users).
1819
1820 *** PBChangeSource: use configurable directory separator, not os.sep
1821
1822 The PBChangeSource, which listens on a TCP socket for change notices
1823 delivered from tools like contrib/svn_buildbot.py, was splitting source
1824 filenames with os.sep . This is inappropriate, because those file names are
1825 coming from the VC repository, not the local filesystem, and the repository
1826 host may be running a different OS (with a different separator convention)
1827 than the buildmaster host. In particular, a win32 buildmaster using a CVS
1828 repository running on a unix box would be confused.
1829
1830 PBChangeSource now takes a sep= argument to indicate the separator character
1831 to use.
1832
1833 *** build saving should work better
1834
1835 windows cannot do the atomic os.rename() trick that unix can, so under win32
1836 the buildmaster falls back to save/delete-old/rename, which carries a slight
1837 risk of losing a saved build log (if the system were to crash between the
1838 delete-old and the rename).
1839
1840 ** new features
1841
1842 *** test-result tracking
1843
1844 Work has begun on fine-grained test-result handling. The eventual goal is to
1845 be able to track individual tests over time, and create problem reports when
1846 a test starts failing (which then are resolved when the test starts passing
1847 again). The first step towards this is an ITestResult interface, and code in
1848 the TrialTestParser to create such results for all non-passing tests (the
1849 ones for which Trial emits exception tracebacks).
1850
1851 These test results are currently displayed in a tree-like display in a page
1852 accessible from each Build's page (follow the numbered link in the yellow
1853 box at the start of each build to get there).
1854
1855 This interface is still in flux, as it really wants to be able to accomodate
1856 things like compiler warnings and tests that are skipped because of missing
1857 libraries or unsupported architectures.
1858
1859 ** bug fixes
1860
1861 *** VC updates should survive temporary failures
1862
1863 Some VC systems (CVS and SVN in particular) get upset when files are turned
1864 into directories or vice versa, or when repository items are moved without
1865 the knowledge of the VC system. The usual symptom is that a 'cvs update'
1866 fails where a fresh checkout succeeds.
1867
1868 To avoid having to manually intervene, the build slaves' VC commands have
1869 been refactored to respond to update failures by deleting the tree and
1870 attempting a full checkout. This may cause some unnecessary effort when,
1871 e.g., the CVS server falls off the net, but in the normal case it will only
1872 come into play when one of these can't-cope situations arises.
1873
1874 *** forget about an existing build when the slave detaches
1875
1876 If the slave was lost during a build, the master did not clear the
1877 .currentBuild reference, making that builder unavailable for later builds.
1878 This has been fixed, so that losing a slave should be handled better. This
1879 area still needs some work, I think it's still possible to get both the
1880 slave and the master wedged by breaking the connection at just the right
1881 time. Eventually I want to be able to resume interrupted builds (especially
1882 when the interruption is the result of a network failure and not because the
1883 slave or the master actually died).
1884
1885 *** large logfiles now consume less memory
1886
1887 Build logs are stored as lists of (type,text) chunks, so that
1888 stdout/stderr/headers can be displayed differently (if they were
1889 distinguishable when they were generated: stdout and stderr are merged when
1890 usePTY=1). For multi-megabyte logfiles, a large list with many short strings
1891 could incur a large overhead. The new behavior is to merge same-type string
1892 chunks together as they are received, aiming for a chunk size of about 10kb,
1893 which should bring the overhead down to a more reasonable level.
1894
1895 There remains an issue with actually delivering large logfiles over, say,
1896 the HTML interface. The string chunks must be merged together into a single
1897 string before delivery, which causes a spike in the memory usage when the
1898 logfile is viewed. This can also break twisted.web.distrib -type servers,
1899 where the underlying PB protocol imposes a 640k limit on the size of
1900 strings. This will be fixed (with a proper Producer/Consumer scheme) in the
1901 next release.
1902
1903
1904 * Release 0.6.0 (30 Sep 2004)
1905
1906 ** new features
1907
1908 *** /usr/bin/buildbot control tool
1909
1910 There is now an executable named 'buildbot'. For now, this just provides a
1911 convenient front-end to mktap/twistd/kill, but eventually it will provide
1912 access to other client functionality (like the 'try' builds, and a status
1913 client). Assuming you put your buildbots in /var/lib/buildbot/master/FOO,
1914 you can do 'buildbot create-master /var/lib/buildbot/master/FOO' and it will
1915 create the .tap file and set up a sample master.cfg for you. Later,
1916 'buildbot start /var/lib/buildbot/master/FOO' will start the daemon.
1917
1918
1919 *** build status now saved in external files, -shutdown.tap unnecessary
1920
1921 The status rewrite included a change to save all build status in a set of
1922 external files. These files, one per build, are put in a subdirectory of the
1923 master's basedir (named according to the 'builddir' parameter of the Builder
1924 configuration dictionary). This helps keep the buildmaster's memory
1925 consumption small: the (potentially large) build logs are kept on disk
1926 instead of in RAM. There is a small cache (2 builds per builder) kept in
1927 memory, but everything else lives on disk.
1928
1929 The big change is that the buildmaster now keeps *all* status in these
1930 files. It is no longer necessary to preserve the buildbot-shutdown.tap file
1931 to run a persistent buildmaster. The buildmaster may be launched with
1932 'twistd -f buildbot.tap' each time, in fact the '-n' option can be added to
1933 prevent twistd from automatically creating the -shutdown.tap file.
1934
1935 There is still one lingering bug with this change: the Expectations object
1936 for each builder (which records how long the various steps took, to provide
1937 an ETA value for the next time) is not yet saved. The result is that the
1938 first build after a restart will not provide an ETA value.
1939
1940 0.6.0 keeps status in a single file per build, as opposed to 0.5.0 which
1941 kept status in many subdirectories (one layer for builds, another for steps,
1942 and a third for logs). 0.6.0 will detect and delete these subdirectories as
1943 it overwrites them.
1944
1945 The saved builds are optional. To prevent disk usage from growing without
1946 bounds, you may want to set up a cron job to run 'find' and delete any which
1947 are too old. The status displays will happily survive without those saved
1948 build objects.
1949
1950 The set of recorded Changes is kept in a similar file named 'changes.pck'.
1951
1952
1953 *** source checkout now uses timestamp/revision
1954
1955 Source checkouts are now performed with an appropriate -D TIMESTAMP (for
1956 CVS) or -r REVISION (for SVN) marker to obtain the exact sources that were
1957 specified by the most recent Change going into the current Build. This
1958 avoids a race condition in which a change might be committed after the build
1959 has started but before the source checkout has completed, resulting in a
1960 mismatched set of source files. Such changes are now ignored.
1961
1962 This works by keeping track of repository-wide revision/transaction numbers
1963 (for version control systems that offer them, like SVN). The checkout or
1964 update is performed with the highest such revision number. For CVS (which
1965 does not have them), the timestamp of each commit message is used, and a -D
1966 argument is created to place the checkout squarely in the middle of the "tree
1967 stable timer"'s window.
1968
1969 This also provides the infrastructure for the upcoming 'try' feature. All
1970 source-checkout commands can now obtain a base revision marker and a patch
1971 from the Build, allowing certain builds to be performed on something other
1972 than the most recent sources.
1973
1974 See source.xhtml and steps.xhtml for details.
1975
1976
1977 *** Darcs and Arch support added
1978
1979 There are now build steps which retrieve a source tree from Darcs and Arch
1980 repositories. See steps.xhtml for details.
1981
1982 Preliminary P4 support has been added, thanks to code from Dave Peticolas.
1983 You must manually set up each build slave with an appropriate P4CLIENT: all
1984 buildbot does is run 'p4 sync' at the appropriate times.
1985
1986
1987 *** Status reporting rewritten
1988
1989 Status reporting was completely revamped. The config file now accepts a
1990 BuildmasterConfig['status'] entry, with a list of objects that perform status
1991 delivery. The old config file entries which controlled the web status port
1992 and the IRC bot have been deprecated in favor of adding instances to
1993 ['status']. The following status-delivery classes have been implemented, all
1994 in the 'buildbot.status' package:
1995
1996 client.PBListener(port, username, passwd)
1997 html.Waterfall(http_port, distrib_port)
1998 mail.MailNotifier(fromaddr, mode, extraRecipients..)
1999 words.IRC(host, nick, channels)
2000
2001 See the individual docstrings for details about how to use each one. You can
2002 create new status-delivery objects by following the interfaces found in the
2003 buildbot.interfaces module.
2004
2005
2006 *** BuildFactory configuration process changed
2007
2008 The basic BuildFactory class is now defined in buildbot.process.factory
2009 rather than buildbot.process.base, so you will have to update your config
2010 files. factory.BuildFactory is the base class, which accepts a list of Steps
2011 to run. See docs/factories.xhtml for details.
2012
2013 There are now easier-to-use BuildFactory classes for projects which use GNU
2014 Autoconf, perl's MakeMaker (CPAN), python's distutils (but no unit tests),
2015 and Twisted's Trial. Each one takes a separate 'source' Step to obtain the
2016 source tree, and then fills in the rest of the Steps for you.
2017
2018
2019 *** CVS/SVN VC steps unified, simplified
2020
2021 The confusing collection of arguments for the CVS step ('clobber=',
2022 'copydir=', and 'export=') have been removed in favor of a single 'mode'
2023 argument. This argument describes how you want to use the sources: whether
2024 you want to update and compile everything in the same tree (mode='update'),
2025 or do a fresh checkout and full build each time (mode='clobber'), or
2026 something in between.
2027
2028 The SVN (Subversion) step has been unified and accepts the same mode=
2029 parameter as CVS. New version control steps will obey the same interface.
2030
2031 Most of the old configuration arguments have been removed. You will need to
2032 update your configuration files to use the new arguments. See
2033 docs/steps.xhtml for a description of all the new parameters.
2034
2035
2036 *** Preliminary Debian packaging added
2037
2038 Thanks to the contributions of Kirill Lapshin, we can now produce .deb
2039 installer packages. These are still experimental, but they include init.d
2040 startup/shutdown scripts, which the the new /usr/bin/buildbot to invoke
2041 twistd. Create your buildmasters in /var/lib/buildbot/master/FOO, and your
2042 slaves in /var/lib/buildbot/slave/BAR, then put FOO and BAR in the
2043 appropriate places in /etc/default/buildbot . After that, the buildmasters
2044 and slaves will be started at every boot.
2045
2046 Pre-built .debs are not yet distributed. Use 'debuild -uc -us' from the
2047 source directory to create them.
2048
2049
2050 ** minor features
2051
2052
2053 *** Source Stamps
2054
2055 Each build now has a "source stamp" which describes what sources it used. The
2056 idea is that the sources for this particular build can be completely
2057 regenerated from the stamp. The stamp is a tuple of (revision, patch), where
2058 the revision depends on the VC system being used (for CVS it is either a
2059 revision tag like "BUILDBOT-0_5_0" or a datestamp like "2004/07/23", for
2060 Subversion it is a revision number like 11455). This must be combined with
2061 information from the Builder that is constant across all builds (something to
2062 point at the repository, and possibly a branch indicator for CVS and other VC
2063 systems that don't fold this into the repository string).
2064
2065 The patch is an optional unified diff file, ready to be applied by running
2066 'patch -p0 <PATCH' from inside the workdir. This provides support for the
2067 'try' feature that will eventually allow developers to run buildbot tests on
2068 their code before checking it in.
2069
2070
2071 *** SIGHUP causes the buildmaster's configuration file to be re-read
2072
2073 *** IRC bot now has 'watch' command
2074
2075 You can now tell the buildbot's IRC bot to 'watch <buildername>' on a builder
2076 which is currently performing a build. When that build is finished, the
2077 buildbot will make an announcement (including the results of the build).
2078
2079 The IRC 'force build' command will also announce when the resulting build has
2080 completed.
2081
2082
2083 *** the 'force build' option on HTML and IRC status targets can be disabled
2084
2085 The html.Waterfall display and the words.IRC bot may be constructed with an
2086 allowForce=False argument, which removes the ability to force a build through
2087 these interfaces. Future versions will be able to restrict this build-forcing
2088 capability to authenticated users. The per-builder HTML page no longer
2089 displays the 'Force Build' buttons if it does not have this ability. Thanks
2090 to Fred Drake for code and design suggestions.
2091
2092
2093 *** master now takes 'projectName' and 'projectURL' settings
2094
2095 These strings allow the buildbot to describe what project it is working for.
2096 At the moment they are only displayed on the Waterfall page, but in the next
2097 release they will be retrieveable from the IRC bot as well.
2098
2099
2100 *** survive recent (SVN) Twisted versions
2101
2102 The buildbot should run correctly (albeit with plenty of noisy deprecation
2103 warnings) under the upcoming Twisted-2.0 release.
2104
2105
2106 *** work-in-progress realtime Trial results acquisition
2107
2108 Jonathan Simms (<slyphon>) has been working on 'retrial', a rewrite of
2109 Twisted's unit test framework that will most likely be available in
2110 Twisted-2.0 . Although it is not yet complete, the buildbot will be able to
2111 use retrial in such a way that build status is reported on a per-test basis,
2112 in real time. This will be the beginning of fine-grained test tracking and
2113 Problem management, described in docs/users.xhtml .
2114
2115
2116 * Release 0.5.0 (22 Jul 2004)
2117
2118 ** new features
2119
2120 *** web.distrib servers via TCP
2121
2122 The 'webPathname' config option, which specifies a UNIX socket on which to
2123 publish the waterfall HTML page (for use by 'mktap web -u' or equivalent),
2124 now accepts a numeric port number. This publishes the same thing via TCP,
2125 allowing the parent web server to live on a separate machine.
2126
2127 This config option could be named better, but it will go away altogether in
2128 a few releases, when status delivery is unified. It will be replaced with a
2129 WebStatusTarget object, and the config file will simply contain a list of
2130 various kinds of status targets.
2131
2132 *** 'master.cfg' filename is configurable
2133
2134 The buildmaster can use a config file named something other than
2135 "master.cfg". Use the --config=foo.cfg option to mktap to control this.
2136
2137 *** FreshCVSSource now uses newcred (CVSToys >= 1.0.10)
2138
2139 The FreshCVSSource class now defaults to speaking to freshcvs daemons from
2140 modern CVSToys releases. If you need to use the buildbot with a daemon from
2141 CVSToys-1.0.9 or earlier, use FreshCVSSourceOldcred instead. Note that the
2142 new form only requires host/port/username/passwd: the "serviceName"
2143 parameter is no longer meaningful.
2144
2145 *** Builders are now configured with a dictionary, not a tuple
2146
2147 The preferred way to set up a Builder in master.cfg is to provide a
2148 dictionary with various keys, rather than a (non-extensible) 4-tuple. See
2149 docs/config.xhtml for details. The old tuple-way is still supported for now,
2150 it will probably be deprecated in the next release and removed altogether in
2151 the following one.
2152
2153 *** .periodicBuildTime is now exposed to the config file
2154
2155 To set a builder to run at periodic intervals, simply add a
2156 'periodicBuildTime' key to its master.cfg dictionary. Again, see
2157 docs/config.xhtml for details.
2158
2159 *** svn_buildbot.py adds --include, --exclude
2160
2161 The commit trigger script now gives you more control over which files are
2162 sent to the buildmaster and which are not.
2163
2164 *** usePTY is controllable at slave mktap time
2165
2166 The buildslaves usually run their child processes in a pty, which creates a
2167 process group for all the children, which makes it much easier to kill them
2168 all at once (i.e. if a test hangs). However this causes problems on some
2169 systems. Rather than hacking slavecommand.py to disable the use of these
2170 ptys, you can now create the slave's .tap file with --usepty=0 at mktap
2171 time.
2172
2173 ** Twisted changes
2174
2175 A summary of warnings (e.g. DeprecationWarnings) is provided as part of the
2176 test-case summarizer. The summarizer also counts Skips, expectedFailures,
2177 and unexpectedSuccesses, displaying the counts on the test step's event box.
2178
2179 The RunUnitTests step now uses "trial -R twisted" instead of "trial
2180 twisted.test", which is a bit cleaner. All .pyc files are deleted before
2181 starting trial, to avoid getting tripped up by deleted .py files.
2182
2183 ** documentation
2184
2185 docs/config.xhtml now describes the syntax and allowed contents of the
2186 'master.cfg' configuration file.
2187
2188 ** bugfixes
2189
2190 Interlocks had a race condition that could cause the lock to get stuck
2191 forever.
2192
2193 FreshCVSSource has a prefix= argument that was moderately broken (it used to
2194 only work if the prefix was a single directory component). It now works with
2195 subdirectories.
2196
2197 The buildmaster used to complain when it saw the "info" directory in a
2198 slave's workspace. This directory is used to publish information about the
2199 slave host and its administrator, and is not a leftover build directory as
2200 the complaint suggested. This complain has been silenced.
2201
2202
2203 * Release 0.4.3 (30 Apr 2004)
2204
2205 ** PBChangeSource made explicit
2206
2207 In 0.4.2 and before, an internal interface was available which allowed
2208 special clients to inject changes into the Buildmaster. This interface is
2209 used by the contrib/svn_buildbot.py script. The interface has been extracted
2210 into a proper PBChangeSource object, which should be created in the
2211 master.cfg file just like the other kinds of ChangeSources. See
2212 docs/sources.xhtml for details.
2213
2214 If you were implicitly using this change source (for example, if you use
2215 Subversion and the svn_buildbot.py script), you *must* add this source to
2216 your master.cfg file, or changes will not be delivered and no builds will be
2217 triggered.
2218
2219 The PBChangeSource accepts the same "prefix" argument as all other
2220 ChangeSources. For a SVN repository that follows the recommended practice of
2221 using "trunk/" for the trunk revisions, you probably want to construct the
2222 source like this:
2223
2224 source = PBChangeSource(prefix="trunk")
2225
2226 to make sure that the Builders are given sensible (trunk-relative)
2227 filenames for each changed source file.
2228
2229 ** Twisted changes
2230
2231 *** step_twisted.RunUnitTests can change "bin/trial"
2232
2233 The twisted RunUnitTests step was enhanced to let you run something other
2234 than "bin/trial", making it easier to use a buildbot on projects which use
2235 Twisted but aren't actually Twisted itself.
2236
2237 *** Twisted now uses Subversion
2238
2239 Now that Twisted has moved from CVS to SVN, the Twisted build processes have
2240 been modified to perform source checkouts from the Subversion repository.
2241
2242 ** minor feature additions
2243
2244 *** display Changes with HTML
2245
2246 Changes are displayed with a bit more pizazz, and a links= argument was
2247 added to allow things like ViewCVS links to be added to the display
2248 (although it is not yet clear how this argument should be used: the
2249 interface remains subject to change untill it has been documented).
2250
2251 *** display ShellCommand logs with HTML
2252
2253 Headers are in blue, stderr is in red (unless usePTY=1 in which case stderr
2254 and stdout are indistinguishable). A link is provided which returns the same
2255 contents as plain text (by appending "?text=1" to the URL).
2256
2257 *** buildslaves send real tracebacks upon error
2258
2259 The .unsafeTracebacks option has been turned on for the buildslaves,
2260 allowing them to send a full stack trace when an exception occurs, which is
2261 logged in the buildmaster's twistd.log file. This makes it much easier to
2262 determine what went wrong on the slave side.
2263
2264 *** BasicBuildFactory refactored
2265
2266 The BasicBuildFactory class was refactored to make it easier to create
2267 derivative classes, in particular the BasicSVN variant.
2268
2269 *** "ping buildslave" web button added
2270
2271 There is now a button on the "builder information" page that lets a web user
2272 initiate a ping of the corresponding build slave (right next to the button
2273 that lets them force a build). This was added to help track down a problem
2274 with the slave keepalives.
2275
2276 ** bugs fixed:
2277
2278 You can now have multiple BuildSteps with the same name (the names are used
2279 as hash keys in the data structure that helps determine ETA values for each
2280 step, the new code creates unique key names if necessary to avoid
2281 collisions). This means that, for example, you do not have to create a
2282 BuildStep subclass just to have two Compile steps in the same process.
2283
2284 If CVSToys is not installed, the tests that depend upon it are skipped.
2285
2286 Some tests in 0.4.2 failed because of a missing set of test files, they are
2287 now included in the tarball properly.
2288
2289 Slave keepalives should work better now in the face of silent connection
2290 loss (such as when an intervening NAT box times out the association), the
2291 connection should be reestablished in minutes instead of hours.
2292
2293 Shell commands on the slave are invoked with an argument list instead of the
2294 ugly and error-prone split-on-spaces approach. If the ShellCommand is given
2295 a string (instead of a list), it will fall back to splitting on spaces.
2296 Shell commands should work on win32 now (using COMSPEC instead of /bin/sh).
2297
2298 Buildslaves under w32 should theoretically work now, and one was running for
2299 the Twisted buildbot for a while until the machine had to be returned.
2300
2301 The "header" lines in ShellCommand logs (which include the first line, that
2302 displays the command being run, and the last, which shows its exit status)
2303 are now generated by the buildslave side instead of the local (buildmaster)
2304 side. This can provide better error handling and is generally cleaner.
2305 However, if you have an old buildslave (running 0.4.2 or earlier) and a new
2306 buildmaster, then neither end will generate these header lines.
2307
2308 CVSCommand was improved, in certain situations 0.4.2 would perform
2309 unnecessary checkouts (when an update would have sufficed). Thanks to Johan
2310 Dahlin for the patches. The status output was fixed as well, so that
2311 failures in CVS and SVN commands (such as not being able to find the 'svn'
2312 executable) make the step status box red.
2313
2314 Subversion support was refactored to make it behave more like CVS. This is a
2315 work in progress and will be improved in the next release.
2316
2317
2318 * Release 0.4.2 (08 Jan 2004)
2319
2320 ** test suite updated
2321
2322 The test suite has been completely moved over to Twisted's "Trial"
2323 framework, and all tests now pass. To run the test suite (consisting of 64
2324 tests, probably covering about 30% of BuildBot's logic), do this:
2325
2326 PYTHONPATH=. trial -v buildbot.test
2327
2328 ** Mail parsers updated
2329
2330 Several bugs in the mail-parsing code were fixed, allowing a buildmaster to
2331 be triggered by mail sent out by a CVS repository. (The Twisted Buildbot is
2332 now using this to trigger builds, as their CVS server machine is having some
2333 difficulties with FreshCVS). The FreshCVS mail format for directory
2334 additions appears to have changed recently: the new parser should handle
2335 both old and new-style messages.
2336
2337 A parser for Bonsai commit messages (buildbot.changes.mail.parseBonsaiMail)
2338 was contributed by Stephen Davis. Thanks Stephen!
2339
2340 ** CVS "global options" now available
2341
2342 The CVS build step can now accept a list of "global options" to give to the
2343 cvs command. These go before the "update"/"checkout" word, and are described
2344 fully by "cvs --help-options". Two useful ones might be "-r", which causes
2345 checked-out files to be read-only, and "-R", which assumes the repository is
2346 read-only (perhaps by not attempting to write to lock files).
2347
2348
2349 * Release 0.4.1 (09 Dec 2003)
2350
2351 ** MaildirSources fixed
2352
2353 Several bugs in MaildirSource made them unusable. These have been fixed (for
2354 real this time). The Twisted buildbot is using an FCMaildirSource while they
2355 fix some FreshCVS daemon problems, which provided the encouragement for
2356 getting these bugs fixed.
2357
2358 In addition, the use of DNotify (only available under linux) was somehow
2359 broken, possibly by changes in some recent version of Python. It appears to
2360 be working again now (against both python-2.3.3c1 and python-2.2.1).
2361
2362 ** master.cfg can use 'basedir' variable
2363
2364 As documented in the sample configuration file (but not actually implemented
2365 until now), a variable named 'basedir' is inserted into the namespace used
2366 by master.cfg . This can be used with something like:
2367
2368 os.path.join(basedir, "maildir")
2369
2370 to obtain a master-basedir-relative location.
2371
2372
2373 * Release 0.4.0 (05 Dec 2003)
2374
2375 ** newapp
2376
2377 I've moved the codebase to Twisted's new 'application' framework, which
2378 drastically cleans up service startup/shutdown just like newcred did for
2379 authorization. This is mostly an internal change, but the interface to
2380 IChangeSources was modified, so in the off chance that someone has written a
2381 custom change source, it may have to be updated to the new scheme.
2382
2383 The most user-visible consequence of this change is that now both
2384 buildmasters and buildslaves are generated with the standard Twisted 'mktap'
2385 utility. Basic documentation is in the README file.
2386
2387 Both buildmaster and buildslave .tap files need to be re-generated to run
2388 under the new code. I have not figured out the styles.Versioned upgrade path
2389 well enough to avoid this yet. Sorry.
2390
2391 This also means that both buildslaves and the buildmaster require
2392 Twisted-1.1.0 or later.
2393
2394 ** reloadable master.cfg
2395
2396 Most aspects of a buildmaster is now controlled by a configuration file
2397 which can be re-read at runtime without losing build history. This feature
2398 makes the buildmaster *much* easier to maintain.
2399
2400 In the previous release, you would create the buildmaster by writing a
2401 program to define the Builders and ChangeSources and such, then run it to
2402 create the .tap file. In the new release, you use 'mktap' to create the .tap
2403 file, and the only parameter you give it is the base directory to use. Each
2404 time the buildmaster starts, it will look for a file named 'master.cfg' in
2405 that directory and parse it as a python script. That script must define a
2406 dictionary named 'BuildmasterConfig' with various keys to define the
2407 builders, the known slaves, what port to use for the web server, what IRC
2408 channels to connect to, etc.
2409
2410 This config file can be re-read at runtime, and the buildmaster will compute
2411 the differences and add/remove services as necessary. The re-reading is
2412 currently triggered through the debug port (contrib/debugclient.py is the
2413 debug port client), but future releases will add the ability to trigger the
2414 reconfiguration by IRC command, web page button, and probably a local UNIX
2415 socket (with a helper script to trigger a rebuild locally).
2416
2417 docs/examples/twisted_master.cfg contains a sample configuration file, which
2418 also lists all the keys that can be set.
2419
2420 There may be some bugs lurking, such as re-configuring the buildmaster while
2421 a build is running. It needs more testing.
2422
2423 ** MaxQ support
2424
2425 Radix contributed some support scripts to run MaxQ test scripts. MaxQ
2426 (http://maxq.tigris.org/) is a web testing tool that allows you to record
2427 HTTP sessions and play them back.
2428
2429 ** Builders can now wait on multiple Interlocks
2430
2431 The "Interlock" code has been enhanced to allow multiple builders to wait on
2432 each one. This was done to support the new config-file syntax for specifying
2433 Interlocks (in which each interlock is a tuple of A and [B], where A is the
2434 builder the Interlock depends upon, and [B] is a list of builders that
2435 depend upon the Interlock).
2436
2437 "Interlock" is misnamed. In the next release it will be changed to
2438 "Dependency", because that's what it really expresses. A new class (probably
2439 called Interlock) will be created to express the notion that two builders
2440 should not run at the same time, useful when multiple builders are run on
2441 the same machine and thrashing results when several CPU- or disk- intensive
2442 compiles are done simultaneously.
2443
2444 ** FreshCVSSource can now handle newcred-enabled FreshCVS daemons
2445
2446 There are now two FreshCVSSource classes: FreshCVSSourceNewcred talks to
2447 newcred daemons, and FreshCVSSourceOldcred talks to oldcred ones. Mind you,
2448 FreshCVS doesn't yet do newcred, but when it does, we'll be ready.
2449
2450 'FreshCVSSource' maps to the oldcred form for now. That will probably change
2451 when the current release of CVSToys supports newcred by default.
2452
2453 ** usePTY=1 on posix buildslaves
2454
2455 When a buildslave is running under POSIX (i.e. pretty much everything except
2456 windows), child processes are created with a pty instead of separate
2457 stdin/stdout/stderr pipes. This makes it more likely that a hanging build
2458 (when killed off by the timeout code) will have all its sub-childred cleaned
2459 up. Non-pty children would tend to leave subprocesses running because the
2460 buildslave was only able to kill off the top-level process (typically
2461 'make').
2462
2463 Windows doesn't have any concept of ptys, so non-posix systems do not try to
2464 enable them.
2465
2466 ** mail parsers should actually work now
2467
2468 The email parsing functions (FCMaildirSource and SyncmailMaildirSource) were
2469 broken because of my confused understanding of how python class methods
2470 work. These sources should be functional now.
2471
2472 ** more irc bot sillyness
2473
2474 The IRC bot can now perform half of the famous AYBABTO scene.
2475
2476
2477 * Release 0.3.5 (19 Sep 2003)
2478
2479 ** newcred
2480
2481 Buildbot has moved to "newcred", a new authorization framework provided by
2482 Twisted, which is a good bit cleaner and easier to work with than the
2483 "oldcred" scheme in older versions. This causes both buildmaster and
2484 buildslaves to depend upon Twisted 1.0.7 or later. The interface to
2485 'makeApp' has changed somewhat (the multiple kinds of remote connections all
2486 use the same TCP port now).
2487
2488 Old buildslaves will get "_PortalWrapper instance has no attribute
2489 'remote_username'" errors when they try to connect. They must be upgraded.
2490
2491 The FreshCVSSource uses PB to connect to the CVSToys server. This has been
2492 upgraded to use newcred too. If you get errors (TODO: what do they look
2493 like?) in the log when the buildmaster tries to connect, you need to upgrade
2494 your FreshCVS service or use the 'useOldcred' argument when creating your
2495 FreshCVSSource. This is a temporary hack to allow the buildmaster to talk to
2496 oldcred CVSToys servers. Using it will trigger deprecation warnings. It will
2497 go away eventually.
2498
2499 In conjunction with this change, makeApp() now accepts a password which can
2500 be applied to the debug service.
2501
2502 ** new features
2503
2504 *** "copydir" for CVS checkouts
2505
2506 The CVS build step can now accept a "copydir" parameter, which should be a
2507 directory name like "source" or "orig". If provided, the CVS checkout is
2508 done once into this directory, then copied into the actual working directory
2509 for compilation etc. Later updates are done in place in the copydir, then
2510 the workdir is replaced with a copy.
2511
2512 This reduces CVS bandwidth (update instead of full checkout) at the expense
2513 of twice the disk space (two copies of the tree).
2514
2515 *** Subversion (SVN) support
2516
2517 Radix (Christopher Armstrong) contributed early support for building
2518 Subversion-based trees. The new 'SVN' buildstep behaves roughly like the
2519 'CVS' buildstep, and the contrib/svn_buildbot.py script can be used as a
2520 checkin trigger to feed changes to a running buildmaster.
2521
2522 ** notable bugfixes
2523
2524 *** .tap file generation
2525
2526 We no longer set the .tap filename, because the buildmaster/buildslave
2527 service might be added to an existing .tap file and we shouldn't presume to
2528 own the whole thing. You may want to manually rename the "buildbot.tap" file
2529 to something more meaningful (like "buildslave-bot1.tap").
2530
2531 *** IRC reconnect
2532
2533 If the IRC server goes away (it was restarted, or the network connection was
2534 lost), the buildmaster will now schedule a reconnect attempt.
2535
2536 *** w32 buildslave fixes
2537
2538 An "rm -rf" was turned into shutil.rmtree on non-posix systems.
2539
2540
2541 * Release 0.3.4 (28 Jul 2003)
2542
2543 ** IRC client
2544
2545 The buildmaster can now join a set of IRC channels and respond to simple
2546 queries about builder status.
2547
2548 ** slave information
2549
2550 The build slaves can now report information from a set of info/* files in
2551 the slave base directory to the buildmaster. This will be used by the slave
2552 administrator to announce details about the system hosting the slave,
2553 contact information, etc. For now, info/admin should contain the name/email
2554 of the person who is responsible for the buildslave, and info/host should
2555 describe the system hosting the build slave (OS version, CPU speed, memory,
2556 etc). The contents of these files are made available through the waterfall
2557 display.
2558
2559 ** change notification email parsers
2560
2561 A parser for Syncmail (syncmail.sourceforge.net) was added. SourceForge
2562 provides examples of setting up syncmail to deliver CVS commit messages to
2563 mailing lists, so hopefully this will make it easier for sourceforge-hosted
2564 projects to set up a buildbot.
2565
2566 email processors were moved into buildbot.changes.mail . FCMaildirSource was
2567 moved, and the compatibility location (buildbot.changes.freshcvsmail) will
2568 go away in the next release.
2569
2570 ** w32 buildslave ought to work
2571
2572 Some non-portable code was changed to make it more likely that the
2573 buildslave will run under windows. The Twisted buildbot now has a
2574 (more-or-less) working w32 buildslave.
2575
2576
2577 * Release 0.3.3 (21 May 2003):
2578
2579 ** packaging changes
2580
2581 *** include doc/examples in the release. Oops again.
2582
2583 ** network changes
2584
2585 *** add keepalives to deal with NAT boxes
2586
2587 Some NAT boxes drop port mappings if the TCP connection looks idle for too
2588 long (maybe 30 minutes?). Add application-level keepalives (dummy commands
2589 sent from slave to master every 10 minutes) to appease the NAT box and keep
2590 our connection alive. Enable this with --keepalive in the slave mktap
2591 command line. Check the README for more details.
2592
2593 ** UI changes
2594
2595 *** allow slaves to trigger any build that they host
2596
2597 Added an internal function to ask the buildmaster to start one of their
2598 builds. Must be triggered with a debugger or manhole on the slave side for
2599 now, will add a better UI later.
2600
2601 *** allow web page viewers to trigger any build
2602
2603 Added a button to the per-build page (linked by the build names on the third
2604 row of the waterfall page) to allow viewers to manually trigger builds.
2605 There is a field for them to indicate who they are and why they are
2606 triggering the build. It is possible to abuse this, but for now the benefits
2607 outweigh the damage that could be done (worst case, someone can make your
2608 machine run builds continuously).
2609
2610 ** generic buildprocess changes
2611
2612 *** don't queue multiple builds for offline slaves
2613
2614 If a slave is not online when a build is ready to run, that build is queued
2615 so the slave will run it when it next connects. However, the buildmaster
2616 used to queue every such build, so the poor slave machine would be subject
2617 to tens or hundreds of builds in a row when they finally did come online.
2618 The buildmaster has been changed to merge these multiple builds into a
2619 single one.
2620
2621 *** bump ShellCommand default timeout to 20 minutes
2622
2623 Used for testing out the win32 twisted builder. I will probably revert this
2624 in the next relese.
2625
2626 *** split args in ShellCommand ourselves instead of using /bin/sh
2627
2628 This should remove the need for /bin/sh on the slave side, improving the
2629 chances that the buildslave can run on win32.
2630
2631 *** add configureEnv argument to Configure step, pass env dict to slave
2632
2633 Allows build processes to do things like 'CFLAGS=-O0 ./configure' without
2634 using /bin/sh to set the environment variable
2635
2636 ** Twisted buildprocess changes
2637
2638 *** warn instead of flunk the build when cReactor or qtreactor tests fail
2639
2640 These two always fail. For now, downgrade those failures to a warning
2641 (orange box instead of red).
2642
2643 *** don't use 'clobber' on remote builds
2644
2645 Builds that run on remote machines (freebsd, OS-X) now use 'cvs update'
2646 instead of clobbering their trees and doing a fresh checkout. The multiple
2647 simultaneous CVS checkouts were causing a strain on Glyph's upstream
2648 bandwidth.
2649
2650 *** use trial --testmodule instead of our own test-case-name grepper
2651
2652 The Twisted coding/testing convention has developers put 'test-case-name'
2653 tags (emacs local variables, actually) in source files to indicate which
2654 test cases should be run to exercise that code. Twisted's unit-test
2655 framework just acquired an argument to look for these tags itself. Use that
2656 instead of the extra FindUnitTestsForFiles build step we were doing before.
2657 Removes a good bit of code from buildbot and into Twisted where it really
2658 belongs.
2659
2660
2661 * Release 0.3.2 (07 May 2003):
2662
2663 ** packaging changes
2664
2665 *** fix major packaging bug: none of the buildbot/* subdirectories were
2666 included in the 0.3.1 release. Sorry, I'm still figuring out distutils
2667 here..
2668
2669 ** internal changes
2670
2671 *** use pb.Cacheable to update Events in remote status client. much cleaner.
2672
2673 *** start to clean up BuildProcess->status.builder interface
2674
2675 ** bug fixes
2676
2677 *** waterfall display was missing a <tr>, causing it to be misrendered in most
2678 browsers (except the one I was testing it with, of course)
2679
2680 *** URL without trailing slash (when served in a twisted-web distributed
2681 server, with a url like "http://twistedmatrix.com/~warner.twistd") should do
2682 redirect to URL-with-trailing-slash, otherwise internal hrefs are broken.
2683
2684 *** remote status clients: forget RemoteReferences at shutdown, removes
2685 warnings about "persisting Ephemerals"
2686
2687 ** Twisted buildprocess updates:
2688
2689 *** match build process as of twisted-1.0.5
2690 **** use python2.2 everywhere now that twisted rejects python2.1
2691 **** look for test-result constants in multiple places
2692 *** move experimental 'trial --jelly' code to separate module
2693 *** add FreeBSD builder
2694 *** catch rc!=0 in HLint step
2695 *** remove RunUnitTestsRandomly, use randomly=1 parameter instead
2696 *** parameterize ['twisted.test'] default test case to make subclassing easier
2697 *** ignore internal distutils warnings in python2.3 builder
2698
2699
2700 * Release 0.3.1 (29 Apr 2003):
2701
2702 ** First release.
2703
2704 ** Features implemented:
2705
2706 change notification from FreshCVS server or parsed maildir contents
2707
2708 timed builds
2709
2710 basic builds, configure/compile/test
2711
2712 some Twisted-specific build steps: docs, unit tests, debuild
2713
2714 status reporting via web page
2715
2716 ** Features still experimental/unpolished
2717
2718 status reporting via PB client
OLDNEW
« no previous file with comments | « third_party/buildbot_7_12/buildbot/CREDITS ('k') | third_party/buildbot_7_12/buildbot/README » ('j') | no next file with comments »

Powered by Google App Engine
This is Rietveld 408576698