index.rst

Release Notes for Buildbot

The following are the release notes for Buildbot .

Master

This version represents a refactoring of Buildbot into a consistent, well-defined application composed of loosely coupled components. The components are linked by a common database backend and a messaging system. This allows components to be distributed across multiple build masters. It also allows the rendering of complex web status views to be performed in the browser, rather than on the buildmasters.

The branch looks forward to committing to long-term API compatibility, but does not reach that goal. The Buildbot-0.9.x series of releases will give the new APIs time to "settle in" before we commit to them. Commitment will wait for Buildbot-1.0.0 (as per http://semver.org). Once Buildbot reaches version 1.0.0, upgrades will become much easier for users.

To encourage contributions from a wider field of developers, the web application is designed to look like a normal AngularJS application. Developers familiar with AngularJS, but not with Python, should be able to start hacking on the web application quickly. The web application is "pluggable", so users who develop their own status displays can package those separately from Buildbot itself.

Other goals:

• An approachable HTTP REST API, used by the web application but available for any other purpose.
• A high degree of coverage by reliable, easily-modified tests.
• "Interlocking" tests to guarantee compatibility. For example, the real and fake DB implementations must both pass the same suite of tests. Then no unseen difference between the fake and real implementations can mask errors that will occur in production.

Requirements

The buildbot-master package requires Python 2.6 -- Python 2.5 is no longer supported. The buildbot-slave package continues to support Python 2.4 through Python 2.7.

No additional software or systems, aside from some minor Python packages, are required.

But the devil is in the details:

• If you want to do web development, or build the buildbot-www package, you'll need Node. It's an Angular app, and that's how such apps are developed. We've taken pains to not make either a requirement for users - you can simply 'pip install' buildbot-www and be on your way. This is the case even if you're hacking on the Python side of Buildbot.
• For a single master, nothing else is required.
• If you want multiple masters, you'll need an external message broker of some sort. Messaging is based on Kombu, and supports the backends that Kombu supports.

Minor Python Packages

• Buildbot requires at least Twisted-11.0.0.

Features

• Both the P4 source step and P4 change source support ticket-based authentication.

• Buildbot now supports plugins. They allow Buildbot to be extended by using components distributed independently from the main code. They also provide for a unified way to access all components. When previously the following construction was used:

  from buildbot.kind.other.bits import ComponentClass
  
  ... ComponentClass ...

  the following construction achieves the same result:

  from buildbot.plugins import kind
  
  ... kind.ComponentClass ...

  Kinds of components that are available this way are described in ../manual/plugins.

  Note

  While the components can be still directly imported as buildbot.kind.other.bits, this might not be the case after Buildbot v1.0 is released.

• ~buildbot.status.status_gerrit.GerritStatusPush supports specifying an SSH identity file explicitly.

Fixes

• Buildbot is now compatible with SQLAlchemy 0.8 and higher, using the newly-released SQLAlchemy-Migrate.
• The :bbHTTPStep step's requeset parameters are now renderable.
• With Git(), force the updating submodules to ensure local changes by the build are overwitten. This both ensures more consistent builds and avoids errors when updating submodules.

• Buildbot is now compatible with Gerrit v2.6 and higher.

  To make this happen, the return result of reviewCB and summaryCB callback has changed from

  (message, verified, review)

  to

  {'message': message,
   'labels': {'label-name': value,
             ...
             }
  }

  The implications are:

  • there are some differences in behaviour: only those labels that were provided will be updated
  • Gerrit server must be able to provide a version, if it can't the :bbGerritStatusPush will not work

  Note

  If you have an old style reviewCB and/or summaryCB implemented, these will still work, however there could be more labels updated than anticipated.

  More detailed information is available in :bbGerritStatusPush section.

• :bbP4Source's server_tz parameter now works correctly.
• The revlink in changes broduced by the Bitbucket hook now correctly includes the changes/ portion of the URL.

• :bbPBChangeSource's git hook contrib/git_buildbot.py now supports git tags

  A pushed git tag generates a change event with the branch property equal to the tag name. To schedule builds based on buildbot tags, one could use something like this:

  c['schedulers'].append(
     SingleBranchScheduler(name='tags',
        change_filter=filter.ChangeFilter(
           branch_re='v[0-9]+\.[0-9]+\.[0-9]+(?:-pre|rc[0-9]+|p[0-9]+)?')
        treeStableTimer=None,
        builderNames=['tag_build']))

Deprecations, Removals, and Non-Compatible Changes

WebStatus

The old, clunky WebStatus has been removed. You will like the new interface! RIP WebStatus, you were a good friend.

If you have code like this in your configuration (and you probably do!)

from buildbot.status import html
c['status'].append(html.WebStatus(http_port=8010, allowForce=True)

remove it and replace it with :bbwww configuration <www>.

Requirements

• Buildbot's tests now require at least Mock-0.8.0.
• SQLAlchemy-Migrate-0.6.1 is no longer supported.
• Bulider names are now restricted to unicode strings or ASCII bytestrings. Encoded bytestrings are not accepted.

Steps

• Old-style source steps (imported directly from buildbot.steps.source) are no longer supported on the master.

Changes and Removals

• Buildslave names must now be 50-character identifier <type-identifier>. Note that this disallows some common characters in bulidslave names, including spaces, /, and ..
• Builders now have "tags" instead of a category. Builders can have multiple tags, allowing more flexible builder displays.
• :bbForceScheduler has the following changes:
  • The default configuration no longer contains four AnyPropertyParameter instances.
  • Configuring codebases is now mandatory, and the deprecated branch, repository, project, revision are not supported anymore in ForceScheduler
  • :pybuildbot.schedulers.forcesched.BaseParameter.updateFromKwargs now takes a collector parameter used to collect all validation errors
• Logs are now stored as Unicode strings, and thus must be decoded properly from the bytestrings provided by shell commands. By default this encoding is assumed to be UTF-8, but the :bblogEncoding parameter can be used to select an alternative. Steps and individual logfiles can also override the global default.

• The PB status service uses classes which have now been removed, and anyway is redundant to the REST API, so it has been removed. It has taken the following with it:

  • buildbot statuslog
  • buildbot statusgui (the GTK client)
  • buildbot debugclient

  The PBListener status listener is now deprecated and does nothing. Accordingly, there is no external access to status objects via Perspective Broker, aside from some compatibility code for the try scheduler.

  The debugPassword configuration option is no longer needed and is thus deprecated.

• The undocumented and un-tested TinderboxMailNotifier, designed to send emails suitable for the abandoned and insecure Tinderbox tool, has been removed.
• Buildslave info is no longer available via Interpolate and the SetSlaveInfo buildstep has been removed.

Changes for Developers

• The sourcestamp DB connector now returns a patchid field.
• Buildbot no longer polls the database for jobs. The db_poll_interval configuration parameter and the :bbdb key of the same name are deprecated and will be ignored.

• The interface for adding changes has changed. The new method is master.data.updates.addChange (implemented by :py~buildbot.data.changes.ChangeResourceType.addChange), although the old interface (master.addChange) will remain in place for a few versions. The new method:

  • returns a change ID, not a Change instance;
  • takes its when_timestamp argument as epoch time (UNIX time), not a datetime instance; and
  • does not accept the deprecated parameters who, isdir, is_dir, and when.
  • requires that all strings be unicode, not bytestrings.

  Please adjust any custom change sources accordingly.

• A new build status, CANCELLED, has been added. It is used when a step or build is deliberately cancelled by a user.
• This upgrade will delete all rows from the buildrequest_claims table. If you are using this table for analytical purposes outside of Buildbot, please back up its contents before the upgrade, and restore it afterward, translating object IDs to scheduler IDs if necessary. This translation would be very slow and is not required for most users, so it is not done automatically.
• All of the schedulers DB API methods now accept a schedulerid, rather than an objectid. If you have custom code using these methods, check your code and make the necessary adjustments.
• The addBuildsetForSourceStamp method has become addBuildsetForSourceStamps, and its signature has changed. The addBuildsetForSourceStampSetDetails method has become addBuildsetForSourceStampsWithDefaults, and its signature has changed. The addBuildsetForSourceStampDetails method has been removed. The addBuildsetForLatest method has been removed. It is equivalent to addBuildsetForSourceStampDetails with sourcestamps=None. These methods are not yet documented, and their interface is not stable. Consult the source code for details on the changes.
• The triggerable schedulers` trigger method now requires a list of sourcestamps, rather than a dictionary.
• The :py~buildbot.sourcestamp.SourceStamp class is no longer used. It remains in the codebase to support loading data from pickles on upgrade, but should not be used in running code.
• The :py~buildbot.process.buildrequest.BuildRequest class no longer has full source or sources attributes. Use the data API to get this information (which is associated with the buildset, not the build request) instead.
• The undocumented BuilderControl method submitBuildRequest has been removed.
• The debug client no longer supports requesting builds (the requestBuild method has been removed). If you have been using this method in production, consider instead creating a new change source, using the :bbForceScheduler, or using one of the try schedulers.
• The buildbot.misc.SerializedInvocation class has been removed; use :pybuildbot.util.debounce.method instead.

Slave

Features

Fixes

Deprecations, Removals, and Non-Compatible Changes

• While the buildmaster no longer supports old-style source steps, the buildslave still does. Buildbot-slave-0.9.0 will still run successfully against Buildbot-0.8.9 or earlier, configured to use old-style steps. However, the support is stlil deprecated and will be removed as soon as it is inconvenient for developers.
• On Windows, if a :bbShellCommand step in which command was specified as a list is executed, and a list element is a string consisting of a single pipe character, it no longer creates a pipeline. Instead, the pipe character is passed verbatim as an argument to the program, like any other string. This makes command handling consistent between Windows and Unix-like systems. To have a pipeline, specify command as a string.

Details

For a more detailed description of the changes made in this version, see the git log itself:

git log v0.8.9..master

Older Versions

Release notes for older versions of Buildbot are available in the master/docs/relnotes/ directory of the source tree. Newer versions are also available here:

0.8.9 0.8.8 0.8.7 0.8.6