index.rst

Release Notes for Buildbot

The following are the release notes for Buildbot .

Master

Features

• The following optional parameters have been added to :pyEC2LatentBuildSlave

  • Boolean parameter spot_instance, default False, creates a spot instance.
  • Float parameter max_spot_price defines the maximum bid for a spot instance.
  • List parameter volumes, takes a list of (volume_id, mount_point) tuples.
  • String parameter placement is appended to the region parameter, e.g. region='us-west-2', placement='b' will result in the spot request being placed in us-west-2b.
  • Float parameter price_multiplier specifies the percentage bid above the 24-hour average spot price.
  • Dict parameter tags specifies AWS tags as key/value pairs to be applied to new instances.

  With spot_instance=True, an EC2LatentBuildSlave will attempt to create a spot instance with the provided spot price, placement, and so on.

• The attributes description, descriptionDone and descriptionSuffix have been moved from :pyShellCommand to its superclass :pyBuildStep so that any class that inherits from :pyBuildStep can provide a suitable description of itself.
• A new :pyFlattenList Renderable has been added which can flatten nested lists.
• Builder configurations can now include a description, which will appear in the web UI to help humans figure out what the builder does.
• The web UI now supports a PNG Status Resource that can be accessed publicly from for example README.md files or wikis or whatever other resource. This view produces an image in PNG format with information about the last build for the given builder name or whatever other build number if is passed as an argument to the view.
• The web hooks now include support for Bitbucket and GitLab.
• The 'Rebuild' button on the web pages for builds features a dropdown to choose whether to rebuild from exact revisions or from the same sourcestamps (ie, update branch references)
• The start, restart, and reconfig commands will now wait for longer than 10 seconds as long as the master continues producing log lines indicating that the configuration is progressing.
• Git source checkout step now supports reference repositories.
• P4 source step now supports more advanced options.
• The comments field of changes is no longer limited to 1024 characters on MySQL and Postgres. See :bb2367 and :bb736.
• The WebStatus builder page can now filter pending/current/finished builds by property parameters of the form ?property.<name>=<value>.
• The WebStatus StatusResourceBuilder page can now take the maxsearch argument
• The WebStatus has a new authz "view" action that allows you to require users to logged in to view the WebStatus.
• The WebStatus now shows revisions (+ codebase) where it used to simply say "multiple rev".
• The Console view now supports codebases.
• The GitHub webhook has been updated to work with v3 of the GitHub webhook API.
• Build status can be sent to GitHub. Depends on txgithub package. See :bbGitHubStatus and GitHub Commit Status.

• The web UI for Builders has been updated:

  • shows the build 'reason' and 'interested users'
  • shows sourcestamp information for builders that use multiple codebases (instead of the generic "multiple rev" placeholder that was shown before).

• Each Scheduler type can now take a 'reason' argument to customize the reason it uses for triggered builds.
• Added zsh and bash tab-completions support for 'buildbot' command.
• An example of a declarative configuration is included in :bbmaster/contrib/SimpleConfig.py, with copious comments.
• A new argument createAbsoluteSourceStamps has been added to SingleBranchScheduler for use with multiple codebases.
• The Periodic scheduler now supports codebases.
• The WebStatus Authorization support now includes a view action which can be used to restrict read-only access to the Buildbot instance.
• Information about the buildslaves (admin, host, etc) is now persisted in the database and available even if the slave is not connected.
• Master side source checkout steps now support patches (:bb:bug: 2098).
• The master-side SVN step now supports authentication for mode=export, fixing :bb2463.
• The SVN step will now canonicalize URL's before matching them for better accuracy.
• Master side source checkout steps now support retry option (:bb:bug: 2465).
• The SVN step now obfuscates the password in status logs, fixing :bb2468.
• Gerrit integration with Git Source step on master side (:bb: 2485).
• A new :bbRobocopy step is available for Windows builders (:bb:pull: 728).
• The :bbP4Source changesource now supports Perforce servers in a different timezone than the buildbot master (:bb:pull: 728).
• Revision links for commits on SouceForge (Allura) are now automatically generated.
• The WebStatus now interprets ANSI color codes in stdio output.
• Master-side source steps now respond to the "stop build" button (:bb2356).
• The web hooks now include support for Gitorious.
• It is now possible to select categories to show in the waterfall help
• The web status now has options to cancel some or all pending builds.
• The web status correctly interprets ANSI color escape codes.
• The web status now automatically scrolls output logs (:bb:pull: 1078).
• Added new config option protocols which allows to configure multiple protocols on single master.
• New source step :bbDarcs added on master side.
• RemoteShellCommands can be killed by SIGTERM with the sigtermTime parameter before resorting to SIGKILL (:bb:bug: 751). If the slave's version is less than 0.8.9, the slave will kill the process with SIGKILL regardless of whether sigtermTime is supplied.
• The Git step now uses the git clean option -f twice, to also remove untracked directories managed by another git repository. See :bb2560.
• The Git and Mercurial steps now use their inbuild commands to apply patches (:bb:bug: 2563).
• The slave-side source steps are deprecated in this version of Buildbot, and master-side support will be removed in a future version. Please convert any use of slave-side steps (imported directly from buildbot.steps.source, rather than from a specific module like buildbot.steps.source.svn) to use master-side steps. TODO: update version in deprecation warning.
• New source step :bbMonotone added on master side.
• Introduce an alternative way to deploy Buildbot and try the pyflakes tutorial using Docker <first-run-docker-label>.
• The :bbHTTPStep step can make arbitrary HTTP requests from the master, allowing communication with external APIs. This new feature requires the optional txrequests and requests Python packages.
• :bbCVS source step now checks for "sticky dates" from a previous checkout before updating an existing source directory.
• The IRC bot of :bbIRC will, unless useRevisions is set, shorten long lists of revisions printed when a build starts; it will only show two, and the number of additional revisions included in the build.
• A new argument createAbsoluteSourceStamps has been added to :bbNightly for use with multiple codebases.
• The branch and codebase arguments to the :bbGit step are now renderable.
• The manifestBranch of the bbRepo step is now renderable
• reconf option for GNUAutotools to run autoreconf before ./configure
• A new :bbMultipleFileUpload step was added to allow uploading several files (or directories) in a single step.
• Buildslave info can now be retrieved via Interpolate and a new :bbSetSlaveInfo buildstep.
• The HGPoller and GitPoller now split filenames on newlines, rather than whitespace, so files containing whitespace are handled correctly.
• Add 'pollAtLaunch' flag for polling change sources. This allows a poller to poll immediately on launch and get changes that occurred while it was down.
• The GitPoller can now be configured to poll all available branches (:bb:pull: 1010).
• Systemd unit files for Buildbot are available in the :bbcontrib/ directory.
• Added the :bbBitbucketPullrequestPoller changesource.
• The :bbForceScheduler now takes a buttonName argument to specify the name of the button on the force-build form.
• A new argument summaryCB has been added to GerritStatusPush, to allow sending one review per buildset. Sending a single "summary" review per buildset is now the default if neither summaryCB nor reviewCB are specified.
• ::bbRepo now supports a depth flag when initializing the repo. This controls the amount of git history to download.
• The GitHub webhook can now optionally ignore non-distinct commits (:bb:bug: 1861).
• HTML log files are no longer stored in status pickles (:bb:pull: 1077)
• Builds are now retried after a slave is lost (:bb:pull: 1049).
• Added new build steps for VC12, VS2014 and MsBuild12.
• The mode parameter of the VS steps is now renderable (:bb:bug: 2592).
• The waterfall and atom/rss feeds can be filtered with the project url paramter.
• SVN source step and ShellCommand now support password obfuscation. (:bb:bug: 2468 and :bb🐛 1478).
• The buildbot status client can now access a build properties via the getProperties call.

Forward Compatibility

In preparation for a more asynchronous implementation of build steps in Buildbot 0.9.0, this version introduces support for new-style steps. Existing old-style steps will continue to function correctly in Buildbot 0.8.x releases and in Buildbot 0.9.0, but support will be dropped soon afterward. See New-Style-Build-Steps, below, for guidance on rewriting existing steps in this new style. To eliminate ambiguity, the documentation for this version only reflects support for new-style steps. Refer to the documentation for previous versions for infrormation on old-style steps.

Fixes

• Fixed an issue where the Git and CVS source steps silently changed the workdir to 'build' when the 'copy' method is used.
• The CVS source step now respects the timeout parameter.
• The Git step now uses the git submodule update option --init when updating the submodules of an existing repository, so that it will receive any newly added submodules.
• The web status no longer relies on the current working directory, which is not set correctly by some initscripts, to find the templates/ directory (:bb2586).
• The Perforce source step uses the correct path separator when the master is on Windows and the build slave is on a POSIX OS (:bb1114).
• The source steps now correctly interpolate properties in env.
• GerritStatusPush now supports setting scores with Gerrit 2.6 and newer
• The change hook no longer fails when passing unicode to change_hook_auth (:bb:pull: 996).
• The source steps now correctly interpolate properties in env.
• Whitespace is properly handled for StringParameter, so that appropriate validation errors are raised for required parameters (:bb:pull: 1084).
• Fix a rare case where a buildtep might fail from a GeneratorExit exception (:bb:pull: 1063).
• Fixed an issue where UTF-8 data in logs caused RSS feed exceptions (:bb:bug: 951).
• Fix an issue with unescaped author names causing invalid RSS feeds (:bb:bug: 2596).
• Fixed an issue with pubDate format in feeds.
• Fixed an issue where the step text value could cause a TypeError in the build detail page (:bb:pull: 1061).
• Fix failures where git clean fails but could be clobbered (:bb:pull: 1058).
• Build step now correctly fails when the git clone step fails (:bb:pull: 1057).
• Fixed a race condition in slave shutdown (:bb:pull: 1019).
• Now correctly unsubscribes StatusPush from status updates when reconfiguring (:bb:pull: 997).
• Fixes parsing git commit messages that are blank.
• :bb:step: Git no longer fails when work dir exists but isn't a checkout (:bb:bug: 2531).
• The haltOnFailure and flunkOnFailure attricutes of :bb:step: ShellCommand are now renderable. (:bb:bug 2486).
• The rotateLength and maxRotatedFile arguments are no longer treated as strings in buildbot.tac. This fixes log rotation.
• Builbot no longer specifies a revision when pulling from a mercurial (:bb:bug: 438).
• The WebStatus no longer incorrectly refers to fields that might not be visible.
• The GerritChangeSource now sets a default author, fixing an exception that occured when Gerrit didn't report an owner name/email.
• Respects the RETRY status when an interrupt occurs.
• Fixes an off-by-one error when the tryclient is finding the current git branch.
• Improve the Mercurial source stamp extraction in the try client.
• Fixes some edge cases in timezone handling for python < 2.7.4 (:bb:bug: 2522).
• The EC2LatentBuildSlave will now only consider available AMI's.
• Fixes a case where the first build runs on an old slave instead of a new one after reconfig (:bb:bug: 2507).
• The e-mail address validation for the MailNotifier status receiver has been improved.
• The --db parameter of buildbot create-master is now validated.
• No longer ignores default choice for ForceScheduler list parameters
• Now correctly handles BuilderConfig(..., mergeRequests=False) (:bb:bug: 2555).
• Now excludes changes from sourcestamps when they aren't in the DB (:bb:bug: 2554).
• Fixes a compatibility issue with HPCloud in the OpenStack latent slave.
• Allow _ as a valid character in JSONP callback names.
• Fix build start time retrieval in the WebStatus grid view.
• Increase the length of the DB fields changes.comments and buildset_properties.property_value.

Deprecations, Removals, and Non-Compatible Changes

• Both old-style and new-style steps are supported in this version of Buildbot. Upgrade your steps to new-style now, as support for old-style steps will be dropped after Buildbot-0.9.0. See New-Style-Build-Steps for details.
• slavePortnum option deprecated, please use c['protocols']['pb']['port'] to set up PB port
• The buildbot.process.mtrlogobserver module have been renamed to buildbot.steps.mtrlogobserver.
• The buildmaster now requires at least Twisted-11.0.0.
• The buildmaster now requires at least sqlalchemy-migrate 0.6.1.
• The hgbuildbot Mercurial hook has been moved to contrib/, and does not work with recent versions of Mercurial and Twisted. The runtimes for these two tools are incompatible, yet hgbuildbot attempts to run both in the same Python interpreter. Mayhem ensues.
• The try scheduler's --connect=ssh method no longer supports waiting for results (--wait).
• The former buildbot.process.buildstep.RemoteCommand class and its subclasses are now in :py:mod`buildbot.process.remotecommand`, although imports from the previous path will continue to work. Similarly, the former buildbot.process.buildstep.LogObserver class and its subclasses are now in :py:mod`buildbot.process.logobserver`, although imports from the previous path will continue to work.
• The undocumented BuildStep method checkDisconnect is deprecated and now does nothing as the handling of disconnects is now handled in the failed method. Any custom steps adding this method as a callback or errback should no longer do so.
• The build step MsBuild is now called MsBuild4 as multiple versions are now supported. An alias is provided so existing setups will continue to work, but this will be removed in a future release.

Changes for Developers

• The :pyCompositeStepMixin now provides a runGlob method to check for files on the slave that match a given shell-style pattern.
• The :pyBuilderStatus now allows you to pass a filter_fn arguent to generateBuilds.

Slave

Features

• Added zsh and bash tab-completions support for 'buildslave' command.
• RemoteShellCommands accept the new sigtermTime parameter from master. This allows processes to be killed by SIGTERM before resorting to SIGKILL (:bb:bug: 751)
• Added spot instance support to EC2LatentBuildSlave.
• Commands will now throw a ValueError if mandatory args are not present.
• Added a new remote command :pyGlobPath that can be used to call Python's glob.glob on the slave.

Fixes

• Fixed an issue when buildstep stop() was raising an exception incorrectly if timeout for buildstep wasn't set or was None (see :bb753) thus keeping watched logfiles open (this prevented their removal on Windows in subsequent builds).
• Fixed a bug in P4 source step where the timeout parameter was ignored.
• Fixed a bug in P4 source step where using a custom view-spec could result in failed syncs due to incorrectly generated command-lines.
• The logwatcher will use /usr/xpg4/bin/tail on Solaris, it if is available (:bb:pull: 1065).

Deprecations, Removals, and Non-Compatible Changes

Details

For a more detailed description of the changes made in this version, see the git log itself:

git log v0.8.8..master

Older Versions

Release notes for older versions of Buildbot are available in the :bbmaster/docs/relnotes/ directory of the source tree. Newer versions are also available here:

0.8.8 0.8.7 0.8.6