Several small utilities are available at the top-level buildbot.util package.
This package provides a few useful collection objects.
Note
This module used to be named collections, but without absolute imports (328), this precluded using the standard library's collections module.
This function provides a simple way to say "please do this later". For example:
from buildbot.util.eventual import eventually
def do_what_I_say(what, where):
# ...
return d
eventually(do_what_I_say, "clean up", "your bedroom")The package defines "later" as "next time the reactor has control", so this is a good way to avoid long loops that block other activity in the reactor.
It's often necessary to perform some action in response to a particular type of event. For example, steps need to update their status after updates arrive from the slave. However, when many events arrive in quick succession, it's more efficient to only perform the action once, after the last event has occurred.
The debounce.method(wait) decorator is the tool for the job.
Many Buildbot services perform some periodic, asynchronous operation. Change sources, for example, contact the repositories they monitor on a regular basis. The tricky bit is, the periodic operation must complete before the service stops.
The @poll.method decorator makes this behavior easy and reliable.
This package is just an import of the best available JSON module. Use it instead of a more complex conditional import of simplejson or json:
from buildbot.util import jsonSeveral Buildbot components make use of maildirs to hand off messages between components. On the receiving end, there's a need to watch a maildir for incoming messages and trigger some action when one arrives.
A :py
MaildirServiceinstance watches a maildir for new messages. It should be a child service of some :py~twisted.application.service.MultiServiceinstance. When running, this class uses the linux dirwatcher API (if available) or polls for new files in the 'new' maildir subdirectory. When it discovers a new message, it invokes its :pymessageReceivedmethod.To use this class, subclass it and implement a more interesting :py
messageReceivedfunction.
Similar to maildirs, netstrings are used occasionally in Buildbot to encode data for interchange. While Twisted supports a basic netstring receiver protocol, it does not have a simple way to apply that to a non-network situation.
This module contains a few utilities that are not included with SQLAlchemy.
The classes in the :pybuildbot.util.subscription module are used for master-local subscriptions. In the near future, all uses of this module will be replaced with message-queueing implementations that allow subscriptions and subscribers to span multiple masters.
This module is a copy of https://github.com/taichino/croniter, and provides support for converting cron-like time specifications into actual times.
The classes in the :pybuildbot.util.subscription module are used for dealing with object state stored in the database.
This module is a drop-in replacement for the stdlib pickle or cPickle modules. It adds the ability to load pickles that reference classes that have since been removed from Buildbot. It should be used whenever pickles from Buildbot-0.8.x and earlier are loaded.
This module makes it easy to manipulate identifiers.
This module implements some useful subclasses of Twisted services.
The first two classes are more robust implementations of two Twisted classes, and should be used universally in Buildbot code.
This class is similar to :pytwisted.application.service.MultiService, except that it handles Deferreds returned from child services` startService and stopService methods.
Twisted's service implementation does not support asynchronous startService methods. The reasoning is that all services should start at process startup, with no need to coordinate between them. For Buildbot, this is not sufficient. The framework needs to know when startup has completed, so it can begin scheduling builds. This class implements the desired functionality, with a parent service's startService returning a Deferred which will only fire when all child services startService methods have completed.
This class also fixes a bug with Twisted's implementation of stopService which ignores failures in the stopService process. With :pyAsyncMultiService, any errors in a child's stopService will be propagated to the parent's stopService method.
Service utilities; ClusteredService
Some services in buildbot must have only one "active" instance at any given time. In a single-master configuration, this requirement is trivial to maintain. In a multiple-master configuration, some arbitration is required to ensure that the service is always active on exactly one master in the cluster.
For example, a particular daily scheduler could be configured on multiple masters, but only one of them should actually trigger the required builds.