Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Fetching contributors…

Octocat-spinner-32-eaf2f5

Cannot retrieve contributors at this time

file 396 lines (320 sloc) 16.576 kb
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395
@bcindex c['slaves']

The @code{c['slaves']} key is a list of known buildslaves. In the common case,
each buildslave is defined by an instance of the BuildSlave class. It
represents a standard, manually started machine that will try to connect to
the buildbot master as a slave. Contrast these with the "on-demand" latent
buildslaves, such as the Amazon Web Service Elastic Compute Cloud latent
buildslave discussed below.

The BuildSlave class is instantiated with two values: (slavename,
slavepassword). These are the same two values that need to be provided to the
buildslave administrator when they create the buildslave.

The slavenames must be unique, of course. The password exists to
prevent evildoers from interfering with the buildbot by inserting
their own (broken) buildslaves into the system and thus displacing the
real ones.

Buildslaves with an unrecognized slavename or a non-matching password
will be rejected when they attempt to connect, and a message
describing the problem will be put in the log file (see @ref{Logfiles}).

@example
from buildbot.buildslave import BuildSlave
c['slaves'] = [BuildSlave('bot-solaris', 'solarispasswd')
               BuildSlave('bot-bsd', 'bsdpasswd')
              ]
@end example

@cindex Properties
@code{BuildSlave} objects can also be created with an optional
@code{properties} argument, a dictionary specifying properties that
will be available to any builds performed on this slave. For example:

@example
from buildbot.buildslave import BuildSlave
c['slaves'] = [BuildSlave('bot-solaris', 'solarispasswd',
                    properties=@{'os':'solaris'@}),
              ]
@end example

The @code{BuildSlave} constructor can also take an optional
@code{max_builds} parameter to limit the number of builds that it
will execute simultaneously:

@example
from buildbot.buildslave import BuildSlave
c['slaves'] = [BuildSlave("bot-linux", "linuxpassword", max_builds=2)]
@end example

@menu
* When Buildslaves Go Missing::
* Latent Buildslaves::
@end menu

@node When Buildslaves Go Missing
@subsection When Buildslaves Go Missing

Sometimes, the buildslaves go away. One very common reason for this is
when the buildslave process is started once (manually) and left
running, but then later the machine reboots and the process is not
automatically restarted.

If you'd like to have the administrator of the buildslave (or other
people) be notified by email when the buildslave has been missing for
too long, just add the @code{notify_on_missing=} argument to the
@code{BuildSlave} definition:

@example
c['slaves'] = [BuildSlave('bot-solaris', 'solarispasswd',
                          notify_on_missing="bob@@example.com"),
              ]
@end example

By default, this will send email when the buildslave has been
disconnected for more than one hour. Only one email per
connection-loss event will be sent. To change the timeout, use
@code{missing_timeout=} and give it a number of seconds (the default
is 3600).

You can have the buildmaster send email to multiple recipients: just
provide a list of addresses instead of a single one:

@example
c['slaves'] = [BuildSlave('bot-solaris', 'solarispasswd',
                          notify_on_missing=["bob@@example.com",
                                             "alice@@example.org"],
                          missing_timeout=300, # notify after 5 minutes
                          ),
              ]
@end example

The email sent this way will use a MailNotifier (@pxref{MailNotifier})
status target, if one is configured. This provides a way for you to
control the ``from'' address of the email, as well as the relayhost
(aka ``smarthost'') to use as an SMTP server. If no MailNotifier is
configured on this buildmaster, the buildslave-missing emails will be
sent using a default configuration.

Note that if you want to have a MailNotifier for buildslave-missing
emails but not for regular build emails, just create one with
builders=[], as follows:

@example
from buildbot.status import mail
m = mail.MailNotifier(fromaddr="buildbot@@localhost", builders=[],
                      relayhost="smtp.example.org")
c['status'].append(m)
c['slaves'] = [BuildSlave('bot-solaris', 'solarispasswd',
                          notify_on_missing="bob@@example.com"),
              ]
@end example

@node Latent Buildslaves
@subsection Latent Buildslaves

The standard buildbot model has slaves started manually. The previous section
described how to configure the master for this approach.

Another approach is to let the buildbot master start slaves when builds are
ready, on-demand. Thanks to services such as Amazon Web Services' Elastic
Compute Cloud ("AWS EC2"), this is relatively easy to set up, and can be
very useful for some situations.

The buildslaves that are started on-demand are called "latent" buildslaves.
As of this writing, buildbot ships with an abstract base class for building
latent buildslaves, and a concrete implementation for AWS EC2.

@menu
* Amazon Web Services Elastic Compute Cloud ("AWS EC2")::
* Dangers with Latent Buildslaves::
* Writing New Latent Buildslaves::
@end menu

@node Amazon Web Services Elastic Compute Cloud ("AWS EC2")
@subsubsection Amazon Web Services Elastic Compute Cloud ("AWS EC2")

@url{http://aws.amazon.com/ec2/,,AWS EC2} is a web service that allows you to
start virtual machines in an Amazon data center. Please see their website for
details, incuding costs. Using the AWS EC2 latent buildslaves involves getting
an EC2 account with AWS and setting up payment; customizing one or more EC2
machine images ("AMIs") on your desired operating system(s) and publishing
them (privately if needed); and configuring the buildbot master to know how to
start your customized images for "substantiating" your latent slaves.

@heading Get an AWS EC2 Account

To start off, to use the AWS EC2 latent buildslave, you need to get an AWS
developer account and sign up for EC2. These instructions may help you get
started:

@itemize @bullet
@item
Go to http://aws.amazon.com/ and click to "Sign Up Now" for an AWS account.

@item
Once you are logged into your account, you need to sign up for EC2.
Instructions for how to do this have changed over time because Amazon changes
their website, so the best advice is to hunt for it. After signing up for EC2,
it may say it wants you to upload an x.509 cert. You will need this to create
images (see below) but it is not technically necessary for the buildbot master
configuration.

@item
You must enter a valid credit card before you will be able to use EC2. Do that
under 'Payment Method'.

@item
Make sure you're signed up for EC2 by going to 'Your Account'->'Account
Activity' and verifying EC2 is listed.
@end itemize

@heading Create an AMI

Now you need to create an AMI and configure the master. You may need to
run through this cycle a few times to get it working, but these instructions
should get you started.

Creating an AMI is out of the scope of this document. The
@url{http://docs.amazonwebservices.com/AWSEC2/latest/GettingStartedGuide/,,EC2 Getting Started Guide}
is a good resource for this task. Here are a few additional hints.

@itemize @bullet
@item
When an instance of the image starts, it needs to automatically start a
buildbot slave that connects to your master (to create a buildbot slave,
@pxref{Creating a buildslave}; to make a daemon,
@pxref{Launching the daemons}).

@item
You may want to make an instance of the buildbot slave, configure it as a
standard buildslave in the master (i.e., not as a latent slave), and test and
debug it that way before you turn it into an AMI and convert to a latent
slave in the master.
@end itemize

@heading Configure the Master with an EC2LatentBuildSlave

Now let's assume you have an AMI that should work with the
EC2LatentBuildSlave. It's now time to set up your buildbot master
configuration.

You will need some information from your AWS account: the "Access Key Id" and
the "Secret Access Key". If you've built the AMI yourself, you probably
already are familiar with these values. If you have not, and someone has
given you access to an AMI, these hints may help you find the necessary
values:

@itemize @bullet
@item
While logged into your AWS account, find the "Access Identifiers" link (either
on the left, or via "Your Account" -> "Access Identifiers".

@item
On the page, you'll see alphanumeric values for "Your Access Key Id:" and
"Your Secret Access Key:". Make a note of these. Later on, we'll call the
first one your "identifier" and the second one your "secret_identifier."
@end itemize

When creating an EC2LatentBuildSlave in the buildbot master configuration,
the first three arguments are required. The name and password are the first
two arguments, and work the same as with normal buildslaves. The next
argument specifies the type of the EC2 virtual machine (available options as
of this writing include "m1.small", "m1.large", 'm1.xlarge", "c1.medium",
and "c1.xlarge"; see the EC2 documentation for descriptions of these
machines).

Here is the simplest example of configuring an EC2 latent buildslave. It
specifies all necessary remaining values explicitly in the instantiation.

@example
from buildbot.ec2buildslave import EC2LatentBuildSlave
c['slaves'] = [EC2LatentBuildSlave('bot1', 'sekrit', 'm1.large',
                                   ami='ami-12345',
                                   identifier='publickey',
                                   secret_identifier='privatekey'
                                   )]
@end example

The "ami" argument specifies the AMI that the master should start. The
"identifier" argument specifies the AWS "Access Key Id," and the
"secret_identifier" specifies the AWS "Secret Access Key." Both the AMI and
the account information can be specified in alternate ways.

Note that whoever has your identifier and secret_identifier values can request
AWS work charged to your account, so these values need to be carefully
protected. Another way to specify these access keys is to put them in a
separate file. You can then make the access privileges stricter for this
separate file, and potentially let more people read your main configuration
file.

By default, you can make an .ec2 directory in the home folder of the user
running the buildbot master. In that directory, create a file called aws_id.
The first line of that file should be your access key id; the second line
should be your secret access key id. Then you can instantiate the build slave
as follows.

@example
from buildbot.ec2buildslave import EC2LatentBuildSlave
c['slaves'] = [EC2LatentBuildSlave('bot1', 'sekrit', 'm1.large',
                                   ami='ami-12345')]
@end example

If you want to put the key information in another file, use the
"aws_id_file_path" initialization argument.

Previous examples used a particular AMI. If the Buildbot master will be
deployed in a process-controlled environment, it may be convenient to
specify the AMI more flexibly. Rather than specifying an individual AMI,
specify one or two AMI filters.

In all cases, the AMI that sorts last by its location (the S3 bucket and
manifest name) will be preferred.

One available filter is to specify the acceptable AMI owners, by AWS account
number (the 12 digit number, usually rendered in AWS with hyphens like
"1234-5678-9012", should be entered as in integer).

@example
from buildbot.ec2buildslave import EC2LatentBuildSlave
bot1 = EC2LatentBuildSlave('bot1', 'sekrit', 'm1.large',
                           valid_ami_owners=[11111111111,
                                             22222222222],
                           identifier='publickey',
                           secret_identifier='privatekey'
                           )
@end example

The other available filter is to provide a regular expression string that
will be matched against each AMI's location (the S3 bucket and manifest name).

@example
from buildbot.ec2buildslave import EC2LatentBuildSlave
bot1 = EC2LatentBuildSlave(
    'bot1', 'sekrit', 'm1.large',
    valid_ami_location_regex=r'buildbot\-.*/image.manifest.xml',
    identifier='publickey', secret_identifier='privatekey')
@end example

The regular expression can specify a group, which will be preferred for the
sorting. Only the first group is used; subsequent groups are ignored.

@example
from buildbot.ec2buildslave import EC2LatentBuildSlave
bot1 = EC2LatentBuildSlave(
    'bot1', 'sekrit', 'm1.large',
    valid_ami_location_regex=r'buildbot\-.*\-(.*)/image.manifest.xml',
    identifier='publickey', secret_identifier='privatekey')
@end example

If the group can be cast to an integer, it will be. This allows 10 to sort
after 1, for instance.

@example
from buildbot.ec2buildslave import EC2LatentBuildSlave
bot1 = EC2LatentBuildSlave(
    'bot1', 'sekrit', 'm1.large',
    valid_ami_location_regex=r'buildbot\-.*\-(\d+)/image.manifest.xml',
    identifier='publickey', secret_identifier='privatekey')
@end example

In addition to using the password as a handshake between the master and the
slave, you may want to use a firewall to assert that only machines from a
specific IP can connect as slaves. This is possible with AWS EC2 by using
the Elastic IP feature. To configure, generate a Elastic IP in AWS, and then
specify it in your configuration using the "elastic_ip" argument.

@example
from buildbot.ec2buildslave import EC2LatentBuildSlave
c['slaves'] = [EC2LatentBuildSlave('bot1', 'sekrit', 'm1.large',
                                   'ami-12345',
                                   identifier='publickey',
                                   secret_identifier='privatekey',
                                   elastic_ip='208.77.188.166'
                                   )]
@end example

The EC2LatentBuildSlave supports all other configuration from the standard
BuildSlave. The "missing_timeout" and "notify_on_missing" specify how long
to wait for an EC2 instance to attach before considering the attempt to have
failed, and email addresses to alert, respectively. "missing_timeout"
defaults to 20 minutes.

The "build_wait_timeout" allows you to specify how long an EC2LatentBuildSlave
should wait after a build for another build before it shuts down the EC2
instance. It defaults to 10 minutes.

"keypair_name" and "security_name" allow you to specify different names for
these AWS EC2 values. They both default to "latent_buildbot_slave".

@node Dangers with Latent Buildslaves
@subsubsection Dangers with Latent Buildslaves

Any latent build slave that interacts with a for-fee service, such as the
EC2LatentBuildSlave, brings significant risks. As already identified, the
configuraton will need access to account information that, if obtained by a
criminal, can be used to charge services to your account. Also, bugs in the
buildbot software may lead to unnecessary charges. In particular, if the
master neglects to shut down an instance for some reason, a virtual machine
may be running unnecessarily, charging against your account. Manual and/or
automatic (e.g. nagios with a plugin using a library like boto)
double-checking may be appropriate.

A comparitively trivial note is that currently if two instances try to attach
to the same latent buildslave, it is likely that the system will become
confused. This should not occur, unless, for instance, you configure a normal
build slave to connect with the authentication of a latent buildbot. If the
situation occurs, stop all attached instances and restart the master.

@node Writing New Latent Buildslaves
@subsubsection Writing New Latent Buildslaves

Writing a new latent buildslave should only require subclassing
@code{buildbot.buildslave.AbstractLatentBuildSlave} and implementing
start_instance and stop_instance.

@example
def start_instance(self):
    # responsible for starting instance that will try to connect with this
    # master. Should return deferred. Problems should use an errback. The
    # callback value can be None, or can be an iterable of short strings to
    # include in the "substantiate success" status message, such as
    # identifying the instance that started.
    raise NotImplementedError

def stop_instance(self, fast=False):
    # responsible for shutting down instance. Return a deferred. If `fast`,
    # we're trying to shut the master down, so callback as soon as is safe.
    # Callback value is ignored.
    raise NotImplementedError
@end example

See @code{buildbot.ec2buildslave.EC2LatentBuildSlave} for an example, or see the
test example @code{buildbot.test_slaves.FakeLatentBuildSlave}.

Something went wrong with that request. Please try again.