forked from buildbot/buildbot
-
Notifications
You must be signed in to change notification settings - Fork 1
/
installation.texinfo
727 lines (570 loc) · 27.7 KB
/
installation.texinfo
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
@menu
* Buildbot Components::
* Requirements::
* Installing the code::
* Creating a buildmaster::
* Upgrading an Existing Buildmaster::
* Creating a buildslave::
* Launching the daemons::
* Logfiles::
* Shutdown::
* Maintenance::
* Troubleshooting::
@end menu
@node Buildbot Components
@section Buildbot Components
Buildbot is shipped in two components: the buildmaster (called @code{buildbot}
for legacy reasons) and the buildslave. The buildslave component has far fewer
requirements, and is more broadly compatible than the buildmaster. You will
need to carefully pick the environment in which to run your buildmaster, but
the buildslave should be able to run just about anywhere.
It is possible to install the buildmaster and buildslave on the same system,
although for anything but the smallest installation this arrangement will not
be very efficient.
@node Requirements
@section Requirements
@menu
* Common Requirements::
* Buildmaster Requirements::
@end menu
@node Common Requirements
@subsection Common Requirements
At a bare minimum, you'll need the following for both the buildmaster and a
buildslave:
@itemize @bullet
@item
Python: http://www.python.org
Buildbot requires python-2.4 or later.
@item
Twisted: http://twistedmatrix.com
Both the buildmaster and the buildslaves require Twisted-2.0.x or
later. It has been tested against all releases of Twisted up to
Twisted-2.5.0 (the most recent as of this writing). As always, the
most recent version is recommended.
Twisted is delivered as a collection of subpackages. You'll need at
least "Twisted" (the core package), and you'll also want TwistedMail,
TwistedWeb, and TwistedWords (for sending email, serving a web status
page, and delivering build status via IRC, respectively). You might
also want TwistedConch (for the encrypted Manhole debug port). Note
that Twisted requires ZopeInterface to be installed as well.
@end itemize
Of course, your project's build process will impose additional
requirements on the buildslaves. These hosts must have all the tools
necessary to compile and test your project's source code.
@node Buildmaster Requirements
@subsection Buildmaster Requirements
@itemize @bullet
@item
sqlite3: http://pypi.python.org/pypi/pysqlite
The sqlite3 package is required for python-2.5 and earlier (it is already
included in python-2.5 and later, but the version in python-2.5 has nasty bugs)
@item
simplejson: http://pypi.python.org/pypi/simplejson
The simplejson package is required for python-2.5 and earlier (it is already
included as json in python-2.6 and later)
@item
Jinja2: http://jinja.pocoo.org/2
Jinja2 is a general purpose templating language and is used by Buildbot
to generate the HTML output.
@end itemize
@node Installing the code
@section Installing the code
@cindex installation
Buildbot and Buildslave are installed using the standard python
@code{distutils} process. For either component, after unpacking the tarball,
the process is:
@example
python setup.py build
python setup.py install
@end example
where the install step may need to be done as root. This will put the bulk of
the code in somewhere like @code{/usr/lib/python2.3/site-packages/buildbot}. It
will also install the @code{buildbot} command-line tool in
@code{/usr/bin/buildbot}.
To test this, shift to a different directory (like @code{/tmp}), and run:
@example
buildbot --version
# or
buildslave --version
@end example
If it shows you the versions of Buildbot and Twisted, the install went
ok. If it says @code{no such command} or it gets an @code{ImportError}
when it tries to load the libaries, then something went wrong.
@code{pydoc buildbot} is another useful diagnostic tool.
Windows users will find these files in other places. You will need to
make sure that python can find the libraries, and will probably find
it convenient to have @code{buildbot} on your PATH.
If you wish, you can run the buildbot unit test suite like this:
@example
PYTHONPATH=. trial buildbot.test
# or
PYTHONPATH=. trial buildslave.test
@end example
Nothing should fail, a few might be skipped. If any of the tests fail, you
should stop and investigate the cause before continuing the installation
process, as it will probably be easier to track down the bug early.
If you cannot or do not wish to install the buildbot into a site-wide
location like @file{/usr} or @file{/usr/local}, you can also install
it into the account's home directory or any other location using a tool like @url{http://pypi.python.org/pypi/virtualenv,virtualenv}.
@node Creating a buildmaster
@section Creating a buildmaster
As you learned earlier (@pxref{System Architecture}), the buildmaster
runs on a central host (usually one that is publically visible, so
everybody can check on the status of the project), and controls all
aspects of the buildbot system. Let us call this host
@code{buildbot.example.org}.
You may wish to create a separate user account for the buildmaster,
perhaps named @code{buildmaster}. This can help keep your personal
configuration distinct from that of the buildmaster and is useful if
you have to use a mail-based notification system (@pxref{Change
Sources}). However, the Buildbot will work just fine with your regular
user account.
You need to choose a directory for the buildmaster, called the
@code{basedir}. This directory will be owned by the buildmaster, which
will use configuration files therein, and create status files as it
runs. @file{~/Buildbot} is a likely value. If you run multiple
buildmasters in the same account, or if you run both masters and
slaves, you may want a more distinctive name like
@file{~/Buildbot/master/gnomovision} or
@file{~/Buildmasters/fooproject}. If you are using a separate user
account, this might just be @file{~buildmaster/masters/fooproject}.
Once you've picked a directory, use the @command{buildbot
create-master} command to create the directory and populate it with
startup files:
@example
buildbot create-master -r @var{basedir}
@end example
You will need to create a configuration file (@pxref{Configuration})
before starting the buildmaster. Most of the rest of this manual is
dedicated to explaining how to do this. A sample configuration file is
placed in the working directory, named @file{master.cfg.sample}, which
can be copied to @file{master.cfg} and edited to suit your purposes.
(Internal details: This command creates a file named
@file{buildbot.tac} that contains all the state necessary to create
the buildmaster. Twisted has a tool called @code{twistd} which can use
this .tac file to create and launch a buildmaster instance. twistd
takes care of logging and daemonization (running the program in the
background). @file{/usr/bin/buildbot} is a front end which runs twistd
for you.)
In addition to @file{buildbot.tac}, a small @file{Makefile.sample} is
installed. This can be used as the basis for customized daemon startup,
@xref{Launching the daemons}.
@node Upgrading an Existing Buildmaster
@section Upgrading an Existing Buildmaster
If you have just installed a new version of the Buildbot code, and you
have buildmasters that were created using an older version, you'll
need to upgrade these buildmasters before you can use them. The
upgrade process adds and modifies files in the buildmaster's base
directory to make it compatible with the new code.
@example
buildbot upgrade-master @var{basedir}
@end example
This command will also scan your @file{master.cfg} file for
incompatibilities (by loading it and printing any errors or deprecation
warnings that occur). Each buildbot release tries to be compatible
with configurations that worked cleanly (i.e. without deprecation
warnings) on the previous release: any functions or classes that are
to be removed will first be deprecated in a release, to give you a
chance to start using the replacement.
The @code{upgrade-master} command is idempotent. It is safe to run it
multiple times. After each upgrade of the buildbot code, you should
use @code{upgrade-master} on all your buildmasters.
In general, Buildbot slaves and masters can be upgraded independently, although
some new features will not be available, depending on the master and slave
versions.
@heading Version-specific Notes
@menu
* Upgrading a Buildmaster to Buildbot-0.7.6::
* Upgrading a Buildmaster to Buildbot-0.8.0::
@end menu
@node Upgrading a Buildmaster to Buildbot-0.7.6
@subsection Upgrading a Buildmaster to Buildbot-0.7.6
The 0.7.6 release introduced the @file{public_html/} directory, which
contains @file{index.html} and other files served by the
@code{WebStatus} and @code{Waterfall} status displays. The
@code{upgrade-master} command will create these files if they do not
already exist. It will not modify existing copies, but it will write a
new copy in e.g. @file{index.html.new} if the new version differs from
the version that already exists.
@node Upgrading a Buildmaster to Buildbot-0.8.0
@subsection Upgrading a Buildmaster to Buildbot-0.8.0
Buildbot-0.8.0 introduces a database backend, which is SQLite by default. The
@code{upgrade-master} command will automatically create and populate this
database with the changes the buildmaster has seen. Note that, as of this
release, build history is @i{not} contained in the database, and is thus not
migrated.
The upgrade process renames the Changes pickle (@code{$basedir/changes.pck}) to
@code{changes.pck.old} once the upgrade is complete. To reverse the upgrade,
simply downgrade Buildbot and move this file back to its original name. You
may also wish to delete the state database (@code{state.sqlite}).
@heading Change Encoding Issues
The upgrade process assumes that strings in your Changes pickle are encoded in
UTF-8 (or plain ASCII). If this is not the case, and if there are non-UTF-8
characters in the pickle, the upgrade will fail with a suitable error message.
If this occurs, you have two options. If the change history is not important
to your purpose, you can simply delete @code{changes.pck}.
If you would like to keep the change history, then you will need to figure out
which encoding is in use, and use @code{contrib/fix_changes_pickle_encoding.py}
to rewrite the changes pickle into Unicode before upgrading the master. A
typical invocation (with Mac-Roman encoding) might look like:
@example
$ python $buildbot/contrib/fix_changes_pickle_encoding.py changes.pck macroman
decoding bytestrings in changes.pck using macroman
converted 11392 strings
backing up changes.pck to changes.pck.old
@end example
If your Changes pickle uses multiple encodings, you're on your own, but the
script in contrib may provide a good starting point for the fix.
@node Creating a buildslave
@section Creating a buildslave
Typically, you will be adding a buildslave to an existing buildmaster,
to provide additional architecture coverage. The buildbot
administrator will give you several pieces of information necessary to
connect to the buildmaster. You should also be somewhat familiar with
the project being tested, so you can troubleshoot build problems
locally.
The buildbot exists to make sure that the project's stated ``how to
build it'' process actually works. To this end, the buildslave should
run in an environment just like that of your regular developers.
Typically the project build process is documented somewhere
(@file{README}, @file{INSTALL}, etc), in a document that should
mention all library dependencies and contain a basic set of build
instructions. This document will be useful as you configure the host
and account in which the buildslave runs.
Here's a good checklist for setting up a buildslave:
@enumerate
@item
Set up the account
It is recommended (although not mandatory) to set up a separate user
account for the buildslave. This account is frequently named
@code{buildbot} or @code{buildslave}. This serves to isolate your
personal working environment from that of the slave's, and helps to
minimize the security threat posed by letting possibly-unknown
contributors run arbitrary code on your system. The account should
have a minimum of fancy init scripts.
@item
Install the buildbot code
Follow the instructions given earlier (@pxref{Installing the code}).
If you use a separate buildslave account, and you didn't install the
buildbot code to a shared location, then you will need to install it
with @code{--home=~} for each account that needs it.
@item
Set up the host
Make sure the host can actually reach the buildmaster. Usually the
buildmaster is running a status webserver on the same machine, so
simply point your web browser at it and see if you can get there.
Install whatever additional packages or libraries the project's
INSTALL document advises. (or not: if your buildslave is supposed to
make sure that building without optional libraries still works, then
don't install those libraries).
Again, these libraries don't necessarily have to be installed to a
site-wide shared location, but they must be available to your build
process. Accomplishing this is usually very specific to the build
process, so installing them to @file{/usr} or @file{/usr/local} is
usually the best approach.
@item
Test the build process
Follow the instructions in the INSTALL document, in the buildslave's
account. Perform a full CVS (or whatever) checkout, configure, make,
run tests, etc. Confirm that the build works without manual fussing.
If it doesn't work when you do it by hand, it will be unlikely to work
when the buildbot attempts to do it in an automated fashion.
@item
Choose a base directory
This should be somewhere in the buildslave's account, typically named
after the project which is being tested. The buildslave will not touch
any file outside of this directory. Something like @file{~/Buildbot}
or @file{~/Buildslaves/fooproject} is appropriate.
@item
Get the buildmaster host/port, botname, and password
When the buildbot admin configures the buildmaster to accept and use
your buildslave, they will provide you with the following pieces of
information:
@itemize @bullet
@item
your buildslave's name
@item
the password assigned to your buildslave
@item
the hostname and port number of the buildmaster, i.e. buildbot.example.org:8007
@end itemize
@item
Create the buildslave
Now run the 'buildslave' command as follows:
@example
buildslave create-slave @var{BASEDIR} @var{MASTERHOST}:@var{PORT} @var{SLAVENAME} @var{PASSWORD}
@end example
This will create the base directory and a collection of files inside,
including the @file{buildbot.tac} file that contains all the
information you passed to the @code{buildbot} command.
@item
Fill in the hostinfo files
When it first connects, the buildslave will send a few files up to the
buildmaster which describe the host that it is running on. These files
are presented on the web status display so that developers have more
information to reproduce any test failures that are witnessed by the
buildbot. There are sample files in the @file{info} subdirectory of
the buildbot's base directory. You should edit these to correctly
describe you and your host.
@file{BASEDIR/info/admin} should contain your name and email address.
This is the ``buildslave admin address'', and will be visible from the
build status page (so you may wish to munge it a bit if
address-harvesting spambots are a concern).
@file{BASEDIR/info/host} should be filled with a brief description of
the host: OS, version, memory size, CPU speed, versions of relevant
libraries installed, and finally the version of the buildbot code
which is running the buildslave.
The optional @file{BASEDIR/info/access_uri} can specify a URI which will
connect a user to the machine. Many systems accept @code{ssh://hostname} URIs
for this purpose.
If you run many buildslaves, you may want to create a single
@file{~buildslave/info} file and share it among all the buildslaves
with symlinks.
@end enumerate
@menu
* Buildslave Options::
* Other Buildslave Configuration::
@end menu
@node Buildslave Options
@subsection Buildslave Options
There are a handful of options you might want to use when creating the
buildslave with the @command{buildslave create-slave <options> DIR <params>}
command. You can type @command{buildslave create-slave --help} for a summary.
To use these, just include them on the @command{buildslave create-slave}
command line, like this:
@example
buildslave create-slave --umask=022 ~/buildslave buildmaster.example.org:42012 myslavename mypasswd
@end example
@table @code
@item --usepty
This is a boolean flag that tells the buildslave whether to launch child
processes in a PTY or with regular pipes (the default) when the master does not
specify. This option is deprecated, as this particular parameter is better
specified on the master.
@item --umask
This is a string (generally an octal representation of an integer)
which will cause the buildslave process' ``umask'' value to be set
shortly after initialization. The ``twistd'' daemonization utility
forces the umask to 077 at startup (which means that all files created
by the buildslave or its child processes will be unreadable by any
user other than the buildslave account). If you want build products to
be readable by other accounts, you can add @code{--umask=022} to tell
the buildslave to fix the umask after twistd clobbers it. If you want
build products to be @emph{writable} by other accounts too, use
@code{--umask=000}, but this is likely to be a security problem.
@item --keepalive
This is a number that indicates how frequently ``keepalive'' messages
should be sent from the buildslave to the buildmaster, expressed in
seconds. The default (600) causes a message to be sent to the
buildmaster at least once every 10 minutes. To set this to a lower
value, use e.g. @code{--keepalive=120}.
If the buildslave is behind a NAT box or stateful firewall, these
messages may help to keep the connection alive: some NAT boxes tend to
forget about a connection if it has not been used in a while. When
this happens, the buildmaster will think that the buildslave has
disappeared, and builds will time out. Meanwhile the buildslave will
not realize than anything is wrong.
@item --maxdelay
This is a number that indicates the maximum amount of time the
buildslave will wait between connection attempts, expressed in
seconds. The default (300) causes the buildslave to wait at most 5
minutes before trying to connect to the buildmaster again.
@item --log-size
This is the size in bytes when to rotate the Twisted log files.
@item --log-count
This is the number of log rotations to keep around. You can either
specify a number or @code{None} (the default) to keep all
@file{twistd.log} files around.
@end table
@node Other Buildslave Configuration
@subsection Other Buildslave Configuration
@table @code
@item unicode_encoding
This represents the encoding that buildbot should use when converting unicode
commandline arguments into byte strings in order to pass to the operating
system when spawning new processes.
The default value is what python's sys.getfilesystemencoding() returns, which
on Windows is 'mbcs', on Mac OSX is 'utf-8', and on Unix depends on your locale
settings.
If you need a different encoding, this can be changed in your build slave's
buildbot.tac file by adding a unicode_encoding argument to BuildSlave:
@example
s = BuildSlave(buildmaster_host, port, slavename, passwd, basedir,
keepalive, usepty, umask=umask, maxdelay=maxdelay,
unicode_encoding='utf-8')
@end example
@end table
@node Launching the daemons
@section Launching the daemons
Both the buildmaster and the buildslave run as daemon programs. To
launch them, pass the working directory to the @code{buildbot}
and @code{buildslave} commands, as appropriate:
@example
# start a master
buildbot start @var{BASEDIR}
# start a slave
buildslave start @var{SLAVE_BASEDIR}
@end example
The @var{BASEDIR} is option and can be omitted if the current directory
contains the buildbot configuration (the @file{buildbot.tac} file).
@example
buildbot start
@end example
This command will start the daemon and then return, so normally it
will not produce any output. To verify that the programs are indeed
running, look for a pair of files named @file{twistd.log} and
@file{twistd.pid} that should be created in the working directory.
@file{twistd.pid} contains the process ID of the newly-spawned daemon.
When the buildslave connects to the buildmaster, new directories will
start appearing in its base directory. The buildmaster tells the slave
to create a directory for each Builder which will be using that slave.
All build operations are performed within these directories: CVS
checkouts, compiles, and tests.
Once you get everything running, you will want to arrange for the
buildbot daemons to be started at boot time. One way is to use
@code{cron}, by putting them in a @@reboot crontab entry@footnote{this
@@reboot syntax is understood by Vixie cron, which is the flavor
usually provided with linux systems. Other unices may have a cron that
doesn't understand @@reboot}:
@example
@@reboot buildbot start @var{BASEDIR}
@end example
When you run @command{crontab} to set this up, remember to do it as
the buildmaster or buildslave account! If you add this to your crontab
when running as your regular account (or worse yet, root), then the
daemon will run as the wrong user, quite possibly as one with more
authority than you intended to provide.
It is important to remember that the environment provided to cron jobs
and init scripts can be quite different that your normal runtime.
There may be fewer environment variables specified, and the PATH may
be shorter than usual. It is a good idea to test out this method of
launching the buildslave by using a cron job with a time in the near
future, with the same command, and then check @file{twistd.log} to
make sure the slave actually started correctly. Common problems here
are for @file{/usr/local} or @file{~/bin} to not be on your
@code{PATH}, or for @code{PYTHONPATH} to not be set correctly.
Sometimes @code{HOME} is messed up too.
Some distributions may include conveniences to make starting buildbot
at boot time easy. For instance, with the default buildbot package in
Debian-based distributions, you may only need to modify
@code{/etc/default/buildbot} (see also @code{/etc/init.d/buildbot}, which
reads the configuration in @code{/etc/default/buildbot}).
@node Logfiles
@section Logfiles
@cindex logfiles
While a buildbot daemon runs, it emits text to a logfile, named
@file{twistd.log}. A command like @code{tail -f twistd.log} is useful
to watch the command output as it runs.
The buildmaster will announce any errors with its configuration file
in the logfile, so it is a good idea to look at the log at startup
time to check for any problems. Most buildmaster activities will cause
lines to be added to the log.
@node Shutdown
@section Shutdown
To stop a buildmaster or buildslave manually, use:
@example
buildbot stop @var{BASEDIR}
# or
buildslave stop @var{SLAVE_BASEDIR}
@end example
This simply looks for the @file{twistd.pid} file and kills whatever
process is identified within.
At system shutdown, all processes are sent a @code{SIGKILL}. The
buildmaster and buildslave will respond to this by shutting down
normally.
The buildmaster will respond to a @code{SIGHUP} by re-reading its
config file. Of course, this only works on unix-like systems with
signal support, and won't work on Windows. The following shortcut is
available:
@example
buildbot reconfig @var{BASEDIR}
@end example
When you update the Buildbot code to a new release, you will need to
restart the buildmaster and/or buildslave before it can take advantage
of the new code. You can do a @code{buildbot stop @var{BASEDIR}} and
@code{buildbot start @var{BASEDIR}} in quick succession, or you can
use the @code{restart} shortcut, which does both steps for you:
@example
buildbot restart @var{BASEDIR}
@end example
Buildslaves can similarly be restarted with:
@example
buildslave restart @var{BASEDIR}
@end example
There are certain configuration changes that are not handled cleanly
by @code{buildbot reconfig}. If this occurs, @code{buildbot restart}
is a more robust tool to fully switch over to the new configuration.
@code{buildbot restart} may also be used to start a stopped Buildbot
instance. This behaviour is useful when writing scripts that stop, start
and restart Buildbot.
A buildslave may also be gracefully shutdown from the
@pxref{WebStatus} status plugin. This is useful to shutdown a
buildslave without interrupting any current builds. The buildmaster
will wait until the buildslave is finished all its current builds, and
will then tell the buildslave to shutdown.
@node Maintenance
@section Maintenance
The buildmaster can be configured to send out email notifications when a
slave has been offline for a while. Be sure to configure the buildmaster
with a contact email address for each slave so these notifications are sent
to someone who can bring it back online.
If you find you can no longer provide a buildslave to the project, please
let the project admins know, so they can put out a call for a
replacement.
The Buildbot records status and logs output continually, each time a
build is performed. The status tends to be small, but the build logs
can become quite large. Each build and log are recorded in a separate
file, arranged hierarchically under the buildmaster's base directory.
To prevent these files from growing without bound, you should
periodically delete old build logs. A simple cron job to delete
anything older than, say, two weeks should do the job. The only trick
is to leave the @file{buildbot.tac} and other support files alone, for
which find's @code{-mindepth} argument helps skip everything in the
top directory. You can use something like the following:
@example
@@weekly cd BASEDIR && find . -mindepth 2 i-path './public_html/*' -prune -o -type f -mtime +14 -exec rm @{@} \;
@@weekly cd BASEDIR && find twistd.log* -mtime +14 -exec rm @{@} \;
@end example
Alternatively, you can configure a maximum number of old logs to be kept
using the @code{--log-count} command line option when running @code{buildbot
create-slave} or @code{buildbot create-master}.
@node Troubleshooting
@section Troubleshooting
Here are a few hints on diagnosing common problems.
@menu
* Starting the buildslave::
* Connecting to the buildmaster::
@end menu
@node Starting the buildslave
@subsection Starting the buildslave
Cron jobs are typically run with a minimal shell (@file{/bin/sh}, not
@file{/bin/bash}), and tilde expansion is not always performed in such
commands. You may want to use explicit paths, because the @code{PATH}
is usually quite short and doesn't include anything set by your
shell's startup scripts (@file{.profile}, @file{.bashrc}, etc). If
you've installed buildbot (or other python libraries) to an unusual
location, you may need to add a @code{PYTHONPATH} specification (note
that python will do tilde-expansion on @code{PYTHONPATH} elements by
itself). Sometimes it is safer to fully-specify everything:
@example
@@reboot PYTHONPATH=~/lib/python /usr/local/bin/buildbot start /usr/home/buildbot/basedir
@end example
Take the time to get the @@reboot job set up. Otherwise, things will work
fine for a while, but the first power outage or system reboot you have will
stop the buildslave with nothing but the cries of sorrowful developers to
remind you that it has gone away.
@node Connecting to the buildmaster
@subsection Connecting to the buildmaster
If the buildslave cannot connect to the buildmaster, the reason should
be described in the @file{twistd.log} logfile. Some common problems
are an incorrect master hostname or port number, or a mistyped bot
name or password. If the buildslave loses the connection to the
master, it is supposed to attempt to reconnect with an
exponentially-increasing backoff. Each attempt (and the time of the
next attempt) will be logged. If you get impatient, just manually stop
and re-start the buildslave.
When the buildmaster is restarted, all slaves will be disconnected, and will
attempt to reconnect as usual. The reconnect time will depend upon how long the
buildmaster is offline (i.e. how far up the exponential backoff curve the
slaves have travelled). Again, @code{buildslave restart @var{BASEDIR}} will
speed up the process.