Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Buildbot howto (from Pavel)

solin edited this page · 23 revisions

I wrote down these instructions when I was learning how to set up the buildbot for the first time, so do not expect anything sophisticated. I used instructions from http://buildbot.net/buildbot/docs/0.8.1 but cloned the latest repository from git://github.com/buildbot/buildbot.git.

Our buildbot consists of one master machine running one or more buildmaster daemons (for hermes, hermes-tutorial, hermes-examples, femhub etc.) and one or more slave computers that run the corresponding buildslaves.

Requirements

  • Common requirements: git, python, python-twisted, python-zope

  • Additional buildmaster requirements: sqlite3, python-simplejson, python-jinja2

STEP 1: Clone and install buildmaster

On the master computer, in your folder with repos, clone the buildbot repository from Github::

git clone git://github.com/buildbot/buildbot.git

On the master computer, install buildmaster by going to the subfolder master/ and typing::

python setup.py build
sudo python setup.py install

STEP 2: Clone and install buildslave

On the slave computer, install buildslave by going to the subfolder slave/ and typing::

python setup.py build
sudo python setup.py install

Test the versions::

buildbot --version
buildslave -- version

STEP 3: Test the installations

On both the master and slave computers)::

sudo apt-get install python-pip
sudo pip install mock

On the master computer::

PYTHONPATH=. trial buildbot.test      

On the slave computer::

PYTHONPATH=. trial buildslave.test

STEP 4: Upgrade existing buildmasters

If you have existing buildmasters on the master computer, upgrade them. For example on spilka do::

sudo buildbot upgrade-master home/buildmaster/Buildmasters/femhub
sudo buildbot upgrade-master home/buildmaster/Buildmasters/hermes
etc.

STEP 5: Create new buildmasters

If you want to create new buildmasters (say hermes-examples and hermes-tutorial), you should first create a new user account "buildmaster". Then as "buildmaster" in its home directory do::

buildmaster@spilka:~/Buildmasters$ buildbot create-master -r hermes-examples
buildmaster@spilka:~/Buildmasters$ buildbot create-master -r hermes-tutorial

Check / edit / create the master.cfg file in each buildmaster's directory. These files are essential for the correct operation of the buildbot. In particular, this file contains names and passwords of buildlaves as well as their ports.

STEP 6: Create buildslaves

On the slave computer, create new user "buildslave". In its home directory, as "buildslave", create a subfolder "BuildSlaves". Say that we want to create a buildslave "hermes". Issue::

buildslave create-slave ./hermes buildbot.hpfem.org:8004 ubuntu64 lamp_R3

Here ./hermes is BASEDIR, buildbot.hpfem.org:8004 is MASTERHOST:PORT, ubuntu64 is SLAVENAME, and lamp_R3 is PASSWORD. These must match the master.cfg file of the corresponding buildmaster on the master computer.

Also, edit the file buildbot.tac in BASEDIR, and adjust at its end the port to 9994 (for hermes), 9995 (for hermes-examples) etc. You also need to do use the SLAVENAME and PASSWORD there. Again this must match the master.cfg file of the corresponding buildmaster.

STEP 7: Start the buildmaster daemon

This is done as root by typing in the corresponding directory::

root@spilka:/home/buildmaster/Buildmasters/hermes# buildbot start

Or, if you are not in that directory, you can pass the directory name as command-line argument::

buildmaster start /home/buildmaster/Buildmasters/hermes

You may get a message that some ports are already in use, which means that you need to return to master.cfg and change them. In general, one port is needed for communication between buildmaster and buildslave (I chose 9994 for hermes, 9995 for hermes-examples, etc.) and one more is needed to display the buildbot output (I chose 8004 for hermes, 8005 for hermes-examples, etc.).

You should see new files twistd.pid (with the process ID) and twistd.log with the log message. Use this ID to kill the process if there were any errors. If the start was successful, then the log message should end with something positive, such as::

...
2011-10-10 18:00:00-0700 [-] adding IStatusReceiver <buildbot.status.mail.MailNotifier instance at 0x3055bd8>
2011-10-10 18:00:00-0700 [-] removing 0 old schedulers, updating 0, and adding 1
2011-10-10 18:00:00-0700 [-] adding 1 new changesources, removing 0
2011-10-10 18:00:00-0700 [-] configuration update complete

STEP 8: Start the buildslave daemon

In our case on the slave computer "yuba". Use "buildslave start". Again, either be in the correct directory or supply it as a command-line argument::

root@yuba:/home/buildslave/Buildslaves/hermes# buildslave start

Again there is a twistd.pid file with the ID of the buildslave process. If everything went OK, then you should see something like::

Following twistd.log until startup finished..
Removing stale pidfile /home/buildslave/Buildslaves/hermes/twistd.pid
2011-10-10 18:15:34-0700 [-] Log opened.
2011-10-10 18:15:34-0700 [-] twistd 10.2.0 (/usr/bin/python 2.7.1) starting up.
2011-10-10 18:15:34-0700 [-] reactor class: twisted.internet.selectreactor.SelectReactor.
2011-10-10 18:15:34-0700 [-] Applying patch for http://twistedmatrix.com/trac/ticket/4881
2011-10-10 18:15:34-0700 [-] Starting BuildSlave -- version: 0.8.5-67-gf8255e1
2011-10-10 18:15:34-0700 [-] recording hostname in twistd.hostname
2011-10-10 18:15:34-0700 [-] Starting factory <buildslave.bot.BotFactory instance at 0x140ce18>
2011-10-10 18:15:34-0700 [-] Connecting to buildbot.hpfem.org:9994
2011-10-10 18:15:34-0700 [Broker,client] message from master: attached
The buildslave appears to have (re)started correctly.

STEP 9: Test that changes can be sent to buildbot

For this there is a file (that I created on spilka) /var/www3/hooks/output/command_for_buildbot.txt. Run the command once more in the directory /var/www3/hooks/ and you should see something like::

root@spilka:/var/www3/hooks# buildbot sendchange --branch=master --master=buildbot.hpfem.org:9994 --who='Pavel Solin' --comments='Debug buildbot B' 'TEST'
change sent successfully

Note: the port 9994 is for the hermes buildbot only, as defined in the corresponding file master.cfg.

If the send was successful, then you should see (for "hermes") a new directory /home/buildslave/Buildslaves/hermes/yuba/build/ with the hermes.git repo. If the directory was already there, you should see in it latest changes (wait a little bit for the data to arrive).

Make sure that you are able to build the project manually. So in the build/ directory, type "cmake ." and "make". Until the project builds successfully, it makes no sense to go further.

Something went wrong with that request. Please try again.