Skip to content

Commit

Permalink
make formatting related changes to the affected documents
Browse files Browse the repository at this point in the history 
* this will help the rebase should they have changed in the base branch
* do some whitespace cleanup (trailing whitespaces, empty lines)
* move change hooks to the end of customization.rst (relies on
  non-existent feature, hence it's not clear how to use it)
* a bit of extra formatting (not part of this change, but anyway)
  • Loading branch information
Mikhail Sobolev committed Aug 14, 2014
1 parent 186529a commit e2f3548
Show file tree
Hide file tree
Showing 13 changed files with 616 additions and 554 deletions.
7 changes: 2 additions & 5 deletions master/docs/developer/index.rst
View file
Expand Up @@ -3,15 +3,12 @@
Buildbot Development
====================

This chapter is the official repository for the collected wisdom of the
Buildbot hackers. It is intended both for developers writing patches that will
be included in Buildbot itself, and for advanced users who wish to customize
Buildbot.
This chapter is the official repository for the collected wisdom of the Buildbot hackers.
It is intended both for developers writing patches that will be included in Buildbot itself, and for advanced users who wish to customize Buildbot.

.. toctree::
:maxdepth: 2

general
apis
classes

9 changes: 4 additions & 5 deletions master/docs/manual/cfg-builders.rst
View file
Expand Up @@ -11,11 +11,10 @@ Builder Configuration
:depth: 1
:local:

The :bb:cfg:`builders` configuration key is a list of objects giving
configuration for the Builders. For more information on the function of
Builders in Buildbot, see :ref:`the Concepts chapter <Builder>`. The class
definition for the builder configuration is in :file:`buildbot.config`. In the
configuration file, its use looks like::
The :bb:cfg:`builders` configuration key is a list of objects giving configuration for the Builders.
For more information on the function of Builders in Buildbot, see :ref:`the Concepts chapter <Builder>`.
The class definition for the builder configuration is in :file:`buildbot.config`.
However there is a much simpler way to use it, so in the configuration file, its use looks like::

from buildbot.config import BuilderConfig
c['builders'] = [
Expand Down
21 changes: 10 additions & 11 deletions master/docs/manual/cfg-buildfactories.rst
View file
Expand Up @@ -41,15 +41,15 @@ This factory would then be attached to one builder (or several, if desired)::
c['builders'].append(
BuilderConfig(name='quick', slavenames=['bot1', 'bot2'], factory=f))

It is also possible to pass a list of steps into the
:class:`BuildFactory` when it is created. Using :meth:`addStep` is
usually simpler, but there are cases where it is more convenient to
create the list of steps ahead of time, perhaps using some Python
tricks to generate the steps. ::
It is also possible to pass a list of steps into the :class:`BuildFactory` when it is created.
Using :meth:`addStep` is usually simpler, but there are cases where it is more convenient to create the list of steps ahead of time, perhaps using some Python tricks to generate the steps.

::

from buildbot.steps import source, shell
from buildbot.process import factory


all_steps = [
source.CVS(cvsroot=CVSROOT, cvsmodule="project", mode="update"),
shell.Compile(command=["make", "build"]),
Expand All @@ -63,7 +63,9 @@ Finally, you can also add a sequence of steps all at once::
Attributes
~~~~~~~~~~

The following attributes can be set on a build factory after it is created, e.g., ::
The following attributes can be set on a build factory after it is created, e.g.,

::

f = factory.BuildFactory()
f.useProgress = False
Expand Down Expand Up @@ -428,9 +430,6 @@ Arguments:
run the test cases in random order, which sometimes catches subtle
inter-test dependency bugs. Defaults to ``False``.

The step can also take any of the :class:`ShellCommand` arguments, e.g.,
:attr:`haltOnFailure`.

Unless one of ``tests`` or ``testChanges`` are set, the step will
generate an exception.
The step can also take any of the :class:`ShellCommand` arguments, e.g., :attr:`haltOnFailure`.

Unless one of ``tests`` or ``testChanges`` are set, the step will generate an exception.
106 changes: 66 additions & 40 deletions master/docs/manual/cfg-buildslaves.rst
View file
Expand Up @@ -182,6 +182,11 @@ The following options are available for all latent buildslaves.
Amazon Web Services Elastic Compute Cloud ("AWS EC2")
+++++++++++++++++++++++++++++++++++++++++++++++++++++

..
XXX(sa2ajj): is it a good candidate to be put into a separate file?
The point is: tell about availability, but the detailed guide in a separate doc?

`EC2 <http://aws.amazon.com/ec2/>`_ is a web service that allows you to
start virtual machines in an Amazon data center. Please see their website for
details, including costs. Using the AWS EC2 latent buildslaves involves getting
Expand Down Expand Up @@ -261,8 +266,10 @@ of this writing include ``m1.small``, ``m1.large``, ``m1.xlarge``, ``c1.medium``
and ``c1.xlarge``; see the EC2 documentation for descriptions of these
machines).

Here is the simplest example of configuring an EC2 latent buildslave. It
specifies all necessary remaining values explicitly in the instantiation. ::
Here is the simplest example of configuring an EC2 latent buildslave.
It specifies all necessary remaining values explicitly in the instantiation.

::

from buildbot.buildslave.ec2 import EC2LatentBuildSlave
c['slaves'] = [EC2LatentBuildSlave('bot1', 'sekrit', 'm1.large',
Expand All @@ -283,11 +290,12 @@ the account information can be specified in alternate ways.
separate file, and potentially let more people read your main configuration
file.

By default, you can make an :file:`.ec2` directory in the home folder of the user
running the buildbot master. In that directory, create a file called :file:`aws_id`.
The first line of that file should be your access key id; the second line
should be your secret access key id. Then you can instantiate the build slave
as follows. ::
By default, you can make an :file:`.ec2` directory in the home folder of the user running the buildbot master.
In that directory, create a file called :file:`aws_id`.
The first line of that file should be your access key id; the second line should be your secret access key id.
Then you can instantiate the build slave as follows.

::

from buildbot.buildslave.ec2 import EC2LatentBuildSlave
c['slaves'] = [EC2LatentBuildSlave('bot1', 'sekrit', 'm1.large',
Expand All @@ -304,9 +312,9 @@ specify one or two AMI filters.
In all cases, the AMI that sorts last by its location (the S3 bucket and
manifest name) will be preferred.

One available filter is to specify the acceptable AMI owners, by AWS account
number (the 12 digit number, usually rendered in AWS with hyphens like
"1234-5678-9012", should be entered as in integer). ::
One available filter is to specify the acceptable AMI owners, by AWS account number (the 12 digit number, usually rendered in AWS with hyphens like "1234-5678-9012", should be entered as in integer).

::

from buildbot.buildslave.ec2 import EC2LatentBuildSlave
bot1 = EC2LatentBuildSlave('bot1', 'sekrit', 'm1.large',
Expand All @@ -316,38 +324,43 @@ number (the 12 digit number, usually rendered in AWS with hyphens like
secret_identifier='privatekey'
)

The other available filter is to provide a regular expression string that
will be matched against each AMI's location (the S3 bucket and manifest name). ::
The other available filter is to provide a regular expression string that will be matched against each AMI's location (the S3 bucket and manifest name).

::

from buildbot.buildslave.ec2 import EC2LatentBuildSlave
bot1 = EC2LatentBuildSlave(
'bot1', 'sekrit', 'm1.large',
valid_ami_location_regex=r'buildbot\-.*/image.manifest.xml',
identifier='publickey', secret_identifier='privatekey')

The regular expression can specify a group, which will be preferred for the
sorting. Only the first group is used; subsequent groups are ignored. ::
The regular expression can specify a group, which will be preferred for the sorting.
Only the first group is used; subsequent groups are ignored.

::

from buildbot.buildslave.ec2 import EC2LatentBuildSlave
bot1 = EC2LatentBuildSlave(
'bot1', 'sekrit', 'm1.large',
valid_ami_location_regex=r'buildbot\-.*\-(.*)/image.manifest.xml',
identifier='publickey', secret_identifier='privatekey')

If the group can be cast to an integer, it will be. This allows 10 to sort
after 1, for instance. ::
If the group can be cast to an integer, it will be.
This allows 10 to sort after 1, for instance.

::

from buildbot.buildslave.ec2 import EC2LatentBuildSlave
bot1 = EC2LatentBuildSlave(
'bot1', 'sekrit', 'm1.large',
valid_ami_location_regex=r'buildbot\-.*\-(\d+)/image.manifest.xml',
identifier='publickey', secret_identifier='privatekey')

In addition to using the password as a handshake between the master and the
slave, you may want to use a firewall to assert that only machines from a
specific IP can connect as slaves. This is possible with AWS EC2 by using
the Elastic IP feature. To configure, generate a Elastic IP in AWS, and then
specify it in your configuration using the ``elastic_ip`` argument. ::
In addition to using the password as a handshake between the master and the slave, you may want to use a firewall to assert that only machines from a specific IP can connect as slaves.
This is possible with AWS EC2 by using the Elastic IP feature.
To configure, generate a Elastic IP in AWS, and then specify it in your configuration using the ``elastic_ip`` argument.

::

from buildbot.buildslave.ec2 import EC2LatentBuildSlave
c['slaves'] = [EC2LatentBuildSlave('bot1', 'sekrit', 'm1.large',
Expand All @@ -357,9 +370,11 @@ specify it in your configuration using the ``elastic_ip`` argument. ::
elastic_ip='208.77.188.166'
)]

One other way to configure a slave is by settings AWS tags. They can for example be used to
have a more restrictive security `IAM <http://aws.amazon.com/iam/>`_ policy. To get Buildbot to tag the latent slave
specify the tag keys and values in your configuration using the ``tags`` argument. ::
One other way to configure a slave is by settings AWS tags.
They can for example be used to have a more restrictive security `IAM <http://aws.amazon.com/iam/>`_ policy.
To get Buildbot to tag the latent slave specify the tag keys and values in your configuration using the ``tags`` argument.

::

from buildbot.buildslave.ec2 import EC2LatentBuildSlave
c['slaves'] = [EC2LatentBuildSlave('bot1', 'sekrit', 'm1.large',
Expand All @@ -384,10 +399,10 @@ these AWS EC2 values. They both default to ``latent_buildbot_slave``.
Spot instances
##############

If you would prefer to use spot instances for running your builds, you can accomplish that
by passing in a True value to the ``spot_instance`` parameter to the EC2LatentBuildSlave
constructor. Additionally, you may want to specify ``max_spot_price`` and ``price_multiplier``
in order to limit your builds' budget consumption. ::
If you would prefer to use spot instances for running your builds, you can accomplish that by passing in a True value to the ``spot_instance`` parameter to the EC2LatentBuildSlave constructor.
Additionally, you may want to specify ``max_spot_price`` and ``price_multiplier`` in order to limit your builds' budget consumption.

::

from buildbot.buildslave.ec2 import EC2LatentBuildSlave
c['slaves'] = [EC2LatentBuildSlave('bot1', 'sekrit', 'm1.large',
Expand All @@ -409,12 +424,11 @@ parameter, then a spot request will be sent to Amazon with the above details.
When a spot request fails
#########################

In some cases Amazon may reject a spot request because the spot price, determined by taking
the 24-hour average of that availability zone's spot prices for the given product description,
is lower than the current price. The optional parameters ``retry`` and ``retry_price_adjustment``
allow for resubmitting the spot request with an adjusted price. If the spot request continues to
fail, and the number of attempts exceeds the value of the ``retry`` parameter, an error message
will be logged. ::
In some cases Amazon may reject a spot request because the spot price, determined by taking the 24-hour average of that availability zone's spot prices for the given product description, is lower than the current price.
The optional parameters ``retry`` and ``retry_price_adjustment`` allow for resubmitting the spot request with an adjusted price.
If the spot request continues to fail, and the number of attempts exceeds the value of the ``retry`` parameter, an error message will be logged.

::

from buildbot.buildslave.ec2 import EC2LatentBuildSlave
c['slaves'] = [EC2LatentBuildSlave('bot1', 'sekrit', 'm1.large',
Expand All @@ -441,6 +455,9 @@ as normal and run any pending builds.
Libvirt
+++++++

..
XXX(sa2ajj): Ditto
`libvirt <http://www.libvirt.org/>`_ is a virtualization API for interacting
with the virtualization capabilities of recent versions of Linux and other OSes.
It is LGPL and comes with a stable C API, and Python bindings.
Expand Down Expand Up @@ -508,7 +525,9 @@ Configuring your Master
If you want to add a simple on demand VM to your setup, you only need the following. We
set the username to ``minion1``, the password to ``sekrit``. The base image is called ``base_image``
and a copy of it will be made for the duration of the VM's life. That copy will be thrown
away every time a build is complete. ::
away every time a build is complete.

::

from buildbot.buildslave.libvirt import LibVirtSlave, Connection
c['slaves'] = [LibVirtSlave('minion1', 'sekrit', Connection("qemu:///session"),
Expand Down Expand Up @@ -542,6 +561,10 @@ won't be able to find a VM to start.

OpenStack
+++++++++

..
XXX(sa2ajj): Ditto
`OpenStack <http://openstack.org/>`_ is a series of interconnected components
that facilitates managing compute, storage, and network resources in a
data center. It is available under the Apache License and has a REST interface
Expand Down Expand Up @@ -599,7 +622,9 @@ passed as options to an OpenStack client.
A dictionary of string key-value pairs to pass to the instance. These will
be available under the ``metadata`` key from the metadata service.

Here is the simplest example of configuring an OpenStack latent buildslave. ::
Here is the simplest example of configuring an OpenStack latent buildslave.

::

from buildbot.buildslave.openstack import OpenStackLatentBuildSlave
c['slaves'] = [OpenStackLatentBuildSlave('bot2', 'sekrit',
Expand All @@ -608,10 +633,11 @@ Here is the simplest example of configuring an OpenStack latent buildslave. ::
os_tenant_name='tenant',
os_auth_url='http://127.0.0.1:35357/v2.0')]

The ``image`` argument also supports being given a callable. The callable will
be passed the list of available images and must return the image to use. The
invocation happens in a separate thread to prevent blocking the build master
when interacting with OpenStack. ::
The ``image`` argument also supports being given a callable.
The callable will be passed the list of available images and must return the image to use.
The invocation happens in a separate thread to prevent blocking the build master when interacting with OpenStack.

::

from buildbot.buildslave.openstack import OpenStackLatentBuildSlave

Expand Down

0 comments on commit e2f3548

Please sign in to comment.