Merge branch 'multirepo-doc' of git://github.com/hborkhuis/buildbot

* 'multirepo-doc' of git://github.com/hborkhuis/buildbot:
  rewriting of some sentences multiple-tree concept (by Benoit)
  document that by default schedulers only accept changes with an empty sourcestamp
  added the concept of multiple repositories to documentation

Conflicts:
	master/docs/release-notes.rst

master/docs/manual/cfg-global.rst

@@ -737,20 +737,18 @@ Codebase Generator
 
 ::
 
+    all_repositories = {
+        r'https://hg/hg/mailsuite/mailclient': 'mailexe',
+        r'https://hg/hg/mailsuite/mapilib': 'mapilib',
+        r'https://hg/hg/mailsuite/imaplib': 'imaplib',
+        r'https://github.com/mailinc/mailsuite/mailclient': 'mailexe',
+        r'https://github.com/mailinc/mailsuite/mapilib': 'mapilib',
+        r'https://github.com/mailinc/mailsuite/imaplib': 'imaplib',
+    }
+
     def codebaseGenerator(chdict):
-        all_repositories = {
-            r'https://hg/hg/mailsuite/mailclient': 'mailexe',
-            r'https://hg/hg/mailsuite/mapilib': 'mapilib',
-            r'https://hg/hg/mailsuite/imaplib': 'imaplib',
-            r'https://github.com/mailinc/mailsuite/mailclient': 'mailexe',
-            r'https://github.com/mailinc/mailsuite/mapilib': 'mapilib',
-            r'https://github.com/mailinc/mailsuite/imaplib': 'imaplib',
-        }
-        if chdict['repository'] in all_repositories:
-            return all_repositories[chdict['repository']]
-        else:
-            return ''
-            
+        return all_repositories[chdict['repository']]
+
     c['codebaseGenerator'] = codebaseGenerator
 
 For any incomming change a :ref:`codebase<Attr-Codebase>` is set to ''. This

master/docs/manual/cfg-schedulers.rst

@@ -73,14 +73,21 @@ available with all schedulers.
     When the scheduler processes data from more than 1 repository at the
     same time then a corresponding codebase definition should be passed for each
     repository. A codebase definition is a dictionary with one or more of the 
-    following keys: repository, branch, revision. The codebase definitions are 
-    also to be passed as dictionary::
-        
-        codebases = {'codebase1': {'repository':'....', ...}, 'codebase2': {} }
+    following keys: repository, branch, revision. The codebase definitions have
+    also to be passed as dictionary.
+
+    .. code-block:: python
+
+        codebases = {'codebase1': {'repository':'....',
+                                   'branch':'default',
+                                   'revision': None},
+                     'codebase2': {'repository':'....'} }
 
     .. IMPORTANT:: ``codebases`` behaves also like a change_filter on codebase.
-        If ``codebases`` is set then the scheduler will only process changes  when their 
-        codebases are found in ``codebases``
+        The scheduler will only process changes  when their codebases are found
+        in ``codebases``. By default ``codebases`` is set to ``{'':{}}`` which
+        means that only changes with codebase '' (default value for codebase)
+        will be accepted by the scheduler.
 
     Buildsteps can have a reference to one of the codebases. The step will only
     get information (revision, branch etc.)  that is related to that codebase.

master/docs/manual/concepts.rst

@@ -131,6 +131,61 @@ that revision marker. When the VC system does not provide this, a
 timestamp from the middle of the tree-stable period is used to
 generate the source tree [#]_.
 
+.. _Multiple-source-trees:
+
+Multiple source-trees
++++++++++++++++++++++
+
+What if an end-product is composed of several components and all these components
+have different repositories? Changes may arrive from different repositories within
+the tree-stable-timer period. Buildbot will not only use the source-trees that
+contain changes but also needs the remaining source-trees to build the complete
+product. 
+
+For this reason a :ref:`Scheduler<Scheduling-Builds>` can be configured to base a build on a set
+of several source-trees that can (partly) be overidden by the specific revisions
+inside the arrived :class:`Change`\s.
+
+
+A single source-tree is identified by its repository, branch and revision. The
+repository is a reference to a place where the sources reside. In projects it is
+possible to have the same sources at different places (for example a collection
+of fork of the same repository). For this reason an extra identifier 'codebase'
+has been introduced to make buildbot categorize all copies of the same source-tree
+as the same. Furthermore, the codebase_ allow you to uniquely identify a *part* of
+your project.
+
+A complete multiple repository configuration consists of:
+
+    - a *codebase generator*
+
+        Every relevant change arriving from a VC must contain a codebase.
+        Because VC's do not supply a codebase this has to be done by a so
+        called :bb:cfg:`codebaseGenerator` that is defined in the
+        configuration.
+
+    - some *schedulers*
+
+        The :bb:cfg:`scheduler<schedulers>` has to be configured with a set of all
+        required ``codebases`` to build a product. These codebases indicate the set of
+        required source-trees. In order for the scheduler to be able to produce a
+        complete set for each build, each given codebase will be supplied a default
+        repository, branch, and revision. Those ones will be used for the codebases
+        where no changes arrived.
+
+    - multiple *source steps*
+
+        A :ref:`Builder` has to contains a :ref:`source step<Source-Checkout>` for
+        each needed source-tree. Each of the source steps has a ``codebase`` attribute
+        which is used to retrieve the corresponding repository, branch and revision
+        data. This information comes from the arrived changes or from the
+        scheduler's configured default values.
+
+.. warning::
+
+    Defining a :bb:cfg:`codebaseGenerator` that returns non empty (not
+    ``''``) codebases, change the behavior of all the schedulers.
+
 .. _How-Different-VC-Systems-Specify-Sources:
 
 How Different VC Systems Specify Sources

master/docs/release-notes.rst

@@ -81,13 +81,14 @@ Features
   schedulers can aggregate changes to multiple codebases into source stamp sets
   (with one source stamp for each codebase).  Source steps then check out each
   codebase as required, and the remainder of the build process proceeds
-  normally.  See the documentation for details.
+  normally.  See the :ref:`Multiple-source-trees` for details.
 
 * ``Source`` and ``ShellCommand`` steps now have an optional ``descriptionSuffix``, a suffix to the
    ``description``/``descriptionDone`` values. For example this can help distinguish between
     multiple ``Compile`` steps that are applied to different codebases.
 
 * ``Git`` has a new ``getDescription`` option, which will run `git describe` after checkout
+  normally.  See the documentation for details.
 
 Slave
 -----