update user documentation

master/docs/developer/general.rst

@@ -22,3 +22,4 @@ This section gives some general information about Buildbot development.
     br-claiming
     encodings
     metrics
+    plugins-publish

master/docs/developer/plugins-publish.rst

@@ -0,0 +1,86 @@
+===============================
+How to package Buildbot plugins
+===============================
+
+If you customized an existing component (see :doc:`../manual/customization`) or created a new component that you believe might be useful for others, you have two options:
+
+* submit the change to the Buildbot main tree, however you need to adhere to certain requirements (see :doc:`style`)
+* prepare a Python package that contains the functionality you created
+
+Here we cover the second option.
+
+Package the source
+==================
+
+To begin with, you must package your changes.
+If you do not know what a Python package is, these two tutorials will get you going:
+
+* `Installation and Packaging Tutorial <https://packaging.python.org/en/latest/tutorial.html#creating-your-own-project>`_
+* `The Hitchhiker’s Guide to Packaging <http://guide.python-distribute.org/index.html>`_
+
+The former is more recent and, while it addresses everything that you need to know about Python packages, is still work in progress.
+The latter is a bit dated, though it was the most complete guide for quite some time available for Python developers looking to package their software.
+
+You may also want to check the `sample project <https://github.com/pypa/sampleproject>`_, which exemplifies the best Python packaging practices.
+
+Making the plugin package
+=========================
+
+Buildbot supports several kinds of pluggable components:
+
+* ``buildslave``
+* ``changes``
+* ``schedulers``
+* ``steps``
+* ``status``
+* ``util``
+
+(these are described in :doc:`../manual/plugins`), and
+
+* ``www``
+
+which is described in :doc:`web server configuration <../manual/cfg-www>`.
+
+Once you have your component packaged, it's quite straightforward: you just need to add a few lines to the ``entry_points`` parameter of your call of ``setup`` function in :file:`setup.py` file::
+
+    setup(
+        ...
+        entry_points = {
+            ...,
+            'buildbot.kind': [
+                'PluginName = PluginModule:PluginClass'
+            ]
+        },
+        ...
+    )
+
+(You might have seen different ways to specify the value for ``entry_points``, however they all do the same thing.
+Full description of possible ways is available in `setuptools documentation <http://pythonhosted.org/setuptools/setuptools.html#dynamic-discovery-of-services-and-plugins>`_.)
+
+After the :file:`setup.py` file is updated, you can build and install it::
+
+    $ python setup.py build
+    $ sudo python setup.py install
+
+(depending on your particular setup, you might not need to use :command:`sudo`).
+
+After that the plugin should be available for Buildbot and you can use it in your :file:`master.cfg` as::
+
+    from buildbot.kind import PluginName
+
+    ... PluginName ...
+
+Publish the package
+===================
+
+This is the last step before the plugin is available to others.
+
+Once again, there is a number of options available for you:
+
+* just put a link to your version control system
+* prepare a source tarball with the plugin (``python setup.py sdist``)
+* or publish it on `PyPI <https://pypi.python.org>`_
+
+The last option is probably the best one since it will make your plugin available pretty much to all Python developers.
+
+Once you have published the package, please send a link to `buildbot-devel <mailto:buildbot-devel@lists.sourceforge.net>`_ mailing list, so we can include a link to your plugin to :doc:`../manual/plugins`.

master/docs/examples/git_gerrit.cfg

@@ -4,6 +4,8 @@
 # This is a sample buildmaster config file. It must be installed as
 # 'master.cfg' in your buildmaster's base directory.
 
+from buildbot.plugins import *
+
 # This is the dictionary that the buildmaster pays attention to. We also use
 # a shorter alias to save typing.
 c = BuildmasterConfig = {}
@@ -13,8 +15,7 @@ c = BuildmasterConfig = {}
 # The 'slaves' list defines the set of recognized buildslaves. Each element is
 # a BuildSlave object, specifying a unique slave name and password.  The same
 # slave name and password must be configured on the slave.
-from buildbot.buildslave import BuildSlave
-c['slaves'] = [BuildSlave("example-slave", "pass")]
+c['slaves'] = [buildslave.BuildSlave("example-slave", "pass")]
 
 # 'protocols' contains information about protocols which master will use for
 # communicating with slaves.
@@ -35,23 +36,19 @@ gerrit_user = "gerrit"
 gerrit_port = "29418"
 gerrit_project = "mygerritproject"
 
-gerrit_repo = "ssh://" + gerrit_user + "@" + gerrit_url + ":" + gerrit_port + "/" + gerrit_project
+gerrit_repo = "ssh://%s@%s:%s/%s" % (gerrit_user, gerrit_url, gerrit_port,
+                                     gerrit_project)
 
 #Add comment-added to handled_events to have approvals information (Code-Review...)
-from buildbot.changes.gerritchangesource import GerritChangeSource
-c['change_source'] = GerritChangeSource(gerrit_url, gerrit_user,
-                                        handled_events=["patchset-created", "comment-added"]
-                                        )
+c['change_source'] = changes.GerritChangeSource(gerrit_url, gerrit_user,
+                                                handled_events=["patchset-created",
+                                                                "comment-added"])
 
 ####### SCHEDULERS
 
 # Configure the Schedulers, which decide how to react to incoming changes.  In this
 # case, just kick off a 'runtests' build
 
-from buildbot.schedulers.basic import SingleBranchScheduler
-from buildbot.schedulers.forcesched import ForceScheduler
-from buildbot.changes import filter
-
 #Check there is Code-Review=+2 in Approvals (of comment-added)
 
 def change_code_review_plus_2(change):
@@ -63,12 +60,12 @@ def change_code_review_plus_2(change):
 
 
 c['schedulers'] = []
-c['schedulers'].append(SingleBranchScheduler(
+c['schedulers'].append(schedulers.SingleBranchScheduler(
                             name="all",
-                            change_filter=filter.ChangeFilter(branch_re="master/*", filter_fn=change_code_review_plus_2),
+                            change_filter=util.ChangeFilter(branch_re="master/*", filter_fn=change_code_review_plus_2),
                             treeStableTimer=None,
                             builderNames=["runtests-gcc","runtests-clang"]))
-c['schedulers'].append(ForceScheduler(
+c['schedulers'].append(schedulers.ForceScheduler(
                             name="force",
                             builderNames=["runtests-gcc","runtests-clang"]))
 
@@ -78,41 +75,31 @@ c['schedulers'].append(ForceScheduler(
 # what steps, and which slaves can execute them.  Note that any particular build will
 # only take place on one slave.
 
-from buildbot.process.factory import BuildFactory
-#from buildbot.steps.source.git import Git
-from buildbot.steps.source.gerrit import Gerrit
-from buildbot.steps.shell import ShellCommand
-from buildbot.steps.shell import Compile
-from buildbot.steps.shell import Configure
-
 #Build with GCC
-f_gcc = BuildFactory()
-f_gcc.addStep(Gerrit(repourl=gerrit_repo, mode="full",retry=[60,60],timeout=3600))
-f_gcc.addStep(ShellCommand(command=["bash","./autogen.sh"],timeout=3600))
-f_gcc.addStep(Configure(command=["./configure"]))
-f_gcc.addStep(Compile(command=["make", "-j", "4"]))
-f_gcc.addStep(Compile(command=["make", "test"]))
+f_gcc = util.BuildFactory([
+    steps.Gerrit(repourl=gerrit_repo, mode="full",retry=[60,60],timeout=3600),
+    steps.ShellCommand(command=["bash","./autogen.sh"],timeout=3600),
+    steps.Configure(command=["./configure"]),
+    steps.Compile(command=["make", "-j", "4"]),
+    steps.Compile(command=["make", "test"])
+])
 
 #Build with Clang
-f_clang = BuildFactory()
-f_clang.addStep(Gerrit(repourl=gerrit_repo, mode="full",retry=[60,60],timeout=3600))
-f_clang.addStep(ShellCommand(command=["bash","./autogen.sh"],timeout=3600))
-f_clang.addStep(Configure(command=["./configure"],env={ "CC":"clang", "CXX":"clang++"}))
-f_clang.addStep(Compile(command=["make", "-j", "4"]))
-f_clang.addStep(Compile(command=["make", "test"]))
-
-from buildbot.config import BuilderConfig
+f_clang = BuildFactory([
+    steps.Gerrit(repourl=gerrit_repo, mode="full",retry=[60,60],timeout=3600),
+    steps.ShellCommand(command=["bash","./autogen.sh"],timeout=3600),
+    steps.Configure(command=["./configure"],env={ "CC":"clang", "CXX":"clang++"}),
+    steps.Compile(command=["make", "-j", "4"]),
+    steps.Compile(command=["make", "test"])
+])
 
 c['builders'] = []
 c['builders'].append(
-    BuilderConfig(name="runtests-gcc",
-      slavenames=["example-slave"],
-      factory=f_gcc))
-
+    util.BuilderConfig(name="runtests-gcc", slavenames=["example-slave"],
+                       factory=f_gcc))
 c['builders'].append(
-    BuilderConfig(name="runtests-clang",
-      slavenames=["example-slave"],
-      factory=f_clang))
+    util.BuilderConfig(name="runtests-clang", slavenames=["example-slave"],
+                       factory=f_clang))
 
 
 ####### STATUS TARGETS
@@ -123,9 +110,6 @@ c['builders'].append(
 
 c['status'] = []
 
-from buildbot.status import html
-from buildbot.status.web import authz, auth
-
 authz_cfg=authz.Authz(
     # change any of these to True to enable; see the manual for more
     # options
@@ -140,25 +124,24 @@ authz_cfg=authz.Authz(
 )
 c['status'].append(html.WebStatus(http_port=8010, authz=authz_cfg))
 
-from buildbot.status.status_gerrit import GerritStatusPush
-from buildbot.status.builder import Results, SUCCESS, RETRY
-
 def gerritReviewCB(builderName, build, result, status, arg):
-    if result == RETRY:
+    if result == util.RETRY:
         return dict()
 
     message =  "Buildbot finished compiling your patchset\n"
     message += "on configuration: %s\n" % builderName
-    message += "The result is: %s\n" % Results[result].upper()
+    message += "The result is: %s\n" % util.Results[result].upper()
 
     if arg:
         message += "\nFor more details visit:\n"
         message += status.getURLForThing(build) + "\n"
 
-    return dict(message=message,
-                labels={
-                    'Verified': 1 if result == SUCCESS or -1
-                })
+    if result == util.SUCCESS:
+        verified = 1
+    else:
+        verified = -1
+
+    return dict(message=message, labels={'Verified': verified})
 
 def gerritStartCB(builderName, build, arg):
     print "gerritStartCB..."
@@ -200,13 +183,13 @@ def gerritSummaryCB(buildInfoList, results, status, arg):
                 })
 
 c['buildbotURL'] = 'http://buildbot.example.com/'
-c['status'].append(GerritStatusPush(gerrit_url, gerrit_user,
-                                    reviewCB=gerritReviewCB,
-                                    reviewArg=c['buildbotURL'],
-                                    startCB=gerritStartCB,
-                                    startArg=c['buildbotURL'],
-                                    summaryCB=gerritSummaryCB,
-                                    summaryArg=c['buildbotURL']))
+c['status'].append(status.GerritStatusPush(gerrit_url, gerrit_user,
+                                           reviewCB=gerritReviewCB,
+                                           reviewArg=c['buildbotURL'],
+                                           startCB=gerritStartCB,
+                                           startArg=c['buildbotURL'],
+                                           summaryCB=gerritSummaryCB,
+                                           summaryArg=c['buildbotURL']))
 
 ####### PROJECT IDENTITY
 

master/docs/manual/cfg-builders.rst

@@ -16,10 +16,10 @@ For more information on the function of Builders in Buildbot, see :ref:`the Conc
 The class definition for the builder configuration is in :file:`buildbot.config`.
 However there is a much simpler way to use it, so in the configuration file, its use looks like::
 
-    from buildbot.config import BuilderConfig
+    from buildbot.plugins import util
     c['builders'] = [
-        BuilderConfig(name='quick', slavenames=['bot1', 'bot2'], factory=f_quick),
-        BuilderConfig(name='thorough', slavename='bot1', factory=f_thorough),
+        util.BuilderConfig(name='quick', slavenames=['bot1', 'bot2'], factory=f_quick),
+        util.BuilderConfig(name='thorough', slavename='bot1', factory=f_thorough),
     ]
 
 ``BuilderConfig`` takes the following keyword arguments:

master/docs/manual/cfg-buildfactories.rst

@@ -29,12 +29,11 @@ A :class:`BuildFactory` defines the steps that every build will follow.  Think o
 a glorified script.  For example, a build factory which consists of an SVN checkout
 followed by a ``make build`` would be configured as follows::
 
-    from buildbot.steps import svn, shell
-    from buildbot.process import factory
+    from buildbot.plugins import util, steps
 
-    f = factory.BuildFactory()
-    f.addStep(svn.SVN(svnurl="http://..", mode="incremental"))
-    f.addStep(shell.Compile(command=["make", "build"]))
+    f = util.BuildFactory()
+    f.addStep(steps.SVN(svnurl="http://..", mode="incremental"))
+    f.addStep(steps.Compile(command=["make", "build"]))
 
 This factory would then be attached to one builder (or several, if desired)::
 
@@ -46,15 +45,13 @@ Using :meth:`addStep` is usually simpler, but there are cases where it is more c
 
 ::
 
-    from buildbot.steps import source, shell
-    from buildbot.process import factory
-
+    from buildbot.plugins import steps, util
 
     all_steps = [
-        source.CVS(cvsroot=CVSROOT, cvsmodule="project", mode="update"),
-        shell.Compile(command=["make", "build"]),
+        steps.CVS(cvsroot=CVSROOT, cvsmodule="project", mode="update"),
+        steps.Compile(command=["make", "build"]),
     ]
-    f = factory.BuildFactory(all_steps)
+    f = util.BuildFactory(all_steps)
 
 Finally, you can also add a sequence of steps all at once::
 
@@ -67,7 +64,7 @@ The following attributes can be set on a build factory after it is created, e.g.
 
 ::
 
-    f = factory.BuildFactory()
+    f = util.BuildFactory()
     f.useProgress = False
 
 :attr:`useProgress`
@@ -132,8 +129,8 @@ the default values will be suitable.
 
 Example::
 
-    f = factory.GNUAutoconf(source=source.SVN(svnurl=URL, mode="copy"),
-                            flags=["--disable-nls"])
+    f = util.GNUAutoconf(source=source.SVN(svnurl=URL, mode="copy"),
+                         flags=["--disable-nls"])
 
 Required Arguments: