Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Add a web hook handler for Google Code push notifications #291

Merged
merged 2 commits into from over 2 years ago

2 participants

Louis Opter Dustin J. Mitchell
Louis Opter
  • Supports authentication;
  • Only supports pushes on the `default' branch (Google Code doesn't send the branch name);
  • Google Code entirely sends the changes as an UTF-8 JSON dictionary (as opposed to something like a form-data);
  • Very similar to the GitHub web hook handler.
and others added some commits December 18, 2011
Add a web hook handler for Google Code push notifications
- Supports authentication;
- Only supports pushes on the `default' branch (Google Code doesn't send
  the branch name);
- Google Code entirely sends the changes as an UTF-8 JSON dictionary (as
  opposed to something like a form-data);
- Very similar to the GitHub web hook handler.
350a3b0
Louis Opter Document the Google Code Web Hook handler and make the branch returne…
…d an option

- You can now choose the branch returned by the web hook handler with
  the 'branch' option;
- Also, check if a 'branch' field exists in the JSON dictionary sent by
  Google Code (in the hope that they add it one day…);
- Adjust test accordingly.
0a812c5
Dustin J. Mitchell djmitche merged commit 0a812c5 into from December 18, 2011
Dustin J. Mitchell djmitche closed this December 18, 2011
Dustin J. Mitchell
Owner

Merged - thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 2 unique commits by 2 authors.

Dec 18, 2011
Add a web hook handler for Google Code push notifications
- Supports authentication;
- Only supports pushes on the `default' branch (Google Code doesn't send
  the branch name);
- Google Code entirely sends the changes as an UTF-8 JSON dictionary (as
  opposed to something like a form-data);
- Very similar to the GitHub web hook handler.
350a3b0
Louis Opter Document the Google Code Web Hook handler and make the branch returne…
…d an option

- You can now choose the branch returned by the web hook handler with
  the 'branch' option;
- Also, check if a 'branch' field exists in the JSON dictionary sent by
  Google Code (in the hope that they add it one day…);
- Adjust test accordingly.
0a812c5
This page is out of date. Refresh to see the latest.
97  master/buildbot/status/web/hooks/googlecode.py
... ...
@@ -0,0 +1,97 @@
  1
+# This file is part of Buildbot.  Buildbot is free software: you can
  2
+# redistribute it and/or modify it under the terms of the GNU General Public
  3
+# License as published by the Free Software Foundation, version 2.
  4
+#
  5
+# This program is distributed in the hope that it will be useful, but WITHOUT
  6
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
  7
+# FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more
  8
+# details.
  9
+#
  10
+# You should have received a copy of the GNU General Public License along with
  11
+# this program; if not, write to the Free Software Foundation, Inc., 51
  12
+# Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
  13
+#
  14
+# Copyright 2011, Louis Opter <kalessin@kalessin.fr>
  15
+
  16
+# Quite inspired from the github hook.
  17
+
  18
+import hmac
  19
+import logging
  20
+import sys
  21
+import traceback
  22
+
  23
+from twisted.python import log
  24
+
  25
+from buildbot.util import json
  26
+
  27
+class GoogleCodeAuthFailed(Exception):
  28
+    pass
  29
+
  30
+class Payload(object):
  31
+    def __init__(self, headers, body, branch):
  32
+        self._auth_code = headers['Google-Code-Project-Hosting-Hook-Hmac']
  33
+        self._body = body # we need to save it if we want to authenticate it
  34
+        self._branch = branch
  35
+
  36
+        payload = json.loads(body)
  37
+        self.project = payload['project_name']
  38
+        self.repository = payload['repository_path']
  39
+        self.revisions = payload['revisions']
  40
+        self.revision_count = payload['revision_count']
  41
+
  42
+    def authenticate(self, secret_key):
  43
+        m = hmac.new(secret_key)
  44
+        m.update(self._body)
  45
+        digest = m.hexdigest()
  46
+        return digest == self._auth_code
  47
+
  48
+    def changes(self):
  49
+        changes = []
  50
+
  51
+        for r in self.revisions:
  52
+            files = set()
  53
+            files.update(r['added'], r['modified'], r['removed'])
  54
+            changes.append(dict(
  55
+                who=r['author'],
  56
+                files=list(files),
  57
+                comments=r['message'],
  58
+                links=[r['url']],
  59
+                revision=r['revision'],
  60
+                when=r['timestamp'],
  61
+                # Let's hope Google add the branch one day:
  62
+                branch=r.get('branch', self._branch),
  63
+                revlink=r['url'],
  64
+                repository=self.repository,
  65
+                project=self.project
  66
+            ))
  67
+
  68
+        return changes
  69
+
  70
+def getChanges(request, options=None):
  71
+    try:
  72
+        headers = request.received_headers
  73
+        body = request.content.getvalue()
  74
+        #logging.error('headers = {0}, body = {1}'.format(headers, body))
  75
+
  76
+        # Instantiate a Payload object: this will parse the body, get the
  77
+        # authentication code from the headers and remember the branch picked up
  78
+        # by the user (Google Code doesn't send on which branch the changes were
  79
+        # made)
  80
+        payload = Payload(headers, body, options.get('branch', 'default'))
  81
+
  82
+        if 'secret_key' in options:
  83
+            if not payload.authenticate(options['secret_key']):
  84
+                raise GoogleCodeAuthFailed()
  85
+        else:
  86
+            log.msg('Missing secret_key in the Google Code WebHook options: cannot authenticate the request!')
  87
+
  88
+        log.msg('Received {0} changes from Google Code'.format(payload.revision_count))
  89
+        changes = payload.changes()
  90
+    except:
  91
+        logging.error("Can't parse the Google Code WebHook:")
  92
+        for msg in traceback.format_exception(*sys.exc_info()):
  93
+            logging.error(msg.strip())
  94
+        # return something valid even if everything goes wrong:
  95
+        changes = []
  96
+
  97
+    return changes, 'Google Code'
90  master/buildbot/test/unit/test_status_web_change_hooks_googlecode.py
... ...
@@ -0,0 +1,90 @@
  1
+# This file is part of Buildbot.  Buildbot is free software: you can
  2
+# redistribute it and/or modify it under the terms of the GNU General Public
  3
+# License as published by the Free Software Foundation, version 2.
  4
+#
  5
+# This program is distributed in the hope that it will be useful, but WITHOUT
  6
+# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
  7
+# FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more
  8
+# details.
  9
+#
  10
+# You should have received a copy of the GNU General Public License along with
  11
+# this program; if not, write to the Free Software Foundation, Inc., 51
  12
+# Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
  13
+#
  14
+# Copyright 2011 Louis Opter <kalessin@kalessin.fr>
  15
+#
  16
+# Written from the github change hook unit test
  17
+
  18
+import StringIO
  19
+
  20
+import buildbot.status.web.change_hook as change_hook
  21
+from buildbot.test.fake.web import MockRequest
  22
+
  23
+from twisted.trial import unittest
  24
+from twisted.internet import defer
  25
+
  26
+# Sample Google Code commit payload extracted from a Google Code test project
  27
+# {
  28
+#     "repository_path": "https://code.google.com/p/webhook-test/",
  29
+#     "project_name": "webhook-test",
  30
+#     "revision_count": 1,
  31
+#     "revisions": [
  32
+#         {
  33
+#             "added": [],
  34
+#             "parents": ["6574485e26a09a0e743e0745374056891d6a836a"],
  35
+#             "author": "Louis Opter \\u003Clouis@lse.epitech.net\\u003E",
  36
+#             "url": "http://webhook-test.googlecode.com/hg-history/68e5df283a8e751cdbf95516b20357b2c46f93d4/",
  37
+#             "timestamp": 1324082130,
  38
+#             "message": "Print a message",
  39
+#             "path_count": 1,
  40
+#             "removed": [],
  41
+#             "modified": ["/CMakeLists.txt"],
  42
+#             "revision": "68e5df283a8e751cdbf95516b20357b2c46f93d4"
  43
+#         }
  44
+#     ]
  45
+# }
  46
+googleCodeJsonBody = '{"repository_path":"https://code.google.com/p/webhook-test/","project_name":"webhook-test","revisions":[{"added":[],"parents":["6574485e26a09a0e743e0745374056891d6a836a"],"author":"Louis Opter \u003Clouis@lse.epitech.net\u003E","url":"http://webhook-test.googlecode.com/hg-history/68e5df283a8e751cdbf95516b20357b2c46f93d4/","timestamp":1324082130,"message":"Print a message","path_count":1,"removed":[],"modified":["/CMakeLists.txt"],"revision":"68e5df283a8e751cdbf95516b20357b2c46f93d4"}],"revision_count":1}'
  47
+
  48
+class TestChangeHookConfiguredWithGoogleCodeChange(unittest.TestCase):
  49
+    def setUp(self):
  50
+        self.request = MockRequest()
  51
+        # Google Code simply transmit the payload as an UTF-8 JSON body
  52
+        self.request.content = StringIO.StringIO(googleCodeJsonBody)
  53
+        self.request.received_headers = {
  54
+            'Google-Code-Project-Hosting-Hook-Hmac': '85910bf93ba5c266402d9328b0c7a856',
  55
+            'Content-Length': '509',
  56
+            'Accept-Encoding': 'gzip',
  57
+            'User-Agent': 'Google Code Project Hosting (+http://code.google.com/p/support/wiki/PostCommitWebHooks)',
  58
+            'Host': 'buildbot6-lopter.dotcloud.com:19457',
  59
+            'Content-Type': 'application/json; charset=UTF-8'
  60
+        }
  61
+
  62
+        self.changeHook = change_hook.ChangeHookResource(dialects={
  63
+                'googlecode': {
  64
+                    'secret_key': 'FSP3p-Ghdn4T0oqX',
  65
+                    'branch': 'test'
  66
+                }
  67
+        })
  68
+
  69
+    # Test 'base' hook with attributes. We should get a json string representing
  70
+    # a Change object as a dictionary. All values show be set.
  71
+    def testGoogleCodeWithHgChange(self):
  72
+        self.request.uri = "/change_hook/googlecode"
  73
+        d = defer.maybeDeferred(lambda : self.changeHook.render_GET(self.request))
  74
+        def check_changes(r):
  75
+            # Only one changeset has been submitted.
  76
+            self.assertEquals(len(self.request.addedChanges), 1)
  77
+
  78
+            # First changeset.
  79
+            change = self.request.addedChanges[0]
  80
+            self.assertEquals(change['files'], ['/CMakeLists.txt'])
  81
+            self.assertEquals(change["repository"], "https://code.google.com/p/webhook-test/")
  82
+            self.assertEquals(change["when"], 1324082130)
  83
+            self.assertEquals(change["who"], "Louis Opter <louis@lse.epitech.net>")
  84
+            self.assertEquals(change["revision"], '68e5df283a8e751cdbf95516b20357b2c46f93d4')
  85
+            self.assertEquals(change["comments"], "Print a message")
  86
+            self.assertEquals(change["branch"], "test")
  87
+            self.assertEquals(change["revlink"], "http://webhook-test.googlecode.com/hg-history/68e5df283a8e751cdbf95516b20357b2c46f93d4/")
  88
+
  89
+        d.addCallback(check_changes)
  90
+        return d
4  master/docs/manual/cfg-changesources.rst
Source Rendered
@@ -1125,8 +1125,8 @@ Change Hooks (HTTP Notifications)
1125 1125
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1126 1126
 
1127 1127
 Buildbot already provides a web frontend, and that frontend can easily be used
1128  
-to receive HTTP push notifications of commits from services like GitHub.  See
1129  
-:ref:`Change-Hooks` for more information.
  1128
+to receive HTTP push notifications of commits from services like GitHub or
  1129
+GoogleCode. See :ref:`Change-Hooks` for more information.
1130 1130
 
1131 1131
 .. bb:chsrc:: GoogleCodeAtomPoller
1132 1132
 
27  master/docs/manual/cfg-statustargets.rst
Source Rendered
@@ -682,6 +682,33 @@ Note that there is a standalone HTTP server available for receiving GitHub
682 682
 notifications, as well: :file:`contrib/github_buildbot.py`.  This script may be
683 683
 useful in cases where you cannot expose the WebStatus for public consumption.
684 684
 
  685
+Google Code hook
  686
+################
  687
+
  688
+The Google Code hook is quite similar to the GitHub Hook. It has one option
  689
+for the "Post-Commit Authentication Key" used to check if the request is
  690
+legitimate::
  691
+
  692
+    c['status'].append(html.WebStatus(
  693
+        …,
  694
+        change_hook_dialects={'googlecode': {'secret_key': 'FSP3p-Ghdn4T0oqX'}}
  695
+    ))
  696
+
  697
+This will add a "Post-Commit URL" for the project in the Google Code
  698
+administrative interface, pointing to ``/change_hook/googlecode`` relative to
  699
+the root of the web status.
  700
+
  701
+Alternatively, you can use the :ref:`GoogleCodeAtomPoller` :class:`ChangeSource`
  702
+that periodically poll the Google Code commit feed for changes.
  703
+
  704
+.. note::
  705
+
  706
+   Google Code doesn't send the branch on which the changes were made. So, the
  707
+   hook always returns ``'default'`` as the branch, you can override it with the
  708
+   ``'branch'`` option::
  709
+
  710
+      change_hook_dialects={'googlecode': {'secret_key': 'FSP3p-Ghdn4T0oqX', 'branch': 'master'}}
  711
+
685 712
 .. bb:status:: MailNotifier
686 713
 
687 714
 .. index:: single: email; MailNotifier
Commit_comment_tip

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.