Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse code

Adding support for GitHub repo hooks

  • Loading branch information...
commit 2ced2dd3e0088409c900cbd351dd4ffcf6b8dea2 1 parent b3cac89
David Beitey davidjb authored
62 README.rst
Source Rendered
@@ -22,17 +22,70 @@ Features
22 22
23 23 * Repositories: create and modify repositories within an organization
24 24
25   - * Configure all repository properties as per the `GitHub Repos API`_,
  25 + * Configure all repository properties as per the `GitHub Rapes API`_,
26 26 including privacy (public/private), description, and other metadata.
27   - * After inital repository creation happens, updated values in your
  27 + * After the initial repository creation happens, updated values in your
28 28 configuration will replace those on GitHub.
29 29
  30 +* Service hooks: add and modify service hooks for repositories.
  31 +
  32 + * GitHub repositories have support for sending information upon
  33 + certain events taking place (for instance, pushes being made to a
  34 + repository or a fork being taken).
  35 + * After the initial repo creation process takes place, updated values in your
  36 + hook configuration will `replace` those on GitHub.
  37 + * Hooks not present in your configuration (such as those manually added
  38 + on GitHub or those removed from local configuration) will *not* be
  39 + deleted.
  40 +
30 41 * Teams: automatically create teams and modify members
31 42
32 43 * Control permissions for teams (for example: push, pull or admin)
33 44
34 45 * Automatically syncs all of the above with GitHub when the tool is run.
35 46
  47 +Configuration
  48 +=============
  49 +
  50 +Service hooks
  51 +-------------
  52 +
  53 +Configure service hooks in your configuration as per the `GitHub Hooks API`_
  54 +like so::
  55 +
  56 + [hook:my-hook]
  57 + name = web
  58 + config =
  59 + {"url": "http://plone.org",
  60 + "insecure_ssl": "1"
  61 + }
  62 + events = push issues fork
  63 + active = true
  64 +
  65 + [repo:my.project]
  66 + ...
  67 + hooks = my-hook
  68 +
  69 +Values provided here will be coerced into suitable values for posting
  70 +to GitHub's API. For specifications, refer to `https://api.github.com/hooks`_.
  71 +
  72 + `name` (required)
  73 + String identifier for a service hook. Refer to specification for
  74 + available identifiers.
  75 +
  76 + `config` (required)
  77 + JSON consisting of key/value pairs relating to configuration
  78 + of this service. Refer to specifications for applicable config for each
  79 + service. *Note*: in order to prevent this script from attempting
  80 + to update GitHub every run, record Boolean values as string "1" or "0"
  81 + in this JSON - this is how values are stored by GitHub.
  82 +
  83 + `events` (optional)
  84 + List of events the hook should apply to. Different services can
  85 + respond to different events. Refer to API specification for information.
  86 +
  87 + `active` (optional)
  88 + Boolean value of whether the hook is enabled or not.
36 89
37 90 How to install
38 91 ==============
@@ -149,6 +202,10 @@ Changelog
149 202 0.1.4 - unreleased
150 203 ------------------
151 204
  205 + - Allow service hooks to be specified within the configuration.
  206 + For samples, see the example configuration. Any GitHub supported
  207 + hook can be associated with repos.
  208 + [davidjb]
152 209 - Allowing repo properties to be set on creation and editing of config.
153 210 For available options, see http://developer.github.com/v3/repos/#create.
154 211 This facilities private repo creation (if quota available), amongst other
@@ -194,6 +251,7 @@ Changelog
194 251
195 252 .. _`GitHub organizations`: https://github.com/blog/674-introducing-organizations
196 253 .. _`GitHub Repos API`: http://developer.github.com/v3/repos/#create
  254 +.. _`GitHub Hooks API`: http://developer.github.com/v3/repos/hooks/
197 255 .. _`Python2.6`: http://www.python.org/download/releases/2.6/
198 256 .. _`argparse`: http://pypi.python.org/pypi/argparse
199 257 .. _`requests`: http://python-requests.org
26 example.cfg
@@ -7,6 +7,27 @@ members =
7 7 repos =
8 8 snipmate-snippets
9 9
  10 +[hook:my-jenkins]
  11 +name = jenkins
  12 +#JSON object as per schema @ https://api.github.com/hooks
  13 +config =
  14 + {"jenkins_hook_url": "https://eresearch.jcu.edu.au/ci/github-webhook/"}
  15 +events =
  16 + push
  17 +active = true
  18 +
  19 +[hook:my-web]
  20 +name = web
  21 +#JSON object as per schema @ https://api.github.com/hooks
  22 +#Boolean objects should be represented as string "1" or "0" as this is
  23 +#what GitHub stores.
  24 +config =
  25 + {"url": "new-url",
  26 + "insecure_ssl": "1"
  27 + }
  28 +events = push
  29 +active = true
  30 +
10 31 [repo:vim-snipmate]
11 32 fork = garbas/vim-snipmate
12 33 owners = garbas MarcWeber
@@ -19,6 +40,9 @@ owners = honza
19 40 [repo:demo]
20 41 owners = davidjb
21 42 teams = contributors
  43 +hooks =
  44 + my-jenkins
  45 + my-web
22 46 description = My awesome repo
23 47 homepage = http://plone.org
24 48 has_issues = false
@@ -54,5 +78,5 @@ private = true
54 78 # - 3 repos:
55 79 # [{'name': 'vim-snipmate'},
56 80 # {'name': 'snipmate-snippets'}
57   -# {'name': 'demo'}
  81 +# {'name': 'demo', 'hooks': ['web', 'web']}
58 82 # ]
33 githubcollective/config.py
@@ -9,7 +9,9 @@
9 9 import ConfigParser
10 10 import StringIO
11 11 from githubcollective.team import Team
12   -from githubcollective.repo import Repo, REPO_BOOL_OPTIONS
  12 +from githubcollective.repo import Repo, REPO_BOOL_OPTIONS, \
  13 + REPO_RESERVED_OPTIONS
  14 +from githubcollective.hook import Hook, HOOK_BOOL_OPTIONS
13 15
14 16
15 17 BASE_URL = 'https://api.github.com'
@@ -55,6 +57,8 @@ def parse(self, data):
55 57 teams[team.name] = team
56 58
57 59 for repo in data['repos']:
  60 + if repo['hooks']:
  61 + repo['hooks'] = [Hook(**hook) for hook in repo['hooks']]
58 62 repo = Repo(**repo)
59 63 repos[repo.name] = repo
60 64
@@ -120,7 +124,7 @@ def parse(self, data):
120 124 # load configuration for repo
121 125 repo_config = dict(config.items(section))
122 126 # remove reserved properties
123   - for option in ('fork', 'owners', 'teams'):
  127 + for option in REPO_RESERVED_OPTIONS:
124 128 if option in repo_config:
125 129 del repo_config[option]
126 130 # coerce boolean values
@@ -128,7 +132,25 @@ def parse(self, data):
128 132 if option in repo_config:
129 133 repo_config[option] = config.getboolean(section,
130 134 option)
131   - repos[name] = Repo(name=name, **repo_config)
  135 + # load hooks for repo
  136 + hooks = []
  137 + if config.has_option(section, 'hooks'):
  138 + for hook in config.get(section, 'hooks').split():
  139 + hook_section = 'hook:%s' % hook
  140 + hook_config = dict(config.items(hook_section))
  141 + # coerce values into correct formats
  142 + hook_config['config'] = \
  143 + hook_config['config'].replace('\n', '')
  144 + if 'events' in hook_config:
  145 + hook_config['events'] = \
  146 + hook_config['events'].split()
  147 + for option in HOOK_BOOL_OPTIONS:
  148 + if option in hook_config:
  149 + hook_config[option] = config.getboolean(
  150 + hook_section, option)
  151 + hooks.append(Hook(**hook_config))
  152 +
  153 + repos[name] = Repo(name=name, hooks=hooks, **repo_config)
132 154 # add fork
133 155 if config.has_option(section, 'fork'):
134 156 fork_urls[name] = config.get(section, 'fork')
@@ -207,7 +229,10 @@ def _get_repos(self):
207 229 not self._github['repos']:
208 230 self._github['repos'] = {}
209 231 for item in self.github._gh_org_repos():
210   - repo = Repo(**item)
  232 + hooks = []
  233 + for hook in self.github._gh_org_repo_hooks(item['name']):
  234 + hooks.append(Hook(**hook))
  235 + repo = Repo(hooks=hooks, **item)
211 236 self._github['repos'][repo.name] = repo
212 237 return self._github['repos']
213 238 def _set_repos(self, value):
12 githubcollective/github.py
@@ -102,6 +102,9 @@ def _gh_team_repos(self, team_id):
102 102 def _gh_org_repos(self):
103 103 return self._get_request('/orgs/%s/repos' % self.org)
104 104
  105 + def _gh_org_repo_hooks(self, repo):
  106 + return self._get_request('/repos/%s/%s/hooks' % (self.org, repo))
  107 +
105 108 def _gh_org_fork_repo(self, fork_url):
106 109 return self._post_request('/repos/%s/forks' % fork_url,
107 110 {'org': self.org})
@@ -110,11 +113,20 @@ def _gh_org_create_repo(self, repo):
110 113 return self._post_request('/orgs/%s/repos' % self.org,
111 114 json.dumps(repo.dumps()))
112 115
  116 + def _gh_org_create_repo_hook(self, repo, hook):
  117 + return self._post_request('/repos/%s/%s/hooks' % (self.org, repo.name),
  118 + json.dumps(hook.dumps()))
  119 +
113 120 def _gh_org_edit_repo(self, repo, changes):
114 121 changes.update({'name': repo.name}) #Required by API
115 122 return self._patch_request('/repos/%s/%s' % (self.org, repo.name),
116 123 json.dumps(changes))
117 124
  125 + def _gh_org_edit_repo_hook(self, repo, hook_id, hook):
  126 + return self._patch_request('/repos/%s/%s/hooks/%i' % \
  127 + (self.org, repo.name, hook_id),
  128 + json.dumps(hook.dumps()))
  129 +
118 130 def _gh_org_create_team(self, name, permission='pull'):
119 131 assert permission in ['pull', 'push', 'admin']
120 132 return self._post_request('/orgs/%s/teams' % self.org, json.dumps({
23 githubcollective/hook.py
... ... @@ -0,0 +1,23 @@
  1 +import json
  2 +
  3 +HOOK_BOOL_OPTIONS = ('active',)
  4 +
  5 +class Hook(object):
  6 +
  7 + config = ''
  8 +
  9 + def __init__(self, **kw):
  10 + self.__dict__.update(kw)
  11 + #Handle case of unicode coming from GitHub
  12 + self.name = unicode(self.name)
  13 + if isinstance(self.config, str):
  14 + self.config = json.loads(self.config)
  15 +
  16 + def __repr__(self):
  17 + return '<Hook "%s">' % self.name
  18 +
  19 + def __str__(self):
  20 + return self.__repr__()
  21 +
  22 + def dumps(self):
  23 + return self.__dict__
17 githubcollective/repo.py
... ... @@ -1,3 +1,4 @@
  1 +REPO_RESERVED_OPTIONS = ('fork', 'owners', 'teams', 'hooks')
1 2 REPO_BOOL_OPTIONS = ('private', 'has_issues', 'has_wiki', 'has_downloads')
2 3
3 4 class Repo(object):
@@ -12,4 +13,18 @@ def __str__(self):
12 13 return self.__repr__()
13 14
14 15 def dumps(self):
15   - return self.__dict__
  16 + dump = self.__dict__.copy()
  17 + dump['hooks'] = [hook.dumps() for hook in dump['hooks']]
  18 + return dump
  19 +
  20 + def getGroupedHooks(self):
  21 + """GitHub repos can only have 1 of each hook, except `web`."""
  22 + hooks = {}
  23 + for hook in self.hooks:
  24 + if hook.name != 'web':
  25 + hooks[hook.name] = [hook]
  26 + else:
  27 + if 'web' not in hooks:
  28 + hooks['web'] = []
  29 + hooks['web'].append(hook)
  30 + return hooks
59 githubcollective/sync.py
@@ -5,6 +5,7 @@
5 5 import json
6 6
7 7 from githubcollective.team import Team
  8 +from githubcollective.repo import REPO_RESERVED_OPTIONS
8 9
9 10
10 11 class Sync(object):
@@ -22,15 +23,24 @@ def run(self, new, old):
22 23 to_add = new.repos - old.repos
23 24 for repo in to_add:
24 25 fork_url = new.get_fork_url(repo)
  26 + new_repo = new.get_repo(repo)
25 27 if fork_url is None:
26   - self.add_repo(old, new.get_repo(repo))
  28 + self.add_repo(old, new_repo)
27 29 if self.verbose:
28 30 print ' - %s' % repo
29 31 else:
30   - self.fork_repo(old, fork_url, new.get_repo(repo))
  32 + self.fork_repo(old, fork_url, new_repo)
31 33 if self.verbose:
32 34 print ' - %s - FORK OF %s' % (repo, fork_url)
33 35
  36 + for hook in new_repo.hooks:
  37 + import ipdb; ipdb.set_trace()
  38 + self.add_repo_hook(old, new_repo, hook)
  39 + if self.verbose:
  40 + print ' - %s - ADDED HOOK: %s (%r)' % (repo,
  41 + hook.name,
  42 + hook.config)
  43 +
34 44 # REMOVE REPOS
35 45 if self.verbose:
36 46 print 'REPOS TO BE REMOVED:'
@@ -53,7 +63,8 @@ def run(self, new, old):
53 63 changes = {}
54 64 #Go through differences and create dict of changes
55 65 #Settings removed from config are not modified
56   - for setting in vars(new_repo).keys():
  66 + for setting in set(vars(new_repo).keys()) \
  67 + - set(REPO_RESERVED_OPTIONS):
57 68 if not hasattr(old_repo, setting) or \
58 69 getattr(old_repo, setting) != getattr(new_repo, setting):
59 70 changes[setting] = getattr(new_repo, setting)
@@ -62,6 +73,39 @@ def run(self, new, old):
62 73 if self.verbose:
63 74 print ' - %s' % repo
64 75
  76 + #Go through hooks for changes
  77 + #Hooks removed from configuration are left alone as
  78 + #they may have been manually added upstream.
  79 + if new_repo.hooks:
  80 + old_hooks = old_repo.getGroupedHooks()
  81 + new_hooks = new_repo.getGroupedHooks()
  82 + if old_hooks != new_hooks:
  83 + #Conditional updating if change has happened in cfg
  84 + for hook_type, hooks in new_hooks.items():
  85 + for position_id, hook in enumerate(hooks):
  86 + if not hook_type in old_hooks or \
  87 + position_id < len(old_hooks[hook_type]):
  88 + #If attempting to update existing hook
  89 + old_hook = old_hooks[hook_type][position_id]
  90 + old_hook_config = vars(old_hook)
  91 + new_hook_config = old_hook_config.copy()
  92 + new_hook_config.update(vars(hook))
  93 + if old_hook_config != new_hook_config:
  94 + #If any difference at all, update server
  95 + self.edit_repo_hook(old,
  96 + old_repo,
  97 + old_hook.id,
  98 + hook)
  99 + if self.verbose:
  100 + print ' - %s - EDITED HOOK: %s (%r)' \
  101 + % (repo, hook.name, hook.config)
  102 + else:
  103 + #Adding a new hook
  104 + self.add_repo_hook(old, old_repo, hook)
  105 + if self.verbose:
  106 + print ' - %s - ADDED HOOK: %s (%r)' \
  107 + % (repo, hook.name, hook.config)
  108 +
65 109 # CREATE TEAMS
66 110 if self.verbose:
67 111 print 'CREATED TEAMS:'
@@ -150,6 +194,9 @@ def add_repo(self, config, repo):
150 194 config._repos[repo.name] = repo
151 195 return self.github._gh_org_create_repo(repo)
152 196
  197 + def add_repo_hook(self, config, repo, hook):
  198 + return self.github._gh_org_create_repo_hook(repo, hook)
  199 +
153 200 def remove_repo(self, config, repo):
154 201 pass
155 202 #del config._repos[repo.name]
@@ -158,6 +205,12 @@ def remove_repo(self, config, repo):
158 205 def edit_repo(self, config, repo, changes):
159 206 return self.github._gh_org_edit_repo(repo, changes)
160 207
  208 + def edit_repo_hook(self, config, repo, hook_id, hook):
  209 + hook_ids = [h.id for h in config._repos[repo.name].hooks]
  210 + hook_index = hook_ids.index(hook_id)
  211 + config._repos[repo.name].hooks[hook_index] = hook
  212 + return self.github._gh_org_edit_repo_hook(repo, hook_id, hook)
  213 +
161 214 def fork_repo(self, config, fork_url, repo):
162 215 config._repos[repo.name] = repo
163 216 return self.github._gh_org_fork_repo(fork_url)

0 comments on commit 2ced2dd

Please sign in to comment.
Something went wrong with that request. Please try again.