Permalink
Browse files

Pre-emptively abort when approaching rate-limit.

The Github API rate-limits requests to 5000 per hour.  If we hit this
limit part-way through migrating an issue, it could end up permanently
incomplete.  To avoid this, we'll monitor the number of remaining
requests and pre-emptively abort if we're about to migrate an issue
without enough requests held in reserve.
  • Loading branch information...
1 parent a4527d8 commit 59a5e377bd0a26771ff88c8b121333ed8fcbac5a @dnschnur dnschnur committed Sep 27, 2012
Showing with 12 additions and 0 deletions.
  1. +12 −0 migrateissues.py
View
@@ -23,6 +23,11 @@
GOOGLE_MAX_RESULTS = 25
+# The minimum number of remaining Github rate-limited API requests before we pre-emptively
+# abort to avoid hitting the limit part-way through migrating an issue.
+
+GITHUB_SPARE_REQUESTS = 50
+
# Mapping from Google Code issue labels to Github labels
LABEL_MAPPING = {
@@ -89,6 +94,13 @@ def add_issue_to_github(issue):
content = issue.content.text
date = parse_gcode_date(issue.published.text)
+ # Github rate-limits API requests to 5000 per hour, and if we hit that limit part-way
+ # through adding an issue it could end up in an incomplete state. To avoid this we'll
+ # ensure that there are enough requests remaining before we start migrating an issue.
+
+ if github.rate_limiting[0] < GITHUB_SPARE_REQUESTS:
+ raise Exception("Aborting to to impending Github API rate-limit cutoff.")
+
# Build a list of labels to apply to the new issue, including an 'imported' tag that
# we can use to identify this issue as one that's passed through migration.

0 comments on commit 59a5e37

Please sign in to comment.