Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid reaching Github API rate limit #2

Closed
rafaeljusto opened this issue Oct 9, 2015 · 2 comments
Closed

Avoid reaching Github API rate limit #2

rafaeljusto opened this issue Oct 9, 2015 · 2 comments

Comments

@rafaeljusto
Copy link
Owner

As we are today using token bucket strategy to limit the number of requests and the Github API uses requests per hour strategy, we can fall into rate limit when running the scanner. To solve this we could:

  • Change the token bucket parameters and sleep the go routines (until X-Ratelimit-Reset) in case that we reach the rate limit (status 403 Forbidden). The problem here is how to stop filling tokens in the bucket while in rate limit.
  • Rethink our rate limit strategy to query Github API, maybe something controlled by Github HTTP response headers (X-Ratelimit-Limit, X-Ratelimit-Remaining, X-Ratelimit-Reset).
@rafaeljusto
Copy link
Owner Author

rafaeljusto added a commit that referenced this issue Oct 28, 2015
@rafaeljusto
Copy link
Owner Author

I decided to simplify the rate limit strategy. If we got a 403 Forbidden response from Github API, just wait for the next requests window informed in the HTTP headers response (X-RateLimit-Reset). Token bucket was removed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant