-
Notifications
You must be signed in to change notification settings - Fork 76
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[ENG-774] A Naive Solution for Waterbutler API Rate-limiting with Redis #380
Merged
felliott
merged 17 commits into
CenterForOpenScience:feature/wb-rate-limiting
from
cslzchen:feature/wb-rate-limiting-with-redis
Oct 1, 2019
Merged
[ENG-774] A Naive Solution for Waterbutler API Rate-limiting with Redis #380
felliott
merged 17 commits into
CenterForOpenScience:feature/wb-rate-limiting
from
cslzchen:feature/wb-rate-limiting-with-redis
Oct 1, 2019
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cslzchen
changed the title
[WIP] Waterbutler API rate-limiting with Redis [Ticket-TBD]
[WIP] Waterbutler API rate-limiting with Redis [ENG-451] [ ENG-468]
Aug 6, 2019
cslzchen
force-pushed
the
feature/wb-rate-limiting-with-redis
branch
3 times, most recently
from
August 12, 2019 20:45
b7b4325
to
3b555f0
Compare
cslzchen
changed the title
[WIP] Waterbutler API rate-limiting with Redis [ENG-451] [ ENG-468]
A Naive Solution for Waterbutler API Rate-limiting with Redis [ENG-774]
Aug 15, 2019
cslzchen
changed the title
A Naive Solution for Waterbutler API Rate-limiting with Redis [ENG-774]
[ENG-774] A Naive Solution for Waterbutler API Rate-limiting with Redis
Aug 16, 2019
3 tasks
cslzchen
force-pushed
the
feature/wb-rate-limiting-with-redis
branch
from
August 23, 2019 18:56
3b555f0
to
2219ae1
Compare
* Update the OSF auth handler to properly categorize metadata and revision requests as "metadata" and "revisions" actions in the request to the OSF. They had been categorized as "download" actions by virtue of being GET requests. HEAD requests were correctly categorized as "metadata" actions. The OSF relies on this action property to determine whether it should tally a download or view for a given file.
This is a trivial but proof-of-concept implementation with the fixed window algorithm. Users are identified by IP address only. WB allows 3600 requests per hour. Locally, the redis server runs with default image and in default mode via OSF docker-compose locally.
Rewrote the trivial rate limiter into a naive one that recognizes different types of auth and rate-limits each type separately. The 4 types are OSF cookie, OAuth bearer token, basic auth w/ base64- encoded username/password and w/o any of the 3 auth options.
OSF cookie, OAuth access token and base64-encoded username/password are used as redis keys during rate-limiting. We must obfuscate them for the same reason we store only password hashes in database. The `hashlib.hash256().hexdigest()` is used in this case. In addition, a prefix is added to the digest to identifty which type it is. The "No Auth" case is hashed as well (though unnecessarily) so that the keys all have the same look and length.
- Use Redis TTL command to obtain the time until reset. - Refactored `.rate_limit()` signature to return a tuple informing the handler whehter the limit is active and details of the limit. - Use tornado request handler's `set_header()` and `write()` to set the "Retry-After" header and the response body.
In Redis, the INCR command increments the number stored at key by 1. If the key does not exist, it is set to 0 before performing the operation. The python redis implementation makes this even simpler by setting the value to 1 without doing the increment.
* Bump redis dep to 3.3.8. No consequential changes. * Update exceptions to survive unpickling, add to unpickling test. * Fix capitalization of WaterButler in exception class name. * Don't throw errors in the error handling: provide a fallback for the `resource` attribute if rate-limiting kicks in before that has been initialized. * Update some docstrings to clarify return values and process. * Refactor test rate-limiting auth testing to only extract data as needed. * Add docs to settings; use `config.get_bool()` on booleans.
felliott
force-pushed
the
feature/wb-rate-limiting-with-redis
branch
from
October 1, 2019 17:00
9985713
to
e58a737
Compare
LGTM! Added some minor changes / docs / refactors, but nothing earth-shattering. Shall merge now! |
felliott
added a commit
that referenced
this pull request
Oct 1, 2019
…te-limiting [ENG-774] Closes: #380
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Ticket
https://openscience.atlassian.net/browse/ENG-774
Related Tickets
The parent EPIC ticket: https://openscience.atlassian.net/browse/ENG-451
The previous Analysis / POC ticket: https://openscience.atlassian.net/browse/ENG-468
The next ticket: https://openscience.atlassian.net/browse/ENG-775
Purpose
Add an naive solution to rate-limit Waterbutler API V1 using Redis.
Here is the OSF side support for local development: OSF-PR#9199.
DevOps tasks are out of scope for now.
Changes
Preparation
Rate-limiting algorithm: fixed Window.
Maintaining state: using a Redis server
docker-compose
as configured in OSF-PR#9199.What and How To Rate Limit
What's not included in this naive solution
X-WB-RL
headers for every request so benign API users can back-offSide effects
Please refer to Changes
Dev / QA Notes
assert needQa is False
Deployment Notes