Add a rate limited request filter #336
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose
While having the
ThrottledRequestFilter
providing a concurrency limit and implicit back pressure into the caller, another gauge would be helpful in fine tuning request speed. I've added the ability to specify the number of requests per second in addition to the number of concurrent requests allowed at any given time.Implementation Decisions
RateLimiter
to do most of the workmaxWait
argument found in theThrottledRequestFilter
, I decided to keep with that theme. Given that both the semaphore and the rate limiter perform atryAcquire()
, I've decided to split themaxWait
across both of those operations. So if you provide amaxWait
of1000
ms, the algorithm will reduce any ms spent in the semaphore acquisition and pass that reduced amount into the rate limiter'stryAcquire
.