Add a clocked rate limiter #4591
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
yycptt
reviewed
Jul 12, 2023
| // the original method uses time.Now(), and does not allow us to pass in a time.Time. Fortunately, it can be built on | ||
| // top of ReserveN. However, there are some optimizations that we can make. | ||
| func (l ClockedRateLimiter) WaitN(ctx context.Context, token int) error { | ||
| reservation := ClockedReservation{l.rateLimiter.ReserveN(l.clock.Now(), token), l.clock} |
There was a problem hiding this comment.
looks like this will do reservation & cancellation more often than the original impl (when token can't be get within the wait duration), so more likely run into golang/go#56924 this issue: keep doing reservation & cancellation result in effectively a lower rps limit (we saw that in prod before)
But I think the change is ok since the only way to avoid ^ issue is to only call Wait() with context that has no deadline, and it still holds for with this change. so I guess this comment is just FYI.
There was a problem hiding this comment.
Tbh I don't fully understand this issue, but it looks like the Go maintainers closed it because it's working as intended?
yycptt
approved these changes
Jul 18, 2023
MichaelSnowden
force-pushed
the
snowden/rate-limiter-clock
branch
2 times, most recently
from
2af5b93
to
380aaff
Compare
MichaelSnowden
force-pushed
the
snowden/rate-limiter-clock
branch
from
380aaff
to
7bd2fc3
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changed?
Our RateLimiterImpl now uses a ClockedRateLimiter instead of a raw go rate limiter which does not accept a clock.
Why?
To make unit testing / time skipping easier in the future, and to add some test coverage.
How did you test it?
I added a bunch of unit tests for the ClockedRateLimiter.
Potential risks
Is hotfix candidate?
No