New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change the interface to support by-key rate limiting #21

merged 35 commits into from Oct 26, 2018


None yet
1 participant

antifuchs commented Oct 21, 2018

The internal and external interface in ratelimit_meter has long been a thorn in my thigh: We have three implementations of state-keeping (two really, the Allower almost doesn't count), and using ratelimit_meter means having to import traits left and right.

Well, this PR changes all that! It:

  • Restructures all the rate limiting implementations so that they are stateless & only provide the bare algorithm and data structure access abstractions.
  • Introduces a new structure DirectRateLimiter that keeps state and can be parameterized with any of the three rate limiting strategies.
  • make the tests run on bare algorithms and on all the rate limiter structures now.
  • better constructors for keyed rate limiters (this should really get a builder, especially for customizing the evmap).
  • add a test for expiry from the keyed rate limiter
  • make NonConformance an associated type on the rate limiter algorithm & implement traits on each.

antifuchs added some commits Oct 21, 2018

Restructure everything so that algorithms stand alone
Now we have a single rate-limiter struct, responsible for keeping
track of state and of the parameters, and an algorithm that makes
decisions based on in-memory state.
Change the use/export structure for nicer docs
No longer hides the algorithms::* modules, and adds a state::* module
that contains the DirectRateLimiter.
Make Algorithm a public trait
Since buckets are separate from algorithms now, we can allow people to
implement their own buckets OR algorithms. Pretty cool!
Pull in evmap, make changes to the bucket state criteria
Now need to implement Eq & evmap's ShallowClone. These are fairly
trivial, so let's just put them in there.
Don't actually require that ThreadSafeWrapper be mutable
Since it has the Mutex (which regulates mutability), we can get a
mutable item at any time, which helps a lot with the evmap
implementation (which returns non-mut elements, of course).
Implement the first steps towards a keyed rate limiter
This one can make a decision and tracks state per-key.
Destutter the various check methods a little
This should dedup the code a little bit, and allow it all to look a
bit less grody.
Introduce RateLimitState, a trait for rate limiter state housekeeping
This will allow the keyed rate limiter to track which states are
useless & expire them out of the map.
Implement key expiry
This takes all the keys whose states are no longer relevant (i.e.,
have been the same as a "fresh" rate limiter for a given amount of
time), and removes them from the dictionary.
Make LeakyBucket the default rate limiting algorithm
Now the interface is somewhat nicer - users only need to decide
whether they need per-key or direct rate limiting.
Use nonzero_ext's nonzero!() macro to construct nonzero parameters
This is far more readable, faster, and safer too - since it's checked
at compile time, we can be sure our code will never panic at runtime
because I typo'd a constant.
Introduce per-algorithm error types, all implementing a trait
Since each algorithm has its own way of keeping time, this allows us
to return an error in the fastest possible way while remaining useful
to client code: Users can still get all the useful information out of
the negative decisions (how long to wait or the earliest instant they
can retry), and algorithms can concentrate on being really fast.
New benchmarks - algorithms edition
* Make benches/, where we test the "bare" algorithm,
  without a bucket attached.
* Add benchmark for single-threaded direct & keyed buckets, testing
  both algorithms on each.
* Move the no-op bench to using the bare algorithm too, and only bench
  single-element checking.
Add a multi-threaded benchmark for keyed rate limiters too
Works the same as the single-threaded one!
Extract benchmarks' variants/algorithms stuff into a utility crate
This allows us to use (at least the algorithms wrapper) in tests, too;
pretty useful!
Rewrite tests/ in terms of algorithms only
No more unnecessary deps on DirectRateLimiter, and some more
confidence that we're testing the right thing!

antifuchs added some commits Oct 25, 2018

Refactor the Algorithm trait: No more BucketParams
This improves the usability of algorithms quite a bit - fewer things
to implement/document and also allows us to use them in test more
Name the algorithm-construction method `construct`.
params_from_constructor always was a bit badly-named. Let's fix that
while we can.

@antifuchs antifuchs merged commit bead927 into master Oct 26, 2018

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed

@antifuchs antifuchs deleted the by-key branch Oct 26, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment