tower-gcra is a tower middleware that provides rate-limiting; that is, it limits that rate at which requests are processed by the Service which it wraps. It does so via the Generic Cell Rate Algorithm (AKA GCRA). GCRA is a particularly effective variant of the Leaky Bucket approach to rate-limiting.
In particular, tower-gcra provides for both “direct” and “keyed” rate-limiting. The former defines a single quota governing the rate at which all requests will be processed. The latter defines “per-key” state, where a key is some aspect of the requests being serviced (the peer IP address, for instance), so that the quota applies per instance of that key (when the key is peer IP address, this would mean that each peer IP is rate-limited individually). Furthermore, tower-gcra allows per-key quotas as well (so, for instance, we could build a rate-limiting client that applies different rate limits depending on the destination).
tower-gcra depends on the governor crate for the core GCRA implementation. At the time of this writing, the per-key quota functionality depends on an as-yet-unmerged governor PR.
tower ships with a rate-limiting middleware: RateLimit. RateLimit “enforces a rate limit on the number of requests the underlying service can handle over a period of time.” And indeed, it counts the number of requests per given unit of time & will pend any request over & above the given permissible number. In this way it differs from Leaky Bucket in that a client can make requests at an arbitrarily high rate until it exhausts its quota for any given time period.
When rate-limited, calls to poll_ready() will pend (not call()), which seems preferrable. This crate doesn’t do that because without a request, we can’t extract a key for rate-limiting.
RateLimit is not Clone, which can be inconvenient. For instance, RetryLayer requries that the Service it wraps be Clone. The suggested workaround is to wrap it in a BufferLayer. Regrettably, that erases the inner error type to Box<dyn Error + Send + Sync>.
tower-governor is another tower middleware building on the governor crate, but is explicitly targeted at HTTP servers. For instance, it only implements Service for http Requests, and rather than pending rate-limited requests, it returns an HTTP 429 Too Many Requests response, taking care to insert “X-RateLimit-After” and “Retry-After” headers.
It is, however, Clone. It, like this crate, handles rate-limiting in call() (rather than poll_ready()), presumably also because it has access to the incoming request there. Lastly, at the time of this writing, it doesn’t support per-key quotas.
tower-gcra is released under the GPL v3.0.
tower-gcra uses the Rust 2024 edition, so Rust 1.85 is required.
cargo add tower-gcraThis is a new crate. I’ve chosen the version number (0.x) in an attempt to convey that. In particular, I can’t publish this on crates.io until my PR is merged into governor (because crates.io does not allow packages to be published with dependencies on code published outside of crates.io, other than dev dependencies). So if you want to use it, you’ll have to either use a git dependency here or clone this repo & use a path dependency, for now.
Bugs, comments, suggestions and contributions are welcome, on X, Mastodon, in the issues or at sp1ff@pobox.com.