Handle a rate-limited API

[dtolnay]

Hitting a rate limit may either return an error for the request or return a JSON body that is different from the usual. Let's show how to deal with one or both of these cases.

[Michael-F-Bryan]

What kind of example did you have in mind here? The way rate limiting is handled will necessarily vary depending on which API you are communicating with. For example, GitHub always replies with a X-RateLimit-Remaining header and you'll get a 403 if you go over your limit.

So a hypothetical request to some GitHub API might go like this:

• Send request
• deserialize response
• if deserializing fails, check header for X-RateLimit-Remaining and return RateLimitError if equal to "0"
• else check status code and do other error handling to return an appropriate error

Or were you thinking of patterns people can use to make sure they don't go over their rate limit (e.g. the Leaky Bucket algorithm)?

[dtolnay]

@konstin you suggested this one initially. Any thoughts?

[konstin]

The correct solution would be a crate that is hooked into hyper and/or rewquest. This crate would delay all requests until you're allowed to requests again while logging to the user. Unfortunately, no stable crate for that currently exists.

There a two custom solutions: Github's api is friendly enough telling you the number of remaining requests, so you'd just check that header and sleep if the number gets to low.

As most apis aren't that helpfull though, the best solution is essentially trial and error, which I've quickly sketched out below. Consider the code as pseudocode, it's only meant to explain the control flow.

fn rate_limit_agnostic_request(url: Url) -> Result<String, Box<Error>> {
    loop {
        let mut reponse = reqwest::get(&url)?;
        if reponse.status().is_success() {  
            let mut string = String::new();
            reponse.read_to_string(&mut string)?;
            return Ok(string);
        } else if reponse.status() == TooManyRequests {
            info!("Rate limit hit, sleeping for 5 seconds");
            sleep(Seconds::new(5)); // TODO: Find the right functions for that
        } else { // Some other error
            return Err(from::From("Some other error"));
        }
    }
}

[konstin]

Side note: I wouldn't label this easy. Doing this right is surprisingly difficult, esp. when it comes to parallelism. This is made even more difficult by the fact that many services do not even adhere the http standard, which is to return a 429 and a Retry-After. Github returns a 403, twitter its famous 420 and I'm sure there a enough apis just returning a json with an error message.

[clarfonthey]

So fun fact, I was actually working on solving this problem myself and found the ratelimit crate, which makes it actually fairly stratightforward to solve this problem in a proactive (as opposed to reactive) way. I actually submitted brayniac/ratelimit#19 to make the API for this crate easier to work with to solve these types of use cases.

I think it'd be good to have two different rate-limiting examples: one that takes the proactive approach (rate limiting on the client side, only submitting requests at a certain rate) and one that takes the reactive approach (creating a handler that will retry rate-limited requests). They both have different use cases; code that sends requests based upon events would prefer a reactive method, whereas code that sends batches of requests would prefer a proactive method (potentially paired with a reactive method).

It might make sense to add ways of handling the reactive method to reqwest (for all of the sorts of non-standard variations listed), although I think that there should be some examples for the ratelimit crate (at least once the new API lands) to cover this.

[budziq]

Hi @clarcharr,

Thanks for looking into this!

We would certainly love to have both types of rate limiting examples as long as these are relatively bytesized (the draw fractal seams to be near our pain threshold). Originally we have hoped to present some trivial example without additional crates.

At the moment the cookbook tries to follow crates covered by Libz Blitz initiative. These crates are guaranteed to be of (or soon reach) high quality which is crucial for learning/reference resource such as cookbook.

So I would be hesitant to add another relatively new crate just yet. But feel free to submit ratelimit for inclusion on Libz Blitz thread (inclusion of relatively new crate wouldn't certainly be a precedent) 👍

[jbolila]

I can progress this if no one is currently working on it. And handle to specific cases.

• Standard Retry-After https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.37
• and custom Github https://developer.github.com/v3/rate_limit/
• or in alternative https://developer.twitter.com/en/docs/ads/general/guides/rate-limiting

[konstin]

I'd suggest implementing just one version with a comment that you can do all the other ways easily by adapting that code.

[jbolila]

In that case Retry-After (supported in Hyper hyperium/hyper#998) or the Custom headers of one the other two APIs?

Another question is about how this will be tested, since httpbin.org doesn't have a way to test this, and the other ones require an user account (not an issue during the implementation, but).

[budziq]

Personally I would go with github api. It's rate limited (the limit is much stricter if unauthorized)

[budziq]

At this moment we are looking for volunteer to finish up the PR by @jbolila