"Rate limited, will retry in X" question

[nbs]

I wrote my own rudimentary rate limiter that hit the API until it got back a 429 and then waited the amount of time Retry-After sent. Well, apparently that's not acceptable and I kept getting blacklisted.

So, I reached for third party libraries that have solved this. Like this repository.

When running it though and outputting the logs, I keep seeing "Rate limited, will retry in X" and when I scan the source it's because it got back a 429.

I thought the point of these libraries was to calculate the time from the first request and prevent as much as possible any 429s from occurring.

I didn't look through the source too much just saw that if statement and compared it to the logs I was getting and it doesn't look good. I don't want to get blacklisted using this.

Thanks in advance.

[Neamar]

Heya,

You'll get blacklisted if you get too many 429.

What this library does is slowly reduce the concurrency as you get closer to the limit, so that no matter what happens, you only get ONE 429 per bucket.
This seems to be acceptable (I serve 10M+ requests a day with no issues, and get rate limited once pretty much every minute per region. YMMV but it's a good start).

What will get you blacklisted for sure however is to not respect the Rate Limit header and keep sending requests to the API.

tl;dr: one 429 is fine, more will be an issue.

[nbs]

Thank you for your response.

Sure enough I got a 403 back and I just feel I cannot trust the script to run on its own.

Perhaps I am not using the library right.

In a for loop I make my request()'s which the library automatically puts in a queue. And I'm getting multiple "Rate limited, will retry" messages per bucket.

I let the script run until it was completely rate limited and then I captured the log and have attached it here. Please let me know if this is normal.

In the source it says it's configured for dev keys (I have a dev key for now) until the first request tells it otherwise but a default concurrency of 20 (which the script seems to keep going back to) doesn't seem suitable for dev keys.

I hardcoded the concurrency to 1 in the source just to see and it still was giving me two 429s per bucket.

I'm trying to see if configuring throttling can get it to a point that prevents 429s for dev keys, but this just seems weird for default behavior if I'm using it correctly.

log.txt

[Neamar]

Yup, you're definitely having issues with the BR league.

The default rate limit for this region is super low (80 requests at the top of my head?) and the service is pretty slow, so sometimes requests arrive out of order and can mess the concurrency a little.

My advice for this situation would be to use the throttler.

Throttle the BR "league" endpoints, something like (please confirm, can't test from here):

riotLolApi.setThrottler("BR1", "league", 10);

This will reserve 10 "units" of safe-space, which should also ensure concurrency has dropped to 1 by the time you start getting rate limited.

Happy hacking!

[nbs]

I've used the method signature that allows me to set it for all regions and then did that for all the types of calls that I make. I hope that help solves the blacklisting problem.

It's reassuring to know that even a production script like yours can hit a 429 once a minute and be fine.

Thanks for the help.

[Neamar]

Cool, hope everything will be alright for you!

[cohenu]

Hey @nbs :-)
Did you managed to solve that issue? we have lots of 429 too. On our setup we have regional load balancer and in each region (2 in total - US & EU) we have node with this (wonderful, I have to say) library. each node server have 8 instance. we currently have around 10 million +/- requests per day (+ 2-7 hours of downtime from the blacklist) to Riot with throttler setup and we still got 429 (for example today we had 7,300 +/- 429s from eune, league endpoint)

Any help on that issue will be very appreciated, Thanks in advanced!

[Neamar]

@cohenu I've opened #26 to talk about your use case :)