Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rate limit response is sometimes different #49

Closed
dbeckwith opened this issue May 26, 2023 · 3 comments
Closed

Rate limit response is sometimes different #49

dbeckwith opened this issue May 26, 2023 · 3 comments

Comments

@dbeckwith
Copy link
Contributor

I was trying to write a simple loop to fetch every page of systems using the List Systems endpoint, which takes a while. I get normal rate limit 429 responses for a while, and have my code wait for the time provided in the Retry-After header. But after a while (almost 150 pages), I get a different kind of rate limit response. It doesn't have a Retry-After header, and the response body isn't JSON, but what looks like HTML:

<!doctype html><meta charset="utf-8"><meta name=viewport content="width=device-width, initial-scale=1"><title>429</title>429 Too Many Requests

It almost looks like something else is rate limiting my requests, like a proxy or something.

Also, a bit off-topic, but is there a better way to fetch the full list of systems? Seems like the documented page size limit is only 20, but according to the docs there are 5,000 systems, which would be 250 pages, which takes quite a while with the rate-limiting. Maybe I shouldn't be fetching every single system? But then how can I do things like discover nearby systems?

@dbeckwith
Copy link
Contributor Author

#39 would actually be a great solution for my last question about discovering nearby systems

@tyrope
Copy link

tyrope commented May 28, 2023

Unrelated to the issue, but for your purpose, you're looking for /v2/systems.json (though this is rumoured to go away at some point!)

@space-admiral
Copy link
Contributor

Usually that will happen if you are consistently going past the API rate limit, we have an added layer for DDoS protection, and we can't control the headers. If you stay within the defined limit of 2 requests a second, the headers should be visible.

Let me know though if it still doesn't work. Thanks for reporting this! We also added a section in the docs for this rate limiter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants