Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Weblate throttling #6128

Closed
ZloyChert opened this issue Jun 1, 2021 · 11 comments
Closed

Weblate throttling #6128

ZloyChert opened this issue Jun 1, 2021 · 11 comments
Labels
question This is more a question for the support than an issue.

Comments

@ZloyChert
Copy link

ZloyChert commented Jun 1, 2021

Describe the issue
Weblate API throttling works unpredictable. No throttling options are rewritten, I have default 5000 per hour for authorized user. After random number of requests (2-5), Weblate returns Too Many Requests.
In headers of the response I got:

  • X-RateLimit-Limit 5000
  • X-RateLimit-Remaining 4990 (or very close number)
  • X-RateLimit-Reset something (e.g. 370, 2, 30)

I already tried
Requests on different endpoints:

  • /api/units/
  • /api/translations/xxx/xxx/xxx/file
  • /api/translations/xxx/xxx/xxx/units
  • /exports/stats/xxx/xxx

One of them - /exports/stats/xxx/xxx - works fine, other return 429

I read all parts about rate limiting, only one of them is about api rate limiting. And there was nothing useful about why I'm getting 429 after 5-10 requests within 5-10 minutes.

@ZloyChert ZloyChert added the question This is more a question for the support than an issue. label Jun 1, 2021
@github-actions
Copy link

github-actions bot commented Jun 1, 2021

This issue looks more like a support question than an issue. We strive to answer these reasonably fast, but purchasing the support subscription is not only more responsible and faster for your business but also makes Weblate stronger. In case your question is already answered, making a donation is the right way to say thank you!

@nijel
Copy link
Member

nijel commented Jun 2, 2021

The /exports/ endpoint are not covered by the API endpoints (and do not accept API like authentication).

For the /api/ are you sure all your requests are authenticated? In case they are not, you are having way lower limits and these requests would fail earlier...

@ZloyChert
Copy link
Author

Authenticated requests are requests with Authorization header, right? I have authorization header.
I've tried several formats - "Authorization: Token XXXXXXXXX" and "Authorization: XXXXXXXXX". I still don't know what is the right way. In examples "Token" word exists, but in description of authorization nothing about it.
I've thought about unauthorized requests, because 429 have appeared too fast. And the result message of 429 request is "Request was throttled. Expected available in 1895 seconds", Yesterday the number of seconds was 60k+ for sure, it is close to 24 hours - like in unauthorized requests.
How can I make sure I am authorized? Maybe something in responses can help it.

@ZloyChert
Copy link
Author

ZloyChert commented Jun 2, 2021

I have tried again different combinations of header. Now it works.

The problem was in header. I've tried just token and token with word Token, but I've forgot to add space between token and word "Token"

Thanks

@github-actions
Copy link

github-actions bot commented Jun 2, 2021

The issue you have reported is now resolved. If you don’t feel it’s right, please follow its labels to get a clue for further steps.

  • In case you see a similar problem, please open a separate issue.
  • If you are happy with the outcome, don’t hesitate to support Weblate by making a donation.

@nijel
Copy link
Member

nijel commented Jun 2, 2021

@nijel
Copy link
Member

nijel commented Jun 2, 2021

...or use wlc which does that properly for you.

nijel added a commit that referenced this issue Jun 2, 2021
Apparently some users don't read the examples, so put it here as well.

Issue #6128
@ZloyChert
Copy link
Author

ZloyChert commented Jun 2, 2021

By the way, I'm getting 429 again, with proper authentication. And "Retry-After" 5000sec.
Everything was fine for a 3 hours, now it is throttled again.

And I really think, that throttling is not kind of feature, that should be turned on by default.

@ZloyChert ZloyChert reopened this Jun 2, 2021
@nijel
Copy link
Member

nijel commented Jun 2, 2021

I think you are still doing some unauthenticated requests (the retry after is longer that the rate limit window for authenticated users). If you really need more than 5000 requests per hour, you can adjust your configuration. I think this is reasonable default, for example it matches what GitHub does.

@ZloyChert
Copy link
Author

Yes, I agree. And I've investigated this.
I think, I've found the issue. It's trailing slash in url. Urls without slash are redirected (via 301), headers are not copied. Seems so.

@github-actions
Copy link

github-actions bot commented Jun 2, 2021

The issue you have reported is now resolved. If you don’t feel it’s right, please follow its labels to get a clue for further steps.

  • In case you see a similar problem, please open a separate issue.
  • If you are happy with the outcome, don’t hesitate to support Weblate by making a donation.

nijel added a commit that referenced this issue Jun 2, 2021
This can easily break things, so force people to use correct URLs.

Issue #6128
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question This is more a question for the support than an issue.
Projects
None yet
Development

No branches or pull requests

2 participants