Domain raw.githubusercontent.com rate limits for unauthenticated requests (error: 429)

[adrianmusante]

Describe the bug

Hi! I’ve noticed that Bazarr is currently using several URLs pointing to https://raw.githubusercontent.com to fetch resources, for example:

• https://raw.githubusercontent.com/morpheus65535/bazarr-binaries/master/announcements.json
• https://raw.githubusercontent.com/Anime-Lists/anime-lists/master/anime-list.xml
• https://raw.githubusercontent.com/Fribb/anime-lists/master/anime-list-mini.json

Recently, GitHub updated their rate limits for unauthenticated requests to raw.githubusercontent.com (see announcement), and now I’m frequently hitting HTTP 429 errors (Too Many Requests) due to exceeding the anonymous request limit (which I believe is enforced per public IP).

In some other projects, I’ve worked around this by using jsDelivr as a CDN for GitHub files (e.g., https://cdn.jsdelivr.net/gh/morpheus65535/bazarr-binaries@master/announcements.json). However, I realize this might not be ideal for Bazarr, since these URLs often point to the latest branch and jsDelivr’s cache might not update immediately.

Maybe, possible solutions could be:

• Allowing users to configure a GitHub token for authenticated requests, which would raise the rate limit.
• Using an intermediate proxy or mirror that fetches and caches these files more frequently (e.g., every hour), to avoid hitting the GitHub anonymous rate limit (for example, hosted on https://www.bazarr.media/).

To Reproduce
Steps to reproduce the behavior:

1. Go to 'Settings' -> 'Providers'
2. Scroll down to 'Integrations'
3. Enable/Add AniDB integration

Expected behavior
Avoid rate-limit to GitHub.

Screenshots

Software (please complete the following information):

• Bazarr: v1.5.3
• Radarr version v5.28.0.10274
• Sonarr version v4.0.15.2941
• OS: Docker (host: Ubuntu Server)

Additional context
I suspect I’m hitting the rate limit more often because I have several tools that access GitHub in the same way.

[adrianmusante]

Just a quick note regarding the first solution Allowing users to configure a GitHub token for authenticated requests, which would raise the rate limit. The domain raw.githubusercontent.com does not support authentication headers. Just to clarify, the curl example comes from adapting the original URL:

https://raw.githubusercontent.com/morpheus65535/bazarr-binaries/master/announcements.json

Since raw.githubusercontent.com does not support authentication headers, you would need to use the GitHub API instead, like:

curl -H "Authorization: token <TOKEN>" -H "Accept: application/vnd.github.raw+json" "https://api.github.com/repos/morpheus65535/bazarr-binaries/contents/announcements.json?ref=master"

This makes the first solution much more complex than just changing the URL. Setting up a proxy or mirror to cache these files and updating the URLs to point there would be a much simpler and more robust approach.

[morpheus65535]

Should be fixed in upcoming beta using statically.io. Thanks for reporting this issue!