bug: e2e testing run failed due to huggingface rate limit

[hiento09]

Given that

Huggingface API rate limit: https://huggingface.co/docs/api-inference/en/rate-limits
User Tier Rate Limit:

• Unregistered Users: 1 request per hour
• Signed-up Users: 50 requests per hour
• PRO and Enterprise Users: 500 requests per hour

Issue encountered

e2e testing runs successfully and passes test cases, but if multiple PRs are executed within the same hour, some PRs will fail e2e testing due to the rate limit, and we have to wait an hour before rerunning for a pass.

Objective

larify whether this issue affects users. If it does, we need to discuss with the team about using the Huggingface API. If not, find a way to resolve this rate limit issue in CI.

[hiento09]

cc @dan-homebrew @gabrielle-ong @0xSage @vansangpfiev

[gabrielle-ong]

@hiento09 is there a PR for this e2e tests issue?

[hiento09]

@hiento09 is there a PR for this e2e tests issue?

This PR @gabrielle-ong #1441

[gabrielle-ong]

To clarify @hiento09,

1. With this token, we are now allowed 50 requests per hour?
2. On the clarification if it affects users - what scenario is this, eg when cortex scales and if we have users pulling >50 models an hour - will it hit this rate limit?
3. Do we have visibility on how often this error occurs (to monitor if it happens more frequently in the future)

[hiento09]

To clarify @hiento09,

1. With this token, we are now allowed 50 requests per hour?
2. On the clarification if it affects users - what scenario is this, eg when cortex scales and if we have users pulling >50 models an hour - will it hit this rate limit?
3. Do we have visibility on how often this error occurs (to monitor if it happens more frequently in the future)

@gabrielle-ong I’d like to clarify something:

• This rate limit issue isn't related to downloading models; it’s about making unauthorized calls to the Hugging Face API. Specifically, when we make many requests to the Hugging Face API without including a token in the header, we hit Hugging Face’s rate limit.
• I searched on Google but couldn’t find Hugging Face mentioning the exact limit. The number I provided in the description applies to the API inference endpoint of Hugging Face; the API for getting models likely has a different limit.
• We discovered this rate limit while writing our end-to-end tests, but we should definitely note this for users. It’s also worth mentioning that users will need a token for authentication when downloading private models from Hugging Face.

[gabrielle-ong]

Thanks @hiento09 for explaining. Marking as complete, thanks for unblocking the e2e tests.
Will keep in mind if users get this error
Noted to add a note in docs that users need an authentication token when downloading private models from Hugging Face