Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle second/minute-rates limits on Flavors and Analyzers #164

Closed
amr-cossi opened this issue Jan 30, 2019 · 2 comments
Closed

Handle second/minute-rates limits on Flavors and Analyzers #164

amr-cossi opened this issue Jan 30, 2019 · 2 comments
Assignees
Milestone

Comments

@amr-cossi
Copy link

@amr-cossi amr-cossi commented Jan 30, 2019

Request Type

Feature Request

The goal of this feature would be to handle APIs limiting the number of requests per second or minute. Right now, if you launch a lot of queries simultaneously, CORTEX launches them all together (at the parallelization rate configured) but fails for all queries hitting the short rate limit.
It would be great to be able to configure a rate limit in # queries per # seconds for a Flavor (endpoint limit) and an Analyzer (service limit) and have CORTEX wait between queries to avoid the threshold.

@To-om To-om added this to the 3.0.1 milestone Jan 18, 2020
To-om added a commit that referenced this issue Jan 18, 2020
@To-om To-om removed their assignment Jan 19, 2020
nadouani added a commit that referenced this issue Jan 20, 2020
@nadouani nadouani closed this Jan 20, 2020
@amr-cossi

This comment has been minimized.

Copy link
Author

@amr-cossi amr-cossi commented Jan 20, 2020

Does this implementation handle a wait time between queries managed by CORTEX ?
In my understanding, with this modification, queries will fail because of the CORTEX limit instead of the distant endpoint limit.
The real feature would be to avoid failed queries altogether.

@nadouani

This comment has been minimized.

Copy link
Contributor

@nadouani nadouani commented Jan 20, 2020

Your understanding is right: Cortex will not enqueue the jobs based on that rate limit.

We have probably misunderstood the feature request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.