Add rate limit toggles - #884
Merged
gabriel-samfira merged 2 commits intoSep 12, 2026
Merged
Conversation
This change adds 2 new toggles for tare limits in github credentials, and wires them up in the api, models, cli and webapp. These will be used to make decisions in the various worker loops that need to make API calls to GH forges. Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
With the 2 new knobs we added that control IF we want to reserve some of the rate limit quota for cleanup operations (critical from the perspective of managing IaaS cost) and the percentage of how much we should allocate for the critical ops, this change now takes advantage of them. This change enables pausing worker loops based on whether or not we are calling into a forge API that has rate limits enabled. There are 2 thresholds for rate limit decisions: * non critical loops * critical loops Non critical loops handle scale up or scale down events, react to new jobs requesting a runner, etc. Critical loops are loops that clean up runners that are marked for deletion, either because they finished a job or because the user requested the deletion. Essentially, we can reserve API calls for ops that save on cost. We can reserve a certain percentage of the available rate limit quota for critical ops (up to 50%). As long as the reserved threshold is not reached, all operations run normally. Once the threshold is reached, non critical ops are paused. Once rate limits reach 0 everything that requires an API call to the forge, stops until rate limits reset. Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This change enables pausing worker loops based on whether or not we are calling into a forge API that has rate limits enabled. There are 2 thresholds for rate limit decisions:
Non critical loops handle scale up or scale down events, react to new jobs requesting a runner, etc.
Critical loops are loops that clean up runners that are marked for deletion, either because they finished a job or because the user requested the deletion. Essentially, we can reserve API calls for ops that save on cost.
We can reserve a certain percentage of the available rate limit quota for critical ops (up to 50%).
As long as the reserved threshold is not reached, all operations run normally. Once the threshold is reached, non critical ops are paused. Once rate limits reach 0 everything that requires an API call to the forge, stops until rate limits reset.
Rate limit reservations are set on the credential level. To reserve some quota for critical ops and set it to a certain percentage you can use:
Rate limits are implemented by GitHub forges only.
Fixes: #616
Fixes: #480