github: Detect secondary rate limit #11
Conversation
This commit adds logic to the git call retrier to detect the GitHub secondary rate limit. When detecting this limit, the github client will wait for the set time (1 min) plus a few seconds more. The idea behind this is that when doing calls in parallel we don't want to set loose all workers to the API at the same time. Signed-off-by: Adolfo García Veytia (Puerco) <adolfo.garcia@uservers.net>
|
@puerco: GitHub didn't allow me to request PR reviews from the following users: kubernetes/release-engineering. Note that only kubernetes-sigs members and repo collaborators can review this PR, and authors cannot review their own PRs. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
k8s-ci-robot
added
kind/failing-test
k8s-ci-robot
added
the
lgtm
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: cpanato, puerco The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
| if strings.Contains(err.Error(), "secondary rate limit. Please wait") { | ||
| rtime, err := rand.Int(rand.Reader, big.NewInt(30)) | ||
| if err != nil { | ||
| logrus.Error(err) | ||
| return false | ||
| } | ||
| waitDuration := time.Duration(rtime.Int64()*int64(time.Second)) + defaultGithubSleep | ||
| logrus. | ||
| WithField("err", err). | ||
| Infof("Hit the GitHub secondary rate limit on try %d, sleeping for %s", try, waitDuration) | ||
| sleeper(waitDuration) | ||
| return true | ||
| } |
There was a problem hiding this comment.
How about our idea of locking the gatherer threads if a rate limit occurs? Would that probably prevent the secondary rate limit?
Ref https://docs.github.com/en/rest/overview/resources-in-the-rest-api#secondary-rate-limits
Log the error; and try using a newer release-notes which might be able to handle github secondary rate limit: kubernetes-sigs/release-sdk#11 Note that new release-notes needs --markdown-links option Signed-off-by: Ed Santiago <santiago@redhat.com>
This commit adds logic to the github call retrier to detect the GitHub
secondary rate limit. When detecting the limit, the github client
will wait for the set time (1 min) plus a few seconds more.
The idea behind this is that when doing calls in parallel we don't
want to set loose all workers to the API at the same time.
Permanent fix for kubernetes/release#2324
Fixes kubernetes/release#2225
Fixes kubernetes/release#2302
/kind bug failing-test flake
/priority important-soon
/assign @justaugustus @Verolop @palnabarun @saschagrunert
/cc @kubernetes/release-engineering
/milestone v1.23
Signed-off-by: Adolfo García Veytia (Puerco) adolfo.garcia@uservers.net