Release notes API rate limiting throws 403

[saschagrunert]

The release notes tool seems to get a new API response from GitHub if the rate limit kicks in:

level=fatal msg="gathering release notes: listing release notes: gathering notes: GET https://api.github.com/repos/cri-o/cri-o/commits/1aaf47484dcf1206663610c622a1b37a84e2e91b/pulls?page=1&per_page=100&state=closed: 403 You have exceeded a secondary rate limit. Please wait a few minutes before you try again. []"

Ref: https://github.com/cri-o/cri-o/runs/3422342329?check_suite_focus=true

It seems that we can mitigate by checking for the error and waiting a bit longer before trying it again.

[saschagrunert]

/help
/priority important-soon
/good-first-issue

[k8s-ci-robot]

@saschagrunert:
This request has been marked as suitable for new contributors.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-good-first-issue command.

In response to this:

/help
/priority important-soon
/good-first-issue

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[saschagrunert]

cc @kubernetes/release-managers

[sandipanpanda]

Hi there, I am a new contributor to Kubernetes and hopped in seeing good first issue label, can you please give a little insight on how to get started working on this issue?

[saschagrunert]

Sounds great @sandipanpanda, thank you for taking this opportunity. I think a good starting point would be to reproduce the issue with the release-notes tool: https://github.com/kubernetes/release/tree/master/cmd/release-notes

After that, we can try to catch the error in the retry logic of the release notes gatherer, which is using the go-github API. The retry logic can be found there:

release/pkg/github/internal/retry.go

Lines 63 to 91 in a85ab2b

	func GithubErrChecker(maxTries int, sleeper func(time.Duration)) func(error) bool {
	try := 0
	return func(err error) bool {
	if err == nil {
	return false
	}
	if try >= maxTries {
	logrus.Errorf("Max retries (%d) reached, not retrying anymore: %v", maxTries, err)
	return false
	}
	try++
	if aerr, ok := err.(*github.AbuseRateLimitError); ok {
	waitDuration := defaultGithubSleep
	if d := aerr.RetryAfter; d != nil {
	waitDuration = *d
	}
	logrus.
	WithField("err", aerr).
	Infof("Hit the abuse rate limit on try %d, sleeping for %s", try, waitDuration)
	sleeper(waitDuration)
	return true
	}
	return false
	}
	}

[rayandas]

Hi @sandipanpanda are you working on this issue? If not then I can take this up and give it a try.

[rayandas]

/assign

[justaugustus]

This just burned a stage and should have higher priority.
/remove-priority important-soon
/priority critical-urgent
/milestone v1.23
cc: @kubernetes/release-engineering
ref: https://kubernetes.slack.com/archives/CJH2GBF7Y/p1631645686102400?thread_ts=1631606375.087500&cid=CJH2GBF7Y