Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI tests frequently hit a GitHub API rate limit #1608

Closed
manics opened this issue Jan 3, 2023 · 4 comments · Fixed by #1612 or #1627
Closed

CI tests frequently hit a GitHub API rate limit #1608

manics opened this issue Jan 3, 2023 · 4 comments · Fixed by #1612 or #1627
Labels

Comments

@manics
Copy link
Member

manics commented Jan 3, 2023

Bug description

binderhub/tests/test_build.py often fails with the error

 >       assert launch_events > 0
E       assert 0 > 0

binderhub/tests/test_build.py:87: AssertionError
----------------------------- Captured stdout call -----------------------------
failed: Error resolving ref for gh:binderhub-ci-repos/cached-minimal-dockerfile/HEAD: GitHub rate limit exceeded. Try again in 15 minutes.
_ test_build[gh/binderhub-ci-repos/cached-minimal-dockerfile/596b52f10efb0c9befc0c4ae850cc5175297d71c] _

I think this is related to how often PRs are opned and updated, the required timeout can easily reach 60 minutes

Expected behaviour

GitHub API limits are only occasionally reached for PR workflows

Actual behaviour

Limits are often hit, causing CI tests to incorrectly fail.

How to reproduce

I think opening a PR that runs all tests, and updating if several times, is enough.

I think we should go through the tests and minimise the number of live GH API calls. Note we shouldn't use a GitHub API token since these are PRs.

@manics manics added the bug label Jan 3, 2023
@betatim
Copy link
Member

betatim commented Jan 4, 2023

Is there no way to get a GH token that is limited in scope to "read public repos" to increase the rate limit? I'm thinking a token that doesn't give the PR any new permissions except for raising the rate limit.

@consideRatio
Copy link
Member

consideRatio commented Jan 4, 2023

Maybe we can use secrets.github_token? I think that is around and provides read only access to stuff even when run in PRs. I'm not sure though, and I'm not sure it would work even if it is etc.

@consideRatio
Copy link
Member

Maybe the tests are less frequently failing now, but I've seen that they can still fail sadly.

@minrk
Copy link
Member

minrk commented Jan 10, 2023

We should be able to record and re-use mock responses to avoid so many API calls, right? I think we do some of that already, but perhaps we've added more API calls without adding the recorded responses.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
4 participants