Try to use GitHub PAT when fetching forecast dates; and check response status for rate limit #649
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Followup to #647. Because dates are now being fetched using the GitHub API, scripts that request a lot of dates (e.g. forecast eval pipeline) can run into rate limits. When the API is rate-limited,
get_covidhub_forecast_dates
raises a non-informative warning and returns empty dates, causing downstream failures that are hard to debug.Look for a
GITHUB_PAT
environment variable first, thenGITHUB_TOKEN
if*_PAT
doesn't exist. If either is found, use it to authenticate the GET requests (using this header format). Response status codes are checked and if a 403 status is returned, the function exits with an error. It doesn't make sense to warn and return anyway because the results are empty; or to retry with backoff because the rate limit resets after an hour.Note: I used the API's message as the error message because I'm not sure a 403-status is unique to reaching the rate limit and I want the actual error to be clear, and checking the text of the message wouldn't be robust.