Skip to content

2021-June-05 - Prerelease - 2.6.239

Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 05 Jun 22:49
fff60cb
Parameterize map indexing rate (#9318)

Allows for us to control how quickly map indexing occurs. This is important
to avoid hitting rate limits. In this change we update the core indexing task
to have a back-off between each map indexing execution. This allows us to
stay under github API request rate limits.

Secondly, we parameterize how often to schedule map indexing runs, previously
it was 10 minutes.

Unauthorized API requests (those without a github API token) need to occur at
a relatively slow rate with a max of one API request per minute. Authorized
requests can get 1000 requests per hour before being rate limited.

We could in theory make request rate more dynamic by computing the remaining
rate limit and how many maps are needed. This can be problematic
if we are doing a constant rate and do not need that many requests. For example,
if we only need 50 requests and can do up to 1000 per hour, then firing
one every minute is excessively slow. Worse yet, if an individual indexing
task is slow then we start to have potential for overlapping between the indexing
runs. For these reasons the request rate is currently just pre-computed and
fixed at startup time.