2021-June-05 - Prerelease - 2.6.239
Pre-release
Pre-release
Parameterize map indexing rate (#9318) Allows for us to control how quickly map indexing occurs. This is important to avoid hitting rate limits. In this change we update the core indexing task to have a back-off between each map indexing execution. This allows us to stay under github API request rate limits. Secondly, we parameterize how often to schedule map indexing runs, previously it was 10 minutes. Unauthorized API requests (those without a github API token) need to occur at a relatively slow rate with a max of one API request per minute. Authorized requests can get 1000 requests per hour before being rate limited. We could in theory make request rate more dynamic by computing the remaining rate limit and how many maps are needed. This can be problematic if we are doing a constant rate and do not need that many requests. For example, if we only need 50 requests and can do up to 1000 per hour, then firing one every minute is excessively slow. Worse yet, if an individual indexing task is slow then we start to have potential for overlapping between the indexing runs. For these reasons the request rate is currently just pre-computed and fixed at startup time.