Fix flaky tests caused by installation-level API rate limiting#31556
Merged
Conversation
…y test, upgrade JS live API retry config Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
…omment Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot
AI
changed the title
Fix rate-limit-sensitive tests
Fix flaky tests caused by installation-level API rate limiting
May 11, 2026
Copilot created this pull request from a session on behalf of
pelikhan
May 11, 2026 18:21
View session
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes CI/test flakiness caused by GitHub App installation-level API rate limiting during high-concurrency windows.
Changes:
- Updates the DIFC proxy
ghCLI smoke test in CI to call/rate_limitinstead of/repos/{owner}/{repo}to avoid quota-based 403s. - Switches the live GitHub API integration test retry behavior to use the shared
RATE_LIMIT_RETRY_CONFIGand increases the test timeout to accommodate longer backoff windows.
Show a summary per file
| File | Description |
|---|---|
| actions/setup/js/frontmatter_hash_github_api.test.cjs | Uses shared rate-limit retry config for the live API integration test and increases per-test timeout to match the longer retry window. |
| .github/workflows/ci.yml | Changes the DIFC proxy gh api check to use /rate_limit to reduce flakiness from installation token quota exhaustion. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 2/2 changed files
- Comments generated: 1
Comment on lines
+10
to
+11
| * Uses RATE_LIMIT_RETRY_CONFIG (5 retries, ~30 s initial delay) so that installation-level | ||
| * rate limit bursts during busy CI windows don't cause spurious test failures. |
This was referenced May 11, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug Fix
What was the bug?
Multiple CI jobs were failing with
HTTP 403: API rate limit exceeded for installationduring burst windows when concurrent runs share the same installation token quota.How did you fix it?
ci.yml— DIFC proxyghCLI testSwapped
gh api /repos/${{ github.repository }}forgh api /rate_limit. The/rate_limitendpoint is exempt from GitHub rate limits, so it can never 403 on quota exhaustion while still making a real authenticated call through the proxy.frontmatter_hash_github_api.test.cjs— live API integration testLIVE_API_RETRY_CONFIG(3 retries × 1 s initial delay ≈ 14 s total headroom) with the existingRATE_LIMIT_RETRY_CONFIG(5 retries, 15 s initial delay, 2× backoff, 4 min cap), which is designed for installation-level rate limit reset windows.timeouttoLIVE_API_TEST_TIMEOUT_MS(18 min) to match the longer worst-case retry sequence (~11.5 min) before the 20 min CI job deadline.