Halve the GitHub refresh rate to fit the rate budget - #14
Merged
Conversation
The GitHub half makes roughly one call per repo plus one per workflow, with no conditional requests, so the cost of a pass scales linearly with the fleet and does not fall when nothing has changed. A pass over the 25-repo fleet now measures ~450-490 calls and takes 37s. At the old 300s that is twelve passes an hour, or ~5400-5900 calls against GitHub's ceiling of 5000. The budget runs out mid-hour, and a board that stops refreshing looks identical to a board where nothing is happening - the failure is silent and it lands exactly when someone is watching. 600s halves it to ~2700/hour, leaving headroom for the fleet to grow. The cost is up to ten minutes of staleness rather than five, which is the right trade for a board measured in open PRs and template drift. The compose defaults move too, so a fresh clone is not born over budget. Note they only apply where .env leaves the variable unset; an existing deployment pins it explicitly and needs the line changed by hand. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Raises
JQ_GITHUB_INTERVALfrom 300s to 600s, in.env.exampleand both compose defaults.Why
The GitHub half makes ~1 call per repo plus 1 per workflow, with no conditional requests, so cost scales linearly with fleet size and doesn't drop when nothing has changed.
Measured on the current 25-repo fleet: ~450–490 calls per pass, 37s wall clock.
Over budget means the board silently stops refreshing partway through each hour — and a stale board looks exactly like a board where nothing is happening. The failure mode is indistinguishable from the healthy one, which is the worst property a monitoring board can have.
The trade is up to 10 minutes of staleness instead of 5. For quantities measured in open PRs, CI status and template drift, that is not a meaningful loss.
Caveat for existing deployments
The compose defaults only apply where
.envleaves the variable unset. A server with an explicitJQ_GITHUB_INTERVAL=300keeps it and needs the line changed by hand:grep -n '^JQ_GITHUB_INTERVAL=' .envFollow-up
The real fix is conditional requests — GitHub does not charge 304s against the limit, so ETags would make an unchanged fleet nearly free and let the interval come back down. This change buys headroom; it does not remove the linear scaling.
Testing
Applied locally and confirmed the collector reports
JQ_GITHUB_INTERVAL=600. The 600 figure comes from one measured refresh, not a sustained hour —jq_github_rate_limit_remainingwill confirm or correct it.🤖 Generated with Claude Code