fix(github): reduce rate-limit retry storms

[JSONbored]

Summary

• Cache stable branch-protection 403/404 responses in the existing safe GitHub response cache.
• Add process-local single-flight for concurrent mutable GitHub GETs without Redis persistence.
• Keep volatile single-flight scoped by exact authorization and exclude raw/content reads that callers stream-cap.
• Pre-yield self-host webhook jobs when installation REST headroom is already exhausted.
• Route remaining server-side GitHub REST helpers through the shared timeout/cache/single-flight wrapper where safe.
• Cache public-only focus-manifest repo-file and absent results in a separate public signal cache.

Why

• Required-status branch-protection denials were high-volume repeated REST calls and were not cached because only 200 responses were stored.
• Mutable review reads still need fresh state, but simultaneous identical reads during webhook bursts can safely share one live response.
• Webhook jobs should not have to fail once before the queue learns an installation bucket is exhausted.
• Public local-branch/MCP manifest probes should not repeatedly hit raw GitHub while private API-backed repo policy remains authoritative.

Validation

• npm exec -- vitest run test/unit/github-client.test.ts test/unit/preview-url.test.ts test/unit/selfhost-queue-common.test.ts test/unit/selfhost-sqlite-queue.test.ts test/unit/selfhost-pg-queue.test.ts test/unit/github-app.test.ts test/unit/mcp-output-schemas.test.ts test/unit/focus-manifest-loader.test.ts
• npm run test:ci
• npm audit --audit-level=moderate

Closes #1865

[loopover-orb]

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

_{Review updated: 2026-06-30 18:28:02 UTC}

38 files · 1 AI reviewer · no blockers · readiness 75/100 · CI green · unknown

⏸️ Suggested Action - Manual Review

• Touches a guarded path — held for manual review

Review summary
This PR centralizes more GitHub REST calls behind timeout/cache/single-flight handling, adds process-local rate-limit observations for self-host admission, persists an optional admission key, and separates public focus-manifest caching from private/API-backed manifest state. The schema and migration line up for the new nullable admission_key column, and the queue admission query preserves legacy unkeyed observations while avoiding keyed observations from other installations. The visible diff is safe enough to proceed; the main concerns are operational hardening around how broadly volatile responses are buffered and how clearly the webhook admission threshold is documented.

Nits — 7 non-blocking

• nit: src/github/client.ts:326 buffers every eligible volatile GET through response.clone().text(), so add a size/content-type guard or skip replay for unexpectedly large JSON responses even though raw/content/tree reads are excluded.
• nit: src/github/client.ts:139 keeps latestRestRateLimitObservations in a process-local Map with no age/reset pruning, which can grow for every installation key seen by a long-lived worker.
• nit: src/selfhost/queue-common.ts:154 uses LOW_REST_RATE_LIMIT_REMAINING as the webhook admission floor while the surrounding wording says exhausted, so rename the concept or document that webhooks pre-yield below the low headroom floor rather than only at zero remaining.
• nit: src/selfhost/sqlite-queue.ts:617 and src/selfhost/pg-queue.ts:620 intentionally apply legacy admission_key IS NULL rows globally, but that migration fallback deserves an inline comment because it can defer unrelated installations until keyed observations take over.
• src/github/client.ts:326: only call replayableResponse for volatile GETs whose content-length is absent or below a small cap, and resolveShared(null) otherwise so followers make their own live request.
• PR author also opened the linked issue — Link an issue that was opened by a different contributor, or provide a rationale for why this self-authored issue represents genuine discovery work.
• Touches a guarded path — held for manual review — A maintainer must review and merge this change.

Signal	Result	Evidence
Code review	✅ No blockers	1 reviewer
Linked issue	✅ Linked	#1865
Related work	⚠️ 2 scoped overlaps	Top overlaps are listed below; lower-confidence bulk is hidden.
Change scope	❌ 8/20	High review scope from cached public metadata (size label size:XL; 1 linked issue).
Validation posture	✅ 25/25	PR body includes validation/test evidence.
Contributor workload	✅ 10/10	Author activity: 68 registered-repo PR(s), 58 merged, 259 issue(s).
Contributor context	✅ Confirmed Gittensor contributor	JSONbored; Gittensor profile; 68 PR(s), 259 issue(s).
Gate result	⚠️ Not blocking	Advisory; not blocking this PR.

Review context

• Author: JSONbored
• Role context: owner (maintainer lane)
• Public audience mode: oss maintainer
• Lane context: Repository registration is not available in the local Gittensory cache.
• Public profile languages: not available
• Official Gittensor activity: 68 PR(s), 259 issue(s).
• Related work: Items reference the same linked issue Reduce GitHub rate-limit retry storms #1865. (issue #1865)
• Related work: Open PR work references issue Reduce GitHub rate-limit retry storms #1865. (issue #1865)

Contributor next steps

• Treat this as maintainer-lane context rather than normal contributor-lane activity.
• Review top overlaps.
• Add a concise scope and risk note.
• Triage stale or unlinked PRs.
• No action.
• Check active issues and PRs before submitting.

Signal definitions

• Related work = same linked issue, overlapping active PRs, or title/path similarity.
• Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
• Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
• Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
• Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

_{🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed}

---

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

• Re-run Gittensory review

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.54%. Comparing base (c9920a9) to head (b7932fa).
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1866      +/-   ##
==========================================
+ Coverage   95.25%   95.54%   +0.29%     
==========================================
  Files         217      217              
  Lines       23727    23856     +129     
  Branches     8569     8635      +66     
==========================================
+ Hits        22600    22793     +193     
+ Misses        539      437     -102     
- Partials      588      626      +38     

Files with missing lines	Coverage Δ
src/auth/github-oauth.ts	94.94% <100.00%> (ø)
src/db/repositories.ts	96.28% <ø> (ø)
src/db/schema.ts	69.46% <ø> (ø)
src/github/app.ts	97.89% <100.00%> (ø)
src/github/backfill.ts	94.15% <100.00%> (ø)
src/github/client.ts	100.00% <100.00%> (ø)
src/github/comments.ts	100.00% <100.00%> (ø)
src/github/labels.ts	95.83% <100.00%> (ø)
src/github/pr-actions.ts	98.68% <100.00%> (ø)
src/github/public.ts	96.72% <100.00%> (ø)
... and 15 more

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.