perf(queue): parallelize regate-sweep per-repo settings resolution

[JSONbored]

Summary

• fanOutAgentRegateSweepJobs (armed every 2 minutes for every registered+configured repo) resolved each repo's settings and drain-state one at a time in a sequential for loop, with both awaits inside the loop body — while the actual per-repo dispatch right below it already correctly uses Promise.all.
• Each repo costs resolveRepositorySettings's own 3 parallel round-trips plus a 4th getLatestRegatedAt D1 query, so this prefix scaled linearly with repo count (~4 x round-trip-latency x repoCount), entirely serialized, before dispatch even started. As a self-host maintainer onboards more repos, this risks colliding with the Workers CPU-time budget for the cron-triggered invocation.
• Replaced the sequential loop with a bounded-concurrency map (mapWithConcurrencyLimit, concurrency 4), reusing the exact worker-pool helper loadRepoFocusManifests already relies on for the same "many small per-repo D1/KV reads" shape — exported it from focus-manifest-loader.ts rather than duplicating it.
• Each worker returns a tagged outcome (ineligible / draining / configured / errored) instead of mutating shared counters directly, preserving the exact existing behavior: per-repo try/catch isolation, the skippedDraining/skippedErrored audit counters, and the silent skip for ineligible repos.

Found via a fresh performance/scalability/accuracy hardening audit of the self-host ORB stack. Tracked under #1667.

Scope

• src/signals/focus-manifest-loader.ts — export mapWithConcurrencyLimit
• src/queue/processors.ts — bounded-concurrency resolution + new SWEEP_FANOUT_RESOLUTION_CONCURRENCY constant
• test/unit/queue.test.ts — new regression test proving real overlap (maxInFlight > 1) AND boundedness (maxInFlight <= 4)

Validation

• npm run typecheck
• npm run test:coverage (full unsharded) — 546 files / 10825 tests passed; changed lines in processors.ts fully covered per lcov.info (cross-checked against the diff hunks — zero overlap with the 51 pre-existing uncovered lines elsewhere in the file)
• Existing fan-out isolation/dedup/in-flight-guard tests all still pass unmodified
• git diff --check clean

Safety

• Behavior-preserving: same eligibility/draining/error semantics per repo, same audit counters, same per-repo isolation. The per-repo dispatch stagger (delaySeconds) still applies — its exact repo-to-delay assignment can now vary run-to-run since worker completion order isn't guaranteed, which doesn't affect correctness (the stagger only needs some spread, not a specific repo-to-delay mapping).

Closes #3899

[superagent-security]

Superagent didn't find any vulnerabilities or security issues in this PR.

[loopover-orb]

Tip

🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩

✅ Gittensory review result - approve/merge recommended

_{Review updated: 2026-07-07 07:04:47 UTC}

3 files · 1 AI reviewer · no blockers · readiness 100/100 · CI green · clean

✅ Suggested Action - Approve/Merge

• safe to merge

Review summary
This PR replaces a sequential `for` loop resolving each repo's settings + drain-state in `fanOutAgentRegateSweepJobs` with a bounded-concurrency map (`mapWithConcurrencyLimit`, limit 4), reusing the existing worker-pool helper from `focus-manifest-loader.ts` instead of duplicating it. The refactor preserves per-repo try/catch isolation and the exact same three outcome buckets (ineligible/draining/configured/errored) via a tagged result type reduced afterward, and `mapWithConcurrencyLimit` preserves input order in its results array so the downstream `configured.map((repo, index) => ...)` dispatch is unaffected by completion order. The new regression test directly measures `maxInFlight` to prove both real overlap and the concurrency bound, which is a solid way to cover this change.

Nits — 4 non-blocking

• src/queue/processors.ts: the concurrency limit is hardcoded as a fresh literal `4` in `SWEEP_FANOUT_RESOLUTION_CONCURRENCY` with a comment saying it matches `REPO_FOCUS_MANIFEST_MAX_CONCURRENT_LOADS` — consider referencing that constant directly so the two can't silently drift apart.
• The flagged `console.error` at processors.ts:1324 is pre-existing error-path logging carried over from the original loop, not a new debug leftover — no action needed.
• Consider importing and reusing `REPO_FOCUS_MANIFEST_MAX_CONCURRENT_LOADS` (if exported) instead of a duplicate literal `4` for `SWEEP_FANOUT_RESOLUTION_CONCURRENCY`, per the nit above.
• PR author also opened the linked issue — Link an issue that was opened by a different contributor, or provide a rationale for why this self-authored issue represents genuine discovery work.

Signal	Result	Evidence
Code review	✅ No blockers	1 reviewer
Linked issue	✅ Linked	#3899
Related work	✅ No active overlap found	No same-issue or scoped active PR overlap found.
Change scope	✅ 20/20	Low review scope from cached public metadata (1 linked issue).
Validation posture	✅ 25/25	PR body includes validation/test evidence.
Contributor workload	✅ 10/10	Author activity: 51 registered-repo PR(s), 43 merged, 343 issue(s).
Contributor context	✅ Confirmed Gittensor contributor	JSONbored; Gittensor profile; 51 PR(s), 343 issue(s).
Gate result	✅ Passing	No configured blocker found.

Review context

• Author: JSONbored
• Role context: owner (maintainer lane)
• Public audience mode: oss maintainer
• Lane context: Repository is configured for direct PR review.
• Public profile languages: Python, TypeScript, JavaScript, Ruby, Go, Kotlin, MDX, Shell
• Official Gittensor activity: 51 PR(s), 343 issue(s).
• PR-specific overlap: none found.

Contributor next steps

• Treat this as maintainer-lane context rather than normal contributor-lane activity.

Signal definitions

• Related work = same linked issue, overlapping active PRs, or title/path similarity.
• Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
• Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
• Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
• Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

_{🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed}

---

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

• Re-run Gittensory review

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.58%. Comparing base (61109fe) to head (86426cb).
⚠️ Report is 4 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #3928   +/-   ##
=======================================
  Coverage   93.58%   93.58%           
=======================================
  Files         358      358           
  Lines       34342    34345    +3     
  Branches    12570    12573    +3     
=======================================
+ Hits        32138    32141    +3     
  Misses       1580     1580           
  Partials      624      624           

Files with missing lines	Coverage Δ
src/queue/processors.ts	94.54% <100.00%> (+<0.01%)	⬆️
src/signals/focus-manifest-loader.ts	99.09% <ø> (ø)

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[loopover-orb]

Gittensory approves — the gate is satisfied and CI is green.