Skip to content

perf(imports): break the github/client <-> db/repositories cycle taxing every test import#8548

Merged
JSONbored merged 2 commits into
mainfrom
perf/import-graph-cycle-cut
Jul 24, 2026
Merged

perf(imports): break the github/client <-> db/repositories cycle taxing every test import#8548
JSONbored merged 2 commits into
mainfrom
perf/import-graph-cycle-cut

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

Follow-up to the TestD1Database template-clone fix: with schema replay gone, the suite's next-largest fixed cost is module import time (~505s aggregate per full run). Profiling found it concentrates in one place: the github/client ↔ db/repositories dependency cycle, which cost ~1.1s of cold import under vitest and taxed nearly every test file that touches the app — and it existed for trivially small reasons:

  • db/repositories imported the whole @loopover/engine barrel (~420ms of the cycle) for exactly one tiny parser. → Engine gains a ./parse-pull-request-target-key subpath export (additive; the barrel still exports it), and repositories imports the subpath.
  • gittensor/api (imported by db/repositories) imported github/client for ONE constant, PRODUCT_USER_AGENT. → The constant moves to a new leaf module src/github/user-agent.ts; client imports and re-exports it, so its ~8 other importers are unchanged.
  • github/client statically imported db/repositories for two call sites (resolveRepoActionMode's freeze read; the suppressed-write audit hook). → Both become lazy await import(...) at the call site — the exact idiom processors.ts already uses — module-cached after first call.

Measured cold-import cost under vitest (same machine):

module before after
github/client 1,155ms 37ms
gittensor/api 1,119ms 11ms
db/repositories 1,106ms 771ms
queue/processors 2,677ms 1,978ms

db/repositories' remaining ~770ms is its own 8.3k lines + drizzle schema — reducible only by splitting the file itself (a #4013-style sequence of its own, deliberately out of scope here).

Behavior is unchanged: no cycle member had import-time side effects the other depended on, both lazy call sites are in already-async functions, and killing a real import cycle is a layering improvement in its own right.

Suite-level effect, measured with a back-to-back A/B on the identical tree (stash → baseline → pop → treatment, same machine conditions): aggregate import time 795s → 715s (−80s, −10%); wall time within noise on a loaded 10-core machine but the import saving concentrates in exactly the files CI's single validate-tests runner grinds through. The honest framing: this is a compounding follow-up to the template-clone fix (the big win), not a second 3× — plus it kills a real dependency cycle, which is worth having at any speed.

Validation

  • npm run typecheck clean; npm run test:coverage (full, unsharded): 21,473 passed / 0 failed on the treatment run (the baseline run of the same A/B had 2 unrelated pre-existing flakes).
  • Both lazy-import lines execute under existing tests (resolveRepoActionMode across the actuation suites; the audit hook via github-app's dry-run suppressed-write tests).

…ng every test import

The cycle cost ~1.1s of cold import under vitest and nearly every test file
paid it, for trivially small reasons: repositories pulled the whole
@loopover/engine barrel (~420ms) for one parser (now an additive
./parse-pull-request-target-key subpath export); gittensor/api pulled all of
github/client for the PRODUCT_USER_AGENT constant (now a leaf module,
re-exported from client so its other importers are unchanged); and client
statically pulled repositories for two call sites, both now lazy await
import() at the call site -- the idiom processors.ts already uses,
module-cached after the first call.

Cold-import cost under vitest: github/client 1155ms -> 37ms, gittensor/api
1119ms -> 11ms, db/repositories 1106ms -> 771ms, queue/processors 2677ms ->
1978ms. Back-to-back A/B on the identical tree: aggregate suite import time
795s -> 715s. Compounds the TestD1Database template-clone fix; also simply
removes a real dependency cycle.
@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
loopover-ui c322a24 Commit Preview URL

Branch Preview URL
Jul 24 2026, 07:29 PM

@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@JSONbored JSONbored self-assigned this Jul 24, 2026
@codecov

codecov Bot commented Jul 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.67%. Comparing base (61f6ec6) to head (28c7b39).
⚠️ Report is 2 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8548      +/-   ##
==========================================
+ Coverage   89.58%   93.67%   +4.08%     
==========================================
  Files          97      697     +600     
  Lines       22706    56845   +34139     
  Branches     3872    20164   +16292     
==========================================
+ Hits        20341    53248   +32907     
- Misses       2187     2613     +426     
- Partials      178      984     +806     
Flag Coverage Δ
backend 93.67% <100.00%> (?)
control-plane ?
rees ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/db/repositories.ts 96.73% <ø> (ø)
src/github/client.ts 100.00% <100.00%> (ø)
src/github/user-agent.ts 100.00% <100.00%> (ø)
src/gittensor/api.ts 98.18% <ø> (ø)
src/review/rag-index.ts 93.86% <100.00%> (ø)
src/review/rag.ts 99.61% <100.00%> (ø)

... and 788 files with indirect coverage changes

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 24, 2026
CI's validate-tests failed 3 rag-index.test.ts tests at the 15s per-test
timeout: the MAX_CHUNKS_PER_REPO cap tests build up to 4000+25 real chunk
rows to prove capping BEHAVIOR, which is fine locally but borderline-to-over
CI's timeout on a shared runner. rag.ts gains maxChunksPerRepo()/
setMaxChunksPerRepoForTest() -- the same ...ForTest override convention as
clearInstallationTokenCacheForTest -- and rag-index.ts's three cap-read sites
switch from the raw constant to the getter; production is unaffected. The
test file runs under a 24-row cap.
@loopover-orb

loopover-orb Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Important

🟪🟪🟪🟪🟪🟪🟪🟪🟪🟪🟪🟪

🔍 LoopOver is reviewing…

AI analysis is in progress. This comment will update when the review is complete.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed · 🟪 Reviewing

@JSONbored
JSONbored merged commit ec6f6e6 into main Jul 24, 2026
4 checks passed
@JSONbored
JSONbored deleted the perf/import-graph-cycle-cut branch July 24, 2026 20:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant