Skip to content

perf(test): eliminate SQLite fsync and real rate-limit backoff wait in tests#8553

Merged
JSONbored merged 1 commit into
mainfrom
perf/test-sqlite-pragma-and-retry-delay
Jul 24, 2026
Merged

perf(test): eliminate SQLite fsync and real rate-limit backoff wait in tests#8553
JSONbored merged 1 commit into
mainfrom
perf/test-sqlite-pragma-and-retry-delay

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

Third follow-up in the CI-speed sequence. Profiling queue-lifecycle-guards.test.ts and backfill.test.ts (still 33s and 19s respectively after the earlier fixes) found two more real per-test costs, both confirmed via a CPU profile that showed the wall time going to idle (waiting on real I/O), not application compute:

  • SQLite fsyncs on every write by default. TestD1Database's clones are real temp files (needed for the template-clone trick), and SQLite's defaults (journal_mode=DELETE, synchronous=FULL) fsync on every autocommit statement — every .prepare(...).run() in app code is its own implicit transaction. Benchmarked: 50 inserts, 13.9ms default vs 0.6ms tuned (23×). journal_mode=MEMORY + synchronous=OFF are safe for a throwaway clone that's unlinked as soon as it's open — crash-consistency durability is meaningless for data nothing outlives the test to read back.
  • The production GitHub rate-limit backoff runs at full real speed inside tests. fetchWithGitHubRetry's exponential backoff (500ms → 1000ms → 2000ms, real setTimeout) is legitimate production behavior for a genuine 403/429 — but several tests deliberately drive a fixture through the full retry loop to assert the resulting sync state (rate-limited status, capped segments), not the delay itself, and paid the full 3.5s wall time for it every time. client.ts gains setGithubRateLimitRetrySleepCapMsForTest() — it caps only what's actually awaited, not rateLimitRetryMs's return value, so the dedicated pure-function test asserting its exact backoff math (github-app.test.ts) is completely untouched and still exercises the real numbers.

Both defaults are wired once, suite-wide, via a new test/helpers/vitest-setup.ts (registered in vitest.config.ts's setupFiles) — the same ...ForTest override convention as clearInstallationTokenCacheForTest / setMaxChunksPerRepoForTest, applied automatically so every current and future test with this shape benefits without an author needing to opt in.

Measured: queue-lifecycle-guards.test.ts 33.7s → 7.2s; backfill.test.ts 19.2s → 1.7s.

Validation

  • npm run typecheck clean.
  • npm run test:coverage (full, unsharded): 21,482 passed / 0 failed.
  • github-app.test.ts's dedicated rateLimitRetryMs backoff-math tests still pass unmodified, confirming the override only touches the awaited delay, not the function's own correctness.

…n tests

Profiling queue-lifecycle-guards.test.ts (33.7s) and backfill.test.ts (19.2s)
found two more real per-test costs, confirmed via a CPU profile showing wall
time going to idle (real I/O wait), not compute.

SQLite fsyncs on every autocommit write by default; TestD1Database's clones
are real temp files (needed for the template-clone trick from the earlier
fix), so every app .prepare(...).run() paid a real fsync. journal_mode=MEMORY
+ synchronous=OFF are safe for a throwaway clone unlinked as soon as it's
open -- crash-consistency durability is meaningless for data nothing outlives
the test to read back. Benchmarked: 50 inserts, 13.9ms default vs 0.6ms tuned.

fetchWithGitHubRetry's exponential backoff (500/1000/2000ms real setTimeout)
is legitimate production behavior for a genuine 403/429, but several tests
deliberately drive a fixture through the full retry loop to assert the
resulting sync state, not the delay -- paying the full 3.5s every time.
client.ts gains setGithubRateLimitRetrySleepCapMsForTest(), capping only what
is actually awaited, not rateLimitRetryMs's return value -- its dedicated
pure-function backoff-math test stays untouched and still exercises the real
numbers.

Both wired once, suite-wide, via a new test/helpers/vitest-setup.ts
(vitest.config.ts setupFiles) -- same ...ForTest convention as
clearInstallationTokenCacheForTest, applied automatically so every current
AND future test with this shape benefits without opting in.

queue-lifecycle-guards.test.ts: 33.7s -> 7.2s. backfill.test.ts: 19.2s -> 1.7s.
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@JSONbored JSONbored self-assigned this Jul 24, 2026
@JSONbored
JSONbored merged commit 59faa47 into main Jul 24, 2026
4 checks passed
@JSONbored
JSONbored deleted the perf/test-sqlite-pragma-and-retry-delay branch July 24, 2026 20:38
@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 24, 2026
@codecov

codecov Bot commented Jul 24, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 75.00000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.50%. Comparing base (61f6ec6) to head (0c2337f).
⚠️ Report is 3 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
src/github/backfill.ts 75.00% 0 Missing and 1 partial ⚠️
src/github/client.ts 75.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8553      +/-   ##
==========================================
+ Coverage   89.58%   92.50%   +2.92%     
==========================================
  Files          97      793     +696     
  Lines       22706    79552   +56846     
  Branches     3872    24036   +20164     
==========================================
+ Hits        20341    73592   +53251     
- Misses       2187     4800    +2613     
- Partials      178     1160     +982     
Flag Coverage Δ
backend 93.67% <75.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/github/backfill.ts 95.90% <75.00%> (ø)
src/github/client.ts 99.62% <75.00%> (ø)

... and 694 files with indirect coverage changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant