Skip to content

perf(mcp): 2.2× codedb_context — line-offset-cache windows for top sites#646

Merged
justrach merged 1 commit into
release/0.2.5828from
perf/context-line-spans
Jul 4, 2026
Merged

perf(mcp): 2.2× codedb_context — line-offset-cache windows for top sites#646
justrach merged 1 commit into
release/0.2.5828from
perf/context-line-spans

Conversation

@justrach

@justrach justrach commented Jul 4, 2026

Copy link
Copy Markdown
Owner

What

codedb_context is the heaviest gated bench case (3.6× the next tool) and the flagship first-touch call — and its "Top sites (±2 lines)" phase was 63% of the wall time: for every hit in every top file it walked the whole file byte-by-byte from offset 0 to locate the [line-2 .. line+2] window.

This PR resolves both window edges through the existing #611 LineOffsetCache in O(log n) instead, borrowing the canonical cached bytes (stable pointer — the cache keys on it; also kills the per-file arena copy) under the shared lock the phase already holds. Uncached files (released contents / CODEDB_LOW_MEMORY) keep the old scanning path as a fallback, so behavior is unchanged everywhere the fast path can't prove itself.

Two small Explorer accessors (cachedContentLocked, lineSpansFor) expose the existing private machinery; no new data structures.

Also ships CODEDB_CONTEXT_PROFILE=1 — per-phase ns breakdown of the composer on stderr (same pattern as CODEDB_LOAD_PROFILE). It's what located this hotspot, and it maps the next ones: the per-keyword search loop (48% of what remains) and the callers/callees render (32%).

Numbers (CI-identical recipe: run-bench-json.py + compare-bench.py, same machine)

base head delta
codedb_context avg 224.8µs 102.6µs −54.4%
phase: sites 122.9µs 0.9µs −99.3%
phase profile total 195.1µs 70.9µs −63.7%

Every other gated tool: OK (no regressions; most improved slightly from run noise).

Quality

  • Slice math is edge-for-edge identical to the scanning loop (EOF-truncated windows, missing trailing newline, hit lines past EOF, empty files — all verified against lineSpans's split semantics).
  • The 1-byte response_bytes delta in the bench is the response's own embedded elapsed-time string (mcpFormatDuration at mcp.zig runToolCall/handleCall) rendering fewer digits because the call got faster — confirmed by A/B'ing the binaries: the only differing bytes are the duration digits.
  • zig build test: 23/23 steps, 865/865. e2e_mcp_test.py: 20/20.

🤖 Generated with Claude Code

The 'Top sites (±2 lines)' phase walked every byte of each top file from
offset 0 once per hit to find its [line-2 .. line+2] window — 63% of
codedb_context wall time (sites 122.9µs of 195.1µs on the bench corpus).
Borrow the canonical cached bytes (stable pointer, no arena copy) under
the already-held shared lock and resolve both window edges through the
line-offset cache (#611) in O(log n), keeping the scanning path as the
fallback for uncached files (released contents / low-memory).

Also ships CODEDB_CONTEXT_PROFILE=1: per-phase ns breakdown of the
composer on stderr (reader/cand/kwloop/rank/render/sites/emit), same
pattern as CODEDB_LOAD_PROFILE — it is what located this hotspot.

Gated bench (scripts/compare-bench.py, same machine, CI recipe):
  codedb_context 224.8µs -> 102.6µs (-54.4%); every other tool OK
  phase profile: sites 122.9µs -> 0.9µs; total 195.1µs -> 70.9µs
Output is byte-identical modulo the response's own embedded elapsed-time
string (mcpFormatDuration) rendering fewer digits because the call got
faster.

zig build test: 23/23 steps, 865/865. e2e_mcp_test.py: 20/20.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 4, 2026

Copy link
Copy Markdown

Benchmark Regression Report

Thresholds: 10.00% and 50,000 ns absolute delta

NOISE means the percentage threshold was exceeded, but the absolute delta was too small to fail CI.

Tool Base (ns) Head (ns) Delta Abs Delta (ns) Status
codedb_bundle 71876 79398 +10.47% +7522 NOISE
codedb_changes 10572 14097 +33.34% +3525 NOISE
codedb_context 728663 285471 -60.82% -443192 OK
codedb_deps 397 334 -15.87% -63 OK
codedb_edit 41994 44141 +5.11% +2147 OK
codedb_find 2634 2612 -0.84% -22 OK
codedb_hot 25206 25222 +0.06% +16 OK
codedb_outline 15882 16543 +4.16% +661 OK
codedb_read 12647 12636 -0.09% -11 OK
codedb_search 64389 65948 +2.42% +1559 OK
codedb_snapshot 81754 67567 -17.35% -14187 OK
codedb_status 9734 9139 -6.11% -595 OK
codedb_symbol 62733 58416 -6.88% -4317 OK
codedb_tree 33675 37387 +11.02% +3712 NOISE
codedb_word 12768 11984 -6.14% -784 OK

@justrach justrach merged commit ca50821 into release/0.2.5828 Jul 4, 2026
2 checks passed
@justrach

justrach commented Jul 4, 2026

Copy link
Copy Markdown
Owner Author

Post-merge addendum for the record — closing the last theoretical gap in the byte-parity claim:

The fast path borrows content via Explorer.cachedContentLockedcontents.get(path). The old path went through readContentForSearch, whose first branch is the very same contents.get(path) (explore.zig:2490) — the two paths read from one source whenever the cache holds the file. In the only case they'd diverge (cache miss → old path reads disk), the fast path is not taken at all: the original scanning code runs unchanged as the fallback. So identical bytes are structural, not coincidental — on top of the edge-for-edge window-math verification (EOF-truncated windows, no-trailing-newline files, hit lines past EOF, empty files) and the empirical A/B, where the only differing response bytes were the embedded mcpFormatDuration digits shrinking because the call got faster.

justrach added a commit that referenced this pull request Jul 4, 2026
…ineOffsetCache (#653)

The BM25 top-k materialization loop called extractLineByNumber per
result — a scan from byte 0 of the whole file — up to max_results times
per ranked query. Resolve the hit line through the line-offset cache
(O(log n)) exactly like the exact-recall tiers (#611) and the context
sites phase (#646) already do, keeping the scanning path as the fallback
when the offset table can't be built.

Same-session gated bench: codedb_context 129.0µs -> 113.6µs (-12%,
context's keyword loop materializes ranked hits); codedb_search flat on
the tiny hit-cached bench corpus — the win scales with file size on real
repos, where each materialized hit was a whole-file walk. Full suite
23/23 steps, 869/873 (4 platform skips).

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant