Skip to content

CodeDB 0.2.5862

Latest

Choose a tag to compare

@justrach justrach released this 27 Oct 02:08
2128707

CodeDB 0.2.5862

Enables the Zig bbtree inline-key dictionary by default for warm mmap word
indexes. Exact lookup and prefix lower bounds use prefix-stripped inline keys,
vector counts, and tied-key comparisons; posting lists remain zero-copy in the
existing word.index mapping. The disk format is unchanged.

Memory and correctness limits

All tree pages share one exact-size aligned slab, not one allocation per node.
The accelerator is capped at 8 MiB per mmap word index. If the dictionary
would exceed that budget, or the allocation fails, CodeDB retains the complete
binary-search dictionary. This does not shrink retrieval breadth. The first
mutation frees the tree before unmapping and promoting to the existing mutable
heap index; no accelerator memory is retained after promotion.

Ranking, query normalization, the hosted-default model, hosted exact fallback,
and ANN candidate breadth are unchanged. This release also includes the isolated
macOS watcher CPU fixes already on main through #758/#771 (derived from #750),
plus the verified-installer prerequisite in 0.2.5861. It deliberately does not
merge unrelated retrieval changes from the broader #750 branch.

Local measurements

Native arm64 macOS, pinned Zig 0.17.0-dev.813+2153f8143, ReleaseFast. The
comparison uses current source corpora from codedb, codegraff and zigrepper.
Sibling repositories and installed indexes are not modified.

Corpus Dictionary words Exact lookup paired improvement Prefix search paired improvement
codedb 13,031 29.57% 18.76%
codegraff 42,100 44.70% 13.24%
zigrepper 36,994 30.24% 11.47%

Seven alternating AB/BA rounds, pre-warmed mappings, production C allocator.
Percentages are medians of paired candidate/baseline ratios, not ratios of
independent medians. Timing is noisy; these are dictionary microbenchmarks,
not hosted or end-to-end context latency claims.

Native CLI peak RSS was measured separately with six alternating pairs of
bench-engine word function 10000, isolated HOME directories, explicit empty
configuration, disabled telemetry/update/daemon/warmup, denied network access,
and disposable source copies. Both implementations returned the same nonzero
hit counts. Maximum resident bytes come from macOS /usr/bin/time -l.

Corpus Median peak RSS increase Relative increase
codedb 0.47 MiB 4.84%
codegraff 1.39 MiB 3.10%
zigrepper 1.21 MiB 3.69%

All three stayed below the 5% relative and 10 MiB absolute acceptance guards.
This is bounded additional memory, not zero-cost memory: the single-word CLI
measurement is not whole-index residency or a long-lived daemon RSS guarantee.
The hard slab ceiling prevents growth with arbitrarily large dictionaries.

Verification

  • Full unit build: all 37 steps pass, including the default-on mmap path,
    binary-search control, promote-on-write cleanup, allocation-failure/budget
    cases, multilevel boundaries, and maximum disk-word prefix lengths. An
    existing semantic retry timeout under concurrent compilation passed when
    rerun without competing builds; unchanged successful suites were cached.
  • ReleaseFast CLI build passes with the pinned release compiler.
  • MCP E2E: 76/76 scenarios pass on the final frozen candidate.
  • All exact lookups and lower bounds match across all three corpora; each
    corpus also matches 1,000 full prefix results and all postings after promotion.
  • Watcher stress: 4,800 files / 1,200 directories, 35-second quiet and churn
    windows. Three clients with watch budget 32 measured 0.54–0.69% idle CPU each
    and 1.37–1.46% under excluded churn. One client at budget 1,024 measured 0.89%
    idle and 1.57% churn. All 122 mutation checks passed; counters stayed
    unchanged during excluded churn. 100% CPU is one core.

Aggregate measurements and individual RSS samples are recorded in
../measurements/bbtree-0.2.5862.json.
Benchmark entry points: experiments/bbtree/run.py and
experiments/bbtree/rss.py. The source port retains upstream attribution and
MIT notice in src/bbtree.LICENSE.