Skip to content

v1.108.147 — single-flight cold index loads and BM25 builds (#370)

Choose a tag to compare

@jgravelle jgravelle released this 20 Jul 10:29

Fixes #370 (reported by @rknighton): concurrent cold searches no longer repeat whole-index work.

At scale (a 665k-symbol, ~0.5 GB SQLite index), 13 simultaneous cold search_symbols calls each took 7.5–11.4 minutes and the worker peaked ~16 GiB, because every cold caller independently hydrated the same repository index and independently built the full-corpus BM25 state. Two fixes:

  • Single-flight cold hydration. load_index now serializes the cold path per repo: one caller reads the rows and builds the CodeIndex, concurrent callers wait and take the freshly cached object. N cold callers cost one hydration's time and memory instead of N.
  • Per-index BM25 build lock. CodeIndex carries a _bm25_lock, used double-checked at every _bm25_cache build site (search_symbols, get_ranked_context, plan_turn, find_implementations, get_repo_map) plus register_edit's clear.

Warm-path behavior, result shapes, and ranking are byte-identical; only duplicated work is removed. Regression tests use an 8-thread barrier asserting exactly one hydration / one BM25 build.

Known residual: cancelling a client request still cannot stop in-flight server-side work (tool bodies run to completion in a worker thread); single-flight bounds the damage to one hydration rather than N.