perf(search): indexed hybrid meets 100k SLO — normalized-L2 vec index + 2GB mmap#225
Merged
Conversation
Report whether hybrid search used sqlite-vec KNN, brute-force vector scan, or no semantic backend so MCP output and tests distinguish fallback from indexed search. Extend Suite F to emit indexed vector and hybrid latency metrics when sqlite-vec is available, and commit the captured Suite F result artifacts.
bun's mock.module applies to every test file in the run, so the stubbed knnSearch (empty hits) and the leaked vecAvailable=true broke tests/db/vec.test.ts whenever both files shared an invocation — the macOS CI failure on this PR. The mock now delegates to the real module except between this file's beforeAll/afterAll (mockEngaged). The real implementations are snapshotted BEFORE mock.module registers: bun patches existing importers too, so a later require returns the mock itself and the delegation self-recurses (observed as a 100% CPU hang).
edheltzel
added a commit
that referenced
this pull request
Jul 13, 2026
… conversion RedTeam gate finding on PR #225: 'exactly preserves ordering' overstated the guarantee. Ordering is exact for practically-distinct vectors; at genuine near-ties (<=~1e-6 cosine-distance separation, float32 noise floor) positions may permute within the top-K while distance values stay accurate to ~5e-7. Top-K membership is unaffected. Noted at the cos_dist = L2^2/2 conversion, the vec0 DDL comment, and the CHANGELOG entry. Refs #217
… embeddingsAvailable on embed failure Implements the #217 review findings for PR #224 (Themis design ruling): - SemanticBackend documents and enforces 'which backend RAN' semantics: the brute-force branch reports 'bruteforce' even on zero hits; 'none' is reserved for the semantic tier never executing (224-2 ruling). - hybridSearch's catch resets embeddingsAvailable=false — the flag was set optimistically before a stale-liveness embed() could throw, so the mode note reported hybrid for a query whose embeddings never materialized (224-1). Semantics change on the FTS-only path disclosed in CHANGELOG. - New tests: knn-throws -> brute-force fallback still surfaces semantic hits and reports 'bruteforce' (224-3); embed-throws -> embeddingsAvailable false + backend 'none' (224-1). - Test-file hygiene (224-6): the sibling embeddings mock.module gets the same mockEngaged gating as the vec mock (delegates to the real module outside this file's tests) and the snapshot comment now states why the in-factory require spread is safe but later lookups self-recurse. - CI (Linux native-load step) now pins tests/benchmarks/suite-f.test.ts and tests/mcp-hybrid-vec-fallback.test.ts so vec-enabled coverage is explicit rather than incidental to the macOS runner image. - CHANGELOG wording: no 'counters' metric exists (conditional latency pairs); semanticBackend is reported in the prose mode note, not a structured field (224-4). Stale line-number docstring refs in Suite F replaced with symbol names (224-5). Refs #217
…er at 100k The vec0 KNN scan was the entire 100k SLO miss (~185ms of the ~180ms hybrid_index p50; FTS, RRF, provenance resolution together ~4ms): sqlite-vec's cosine kernel is scalar while its L2 kernel is SIMD, and the 256MB mmap cap forced page reads on a ~1GB 100k-embedding DB. - vec_embeddings now stores vec_normalize()d vectors under the default L2 metric; for unit vectors L2^2 = 2*(1 - cosine), so knnSearch converts distances back exactly (cos_dist = L2^2/2) — ordering and the VecHit cosine-distance contract are unchanged, verified against brute-force. - createVecTable self-heals a pre-#217 cosine-metric index (metric is baked into vec0 DDL): drop + recreate, next ensureVecIndexSynced re-syncs. - mmap_size raised 256MB -> 2GB (address space, not resident RAM). Refs #217, #146
… conversion RedTeam gate finding on PR #225: 'exactly preserves ordering' overstated the guarantee. Ordering is exact for practically-distinct vectors; at genuine near-ties (<=~1e-6 cosine-distance separation, float32 noise floor) positions may permute within the top-K while distance values stay accurate to ~5e-7. Top-K membership is unaffected. Noted at the cos_dist = L2^2/2 conversion, the vec0 DDL comment, and the CHANGELOG entry. Refs #217
…knn backend Blocking review cluster 225-1 (+ Themis #217 design ruling): - ensureVecIndexSynced latches syncedThisProcess only on SUCCESS (with an in-flight guard): a failed rebuild (e.g. SQLITE_BUSY losing the race to a concurrent writer right after an upgrade drops the legacy index) now retries on the next vector query instead of sticking for the process lifetime, and logs '[recall] vec index rebuild failed' to stderr. The rebuild itself also announces on stderr (one-time ~4s @100k inside the first vector query — not a hang). - vectorSearch treats an empty KNN result over a NON-empty canonical embeddings table as a failure (knnSearch returns [] rather than throwing on an empty index) and falls through to the brute-force scan, so semantic search keeps working and the semanticBackend label stays truthful. Empty KNN over an empty embeddings table remains a valid 'knn' run. - Once-per-process '[recall] vec KNN failed / returned no hits' stderr in the fallback transitions, matching the [recall] convention. - Tests: empty-KNN-with-embeddings now pins the brute-force fallback (was pinned as 'knn with zero hits'); new zero-corpus test pins 'knn' and 'bruteforce' labels over an empty embeddings table. - CHANGELOG: REPEATS=3-before vs REPEATS=12-after protocol caveat (225-5); self-heal timing corrected to 'dropped on next open, rebuilt on first vector query' (225-9). mmap comment notes the SQLITE_MAX_MMAP_SIZE clamp at 0x7fff0000 (225-8). Refs #217
edheltzel
force-pushed
the
perf/knn-l2-normalized-index
branch
from
July 13, 2026 09:29
5dfa462 to
5aafc10
Compare
This was referenced Jul 13, 2026
Harden vec index: zero-vector guard, index-format version stamp, doctor/stats vec observability
#226
Open
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes the 100k hybrid-search SLO miss tracked in #217 (epic #146): hybrid_index_latency_p50 @100k: 180.4ms → 23.5ms (7.7×), p95 41.1ms — both under the ≤50ms acceptable SLO, on the KNN-backed path.
Stacked on #224 (base branch is
feat/semantic-backend-metrics) — retarget tomainafter #224 merges. Do not merge before it; Themis gates both.Profiling attribution (scratch harness, not committed)
At 100k×1024-dim embeddings the vec0 KNN scan was effectively the whole miss:
MATCH, k=20)search()Two compounding causes: sqlite-vec 0.1.9's cosine kernel is scalar while its L2 kernel is SIMD (~2× per row), and the 256MB
mmap_size(#151) overflows on a ~1GB 100k DB, degrading the scan to per-page reads (~2×).The fix (ordering-preserving for practically-distinct vectors)
vec_embeddingsstoresvec_normalize()d vectors under the default L2 metric. For unit vectorsL2² = 2·(1−cos), soknnSearchconverts each distance back (cos_dist = L2²/2) — ordering is exact for practically-distinct vectors, and theVecHitcosine-distance contract is unchanged (asserted against brute-force intests/db/vec.test.ts, incl. a norm-invariance value test with non-unit stored vectors). Precision boundary (RedTeam gate finding): at genuine near-ties (≲1e-6 cosine-distance separation, float32 noise floor) positions may permute within the top-K; distance values stay accurate to ~5e-7 and top-K membership is unaffected. Large-scale ordering-parity coverage is logged as test-debt for a follow-up issue.mmap_size256MB → 2GB (address space, not resident RAM).createVecTabledetects the stale DDL viasqlite_master, drops it, and the nextensureVecIndexSyncedrebuilds from the canonical embedding BLOBs (one-time, ~4s @100k).Rejected alternatives (measured): binary quantization + exact re-rank (10–30ms but 16–34% top-20 recall on synthetic vectors), int8 (123ms and poor recall), cosine+mmap alone (kernel still scalar).
Evidence (committed artifacts)
benchmarks/results/2026-07-12T23-05-01-suite-F.*(REPEATS=3, pre-fix): hybrid_index p50 1.9 / 12.5 / 180.4ms at 1k/10k/100k.benchmarks/results/2026-07-13T00-24-24-suite-F.*(REPEATS=12): hybrid_index p50 1.0 / 3.7 / 23.5ms, p95 41.1ms @100k.hybrid_index_latency_*is emitted only when sqlite-vec loaded (isVecAvailable), and feat(search): expose semantic backend metrics #224'ssemanticBackendinstrumentation distinguishes knn from bruteforce at runtime.Caveats: synthetic deterministic vectors (latency-only, per Suite F design); absolute ms are machine-local (Apple Silicon, Homebrew libsqlite3); the Ollama embed(query) call is excluded by design.
Verification
bun test tests/db/vec.test.ts tests/benchmarks/suite-f.test.ts tests/mcp-hybrid-vec-fallback.test.ts tests/hybrid-search-fts-fallback.test.ts tests/lib/bump-access.test.ts— 29 pass / 0 failbun run lint— cleanbun teston the base branch: 1227/1228 (the 1 fail istests/hooks/RecallExtract-archive-scrub.test.ts, which fails identically on an untouchedmaincheckout on this machine — environmental, pre-existing; CI is green)Refs #217, refs #146. Does not close either — Themis owns closure.
🤖 Generated with Claude Code