v0.4.0
tagged this
15 Aug 15:27
Idin's correction, and it was right: the goal was never to rank results well. Precision is recoverable by whoever reads the results — a useless result costs a moment — while recall is recoverable by nobody, because nothing downstream can retrieve what was never returned. A hundred results of which ninety are garbage is fine; ten results missing the one that mattered is not. Reciprocal rank fusion is gone. Fusing to a single ordering only matters when returning a short list, and returning a short list was the mistake. Three places were destroying recall, all of them cuts I had chosen: a fuzzy floor at 0.72, a candidate pool capped at 20, and a final limit of 5. Selection is now a cascade. Every method runs, and each fills its own quota from what earlier methods did not already claim — so a chunk appears once, at the strongest method that found it, and no method spends its quota on chunks another already returned: exact uncapped — as good as a match gets starts_with shortest chunks first ends_with shortest chunks first contains shortest chunks first fuzzy by score cosine by similarity Anchored tiers order by chunk length because the query is most of a short chunk and a fraction of a long one — length says which is the better match without needing a model to judge. Idin's rule. Semantic runs last, not because it matters least but because a chunk whose words actually appear has already been found by a method that can say so. What reaches that tier is what nothing else could find. All tunable numbers move to search_config.ts. The candidate pool sizes are computed from the quotas rather than stored, since a stored copy drifts from the numbers it was meant to follow. Also adds restWorkersAiEmbedder, so everything the Worker does can be run from a terminal. A binding-only dependency made the embedding — the number this whole design rests on — unverifiable outside the runtime. Verified against the real store with real embeddings: "canine" over 402 chunks returns 77 results, and all three frodo.md chunks are found by cosine at 0.57-0.59 — by meaning, not by the "cannot" letter-coincidence that had been retrieving them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>