Skip to content

docs: finalize RFC 0014 — exact KNN query mode (accept) #19

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
hev merged 2 commits into from
Jul 8, 2026
Merged

docs: finalize RFC 0014 — exact KNN query mode (accept) #19

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
467d79d
docs: finalize RFC 0014 exact KNN mode
hevbot Jul 8, 2026
75fdea5
docs: resolve cost-guardrail question — no guardrail in v1 (hev, PR #19)
hevbot Jul 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
289 changes: 165 additions & 124 deletions docs/rfcs/0014-exact-knn-query-mode.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,142 +1,183 @@
# RFC 0014: Exact KNN as an explicit query mode

> **Status:** draft, proposal. Add an explicit **exact (brute-force) KNN mode**
> to `/query` — `exact: true` — that bypasses the vector index and scans every
> row, returning 100%-recall results. The implementation hook already exists in
> the pinned LanceDB: `VectorQuery::bypass_vector_index()`
> (`lancedb-0.29.0/src/query.rs:1184`); the engine just never calls it. The
> philosophical reference is [LodeDB](https://github.com/Egoist-Machines/LodeDB)'s
> exact-by-default posture; this RFC adopts exact-on-request, not
> exact-by-default.
> **Status:** Accepted (2026-07-08). Add an explicit **exact
> (brute-force) KNN mode** to `/query` with `exact: true`. Exact mode
> bypasses the vector index, scans all committed rows that satisfy the
> query predicate, and returns the engine's 100%-recall nearest-neighbor
> answer for that query shape. The implementation hook already exists in
> the pinned LanceDB: `VectorQuery::bypass_vector_index()`. The
> philosophical reference is
> [LodeDB](https://github.com/Egoist-Machines/LodeDB)'s exact-by-default
> posture; this RFC adopts exact-on-request, not exact-by-default.

## Summary

The engine can already scan brute-force — but only *by accident*. When a
namespace has no vector index, LanceDB "brute-force scans every row, which is
fine for tiny development corpora" (`manager.rs:1520-1524`). The moment an
IVF_PQ index is built, every query goes through it, and there is **no way to
ask for the exact answer**: the query surface exposes only `nprobes`
(`manager.rs:1561`, default 20). Exactness is a side effect of not having
built an index yet, not a capability.
The engine can already scan brute-force, but only as an incidental
consequence of index state. When a namespace has no vector index, LanceDB
linear-scans the table. The moment an IVF_PQ index is built, the vector query
path uses that index and the public query surface exposes only `nprobes`
(`QueryRequest::nprobes`, default 20). Exactness is a side effect of not having
built an index yet, not a requestable capability.

This RFC makes it a capability: `exact: true` on `/query` routes the vector
(and multivector MaxSim) search through `bypass_vector_index()`, prefilter
(`only_if`) and projection behavior unchanged. `k`, `filter`,
`include_vector`, hybrid/FTS composition all keep their semantics; `nprobes`
is ignored (rejected?) when `exact` is set.
This RFC makes exactness a capability. `exact: true` on `/query` routes the
vector leg of a vector or hybrid query through `bypass_vector_index()`.
Prefilter (`only_if`), distance metric, projection, result shape, fuzzy FTS,
and hybrid composition keep their existing semantics. `nprobes` is invalid
when `exact` is set because there are no IVF partitions to probe.

## Motivation

1. **Ground truth for recall measurement, in-engine.** RFC 0011's recall
harness needs exact nearest neighbors to score against. For SIFT/GIST that
ships with the dataset, but for *any other corpus* — a customer namespace, a
BEIR multivector set, a filtered subset — ground truth must be computed
somewhere. With `exact: true`, the harness computes it through the same
engine, same distance metric, same filter path it is evaluating: run the
query exact, run it approximate, diff. This makes recall self-calibrating on
arbitrary workloads and — critically — measures **filtered recall** (RFC
0011's selectivity axis) with ground truth that respects the same prefilter,
which no pre-shipped dataset provides.

2. **The PQ recall ceiling needs an escape hatch.** The benches show recall@100
flat at **0.707 across nprobes** — a quantization ceiling no `nprobes`
increase can pierce (the motivation for RFC 0009's `refine_factor`). Until
0009 lands (and for callers whose k is small and namespace is modest),
`exact: true` is the correct-answer path: an exact scan over a 100k-row
namespace is cheap; a 0.707 ceiling on it is not.

3. **Small namespaces shouldn't pay ANN error at all.** Many real namespaces
(per-tenant demo corpora, `label` sections, dev environments) are tens of
thousands of rows. LodeDB's exact-by-default posture is right for that
regime — exact scan at that scale is milliseconds, and 100% recall is free.
We don't adopt exact-by-*default* (the engine's target regime is large; a
silent default flip at index-build time would be a semantics landmine), but
exact-on-request lets Layer or a bench choose correctness per query.

4. **Debugging surface.** "Is this a recall problem or a filter/fusion
problem?" is currently unanswerable without exporting vectors to an
external tool. `exact: true` answers it with one request.
harness needs exact nearest neighbors to score against. Today the harness
can load SIFT/GIST-style `.ivecs` ground truth or compute synthetic
brute-force ground truth in-process, but any real engine corpus, filtered
subset, or multivector workload still needs the engine itself to answer the
exact query it is evaluating. With `exact: true`, the harness can run the
exact query and the indexed query through the same engine, same distance
metric, same filter path, and compare the result sets. That is the target
state for the bench-evidence lane.

2. **Filtered recall needs the same prefilter path as production.** RFC 0011's
selectivity axis explicitly needs ground truth among rows that satisfy the
predicate. Pre-shipped ANN datasets do not provide that, and external
brute-force scripts easily drift from the engine's DataFusion/Lance filter
behavior. Exact mode makes filtered recall an engine contract rather than a
side calculation.

3. **The PQ recall ceiling needs an escape hatch.** The current benches show
recall@100 flat at about 0.707 across `nprobes` on the FIQA nprobes sweep,
a quantization ceiling that larger `nprobes` cannot pierce. RFC 0009's
`refine_factor` work should raise the approximate ceiling; exact mode is
the reference answer that proves it and the correctness path for modest
namespaces where latency is acceptable.

4. **Small namespaces should not pay ANN error by default.** Many practical
namespaces, including demo corpora, per-label slices, and development
environments, are small enough that a scan is cheap. The engine should not
silently flip global defaults based on size or index state, but callers and
benches should be able to request the correct answer explicitly.

5. **Debugging surface.** "Is this a recall problem or a filter/fusion
problem?" is currently hard to answer without exporting vectors. Exact mode
provides a direct comparison point inside the service boundary.

## Proposal

- **`/query` gains `exact: bool` (default `false`).** When set, the vector leg
of the plan calls `bypass_vector_index()`; everything else (prefilter,
hybrid RRF composition, fuzzy FTS, `include_vector`, multivector MaxSim) is
unchanged. FTS-only queries reject `exact` (nothing to bypass) — or ignore
it; open question.
- **`nprobes` + `exact` together is a 400** — the request contradicts itself.
- **Result parity contract:** an exact query and an approximate query over the
same committed version differ only in the candidate set. Distance metric,
filter semantics, and score computation are identical — this is what makes
the exact/approx diff a recall measurement rather than an apples-to-oranges
comparison. (LanceDB's flat scan and IVF_PQ path both honor deletion
vectors and `only_if`, so this should hold; the harness asserts it.)
- **Caching:** exact results ride the existing result cache keyed on the
manifest generation (`manager.rs:591-651`) — `exact` must be part of the
cache key so exact and approximate answers for the same request never
collide. Same for the semantic cache (or exact queries skip it; open
question — a semantic-similarity hit returning *approximate* results for an
*exact* request violates the contract).
- **Cost guardrail (open):** an exact scan on a 100M-row namespace is a
self-inflicted wound. Options: none in v1 (trusted internal caller — Layer —
behind the `NetworkPolicy`, matching the RFC 0002 posture), or a row-count
soft cap with an explicit override.

## Interactions with other RFCs

- **RFC 0011 (recall harness).** The headline consumer: ground-truth
generation moves in-engine, filtered-recall ground truth becomes possible,
and the harness gains an exact-vs-approx self-check on every dataset it
loads. The harness should also record exact-scan latency per corpus size —
that curve *is* the "when do you need an index" guidance.
- **RFC 0009 (pluggable index / refine_factor).** Complementary: 0009 raises
the approximate ceiling; this RFC provides the reference answer that proves
it. `refine_factor` sweeps in the harness are scored against `exact: true`
runs.
- **`/query` gains `exact: bool` with default `false`.** Existing callers keep
current behavior unless they opt in.
- **Vector-only and hybrid queries honor `exact: true`.** The vector leg calls
`bypass_vector_index()` before execution. For hybrid queries, BM25/fuzzy FTS
remains the same and exactness applies only to the vector candidate leg.
- **FTS-only queries reject `exact: true` with HTTP 400.** There is no vector
index to bypass, and accepting a no-op flag would make request intent
ambiguous.
- **`exact: true` and `nprobes` together reject with HTTP 400.** `nprobes` is
an ANN/IVF query knob; exact scan has no partitions to probe. Rejecting the
combination is clearer than silently ignoring caller input.
- **Semantic cache is disabled for exact queries.** An exact request must not
be served by a near-duplicate cached result. Implement this as a validation
rejection if `semantic_cache.enabled` and `exact` are both set, and ensure
exact queries do not insert into the semantic sidecar.
- **Exact result-cache entries are allowed and keyed distinctly.** The normal
generation-keyed result cache can cache exact results, but `exact` must be
part of the exact-cache key so exact and indexed answers never collide.
- **Result parity contract:** an exact query and an indexed query over the same
committed table version differ only in candidate selection. Distance metric,
filter semantics, deletion-vector handling, projection, result decoding, and
scoring fields use the same engine path wherever LanceDB allows it. The
recall harness should assert this on fixture data.
- **No edge authorization is introduced here.** Exactness is an engine query
plan. If Layer later needs per-tenant policy around exact scans, that remains
an edge authorization concern.

## Bench-Evidence Lane

This RFC is factory evidence plumbing for RFC 0011 and the `bench-evidence`
workflow in PR #10. The current `hevsearch-bench` recall sweep establishes the
shape of the lane: it runs the real `NamespaceService`, computes or loads exact
nearest-neighbor ground truth, self-checks an unindexed linear-scan namespace,
then sweeps IVF_PQ `nprobes` and emits recall/nDCG/latency/build evidence.

After exact query mode lands, the harness should move the engine-ground-truth
case from "separate unindexed namespace or in-process brute force" to
`exact: true` against the same committed namespace being measured. That matters
for indexed namespaces, filtered recall, multivector MaxSim, and any corpus
whose exact ground truth is not shipped as `.ivecs`.

PR #10's workflow currently validates pinned mesh-account ECR digest inputs and
a corpus hash, runs `recall_sweep` against in-job MinIO, and uploads the
evidence markdown/JSON artifact. Its review notes one design flag: as written,
the job records `image_tag` as provenance while building and running the bench
from source at the checkout SHA. That is acceptable context for this RFC, but
bench publication must either run the pinned image or rename that provenance
before downstream evidence is treated as image-execution proof.

## Interactions With Other RFCs

- **RFC 0011 (recall harness).** Exact mode lets the harness generate
ground-truth results in-engine for arbitrary corpora and filters. The harness
should record exact-scan latency by corpus size; that curve becomes the
practical "when do you need an index" guide.
- **RFC 0009 (pluggable index / `refine_factor`).** Complementary. RFC 0009
raises the approximate ceiling; this RFC provides the reference answer used
to prove it. `refine_factor` and future index-kind sweeps should score
against `exact: true`.
- **RFC 0006 (configurable distance metric).** Exact scan must honor the
namespace's metric — it comes free (LanceDB's flat path takes the same
`distance_type`), but the parity contract makes it explicit.
- **RFC 0010 (late interaction).** Multivector MaxSim without an index is
already a brute-force plan (`manager.rs:1520-1524`); `exact: true` makes
that plan requestable after an index exists, giving the BEIR multivector
harness its MaxSim ground truth (and the f16-vs-f32 precision comparison a
clean baseline).
- **RFC 0086 (Layer twin).** Whether/how `exact` surfaces on the
Turbopuffer-shaped inbound wire is Layer's call (Turbopuffer has no such
knob, so it is likely operator/bench-only initially). The engine ships the
capability; the edge decides the exposure.

## Engine vs edge boundary

Exactness is a property of the search plan — engine, unambiguously. The only
edge question is wire exposure (above). A per-tenant "may this caller run
exact scans" policy, if ever wanted, would be Layer's authorization concern,
not an engine knob.

## Open questions

- **FTS-only + `exact`:** reject or ignore? (Hybrid with `exact` applies to
the vector leg only — BM25 is already exact.)
- **Semantic cache:** skip on `exact: true`, or include `exact` in its key?
Skipping is safer (the contract is exactness; similarity-matching it away is
a category error).
- **Cost guardrail:** none (trusted-caller posture), or a soft row-count cap
with override? If none, does `namespace_info` at least surface an
exact-scan cost estimate?
- **Naming:** `exact: true` vs `ann: false` vs `mode: "exact"`. `exact` reads
best on the wire and matches LodeDB/common usage.
namespace metric. The manager already applies `distance_type(...)` on the
vector query; exact mode must preserve that call.
- **RFC 0010 (late interaction).** Multivector MaxSim without an index already
has a brute-force behavior. Exact mode makes that behavior requestable after
an index exists, giving the BEIR multivector harness a clean MaxSim baseline.
- **RFC 0086 (Layer twin).** Whether `exact` surfaces on Layer's
Turbopuffer-shaped inbound wire is Layer's decision. The engine ships the
capability; the edge decides exposure.

## Engine vs Edge Boundary

Exactness is a property of the search plan, so it belongs in the engine. The
engine should validate contradictory query knobs and execute the requested
plan. It should not add tenant policy, authorization, quota, or public exposure
rules for exact scans; those belong in Layer if they are needed.

## Resolved Questions

- **Cost guardrail (resolved 2026-07-08, hev on PR #19):** v1 ships with **no
row-count guardrail** under the trusted-caller posture. Layer is the only
data-path client and owns any caller-facing limits; the engine stays
knob-free until a workload demonstrates the need.

- **Naming:** use `exact: true`. It is direct, matches common exact-search
language, and avoids making ANN the conceptual default in the wire contract.
- **FTS-only + exact:** reject with HTTP 400. BM25 is already the relevant
exact retrieval mode, and the flag has no vector meaning without a vector
leg.
- **`nprobes` + exact:** reject with HTTP 400. The knobs describe mutually
exclusive execution plans.
- **Semantic cache:** reject `semantic_cache.enabled` with exact mode and skip
semantic sidecar insertion for exact results. Semantic-cache hits are
approximate by design and cannot satisfy the exactness contract.
- **Exact result cache:** keep it, but include `exact` in the cache key. The
existing exact result cache is keyed by namespace generation and canonical
query fields; exactness must become one of those fields.

## References

- `~/.cargo` pinned `lancedb-0.29.0/src/query.rs:1184` —
`VectorQuery::bypass_vector_index()`; no version bump needed.
- `crates/hevsearch-core/src/manager.rs:1520-1524` — brute-force scan today
only when no index exists; `:1541-1542,1561` — `nprobes` is the only vector
query knob; `:591-651` — result cache keyed on manifest generation.
- `bench/results/` — recall@100 flat at 0.707 across nprobes (the PQ ceiling;
see RFC 0009).
- RFC 0011 (ground truth + filtered recall), RFC 0009 (refine_factor),
RFC 0010 (MaxSim baseline), RFC 0006 (metric parity) — engine neighbors.
- `crates/hevsearch-core/src/query.rs` — current `QueryRequest` exposes
`nprobes`, text/fuzzy/filter/projection, and `semantic_cache`, but no
`exact` field.
- `crates/hevsearch-core/src/manager.rs` — current query construction applies
`distance_type(...)`, `.nprobes(...)`, `.limit(k)`, optional
`full_text_search(...)`, `only_if(...)`, and projection before execution.
- `crates/hevsearch-core/src/service.rs` and `src/cache/semantic.rs` — exact
result cache, semantic sidecar semantics, and cache-key shape.
- `crates/hevsearch-bench/src/bin/recall_sweep.rs` and
`src/recall.rs` — current RFC 0011 recall harness: unindexed exact baseline,
synthetic brute-force ground truth, `.ivecs` ground truth, and IVF_PQ
`nprobes` sweep.
- `bench/results/beir_multivector_raw/nprobes_sweep_fiqa/` and RFC 0009 —
recall@100 flat near 0.707 across `nprobes`, motivating an exact reference
and future refine/index-kind work.
- RFC 0011 (recall harness), RFC 0009 (refine/index-kind work), RFC 0010
(MaxSim baseline), RFC 0006 (metric parity), and Layer RFC 0086.
- [LodeDB](https://github.com/Egoist-Machines/LodeDB) — exact-by-default
reference ("Vector search: exact scan by default (100% recall); ANN is
opt-in").
reference posture.
Loading