Skip to content

test(vector): assert routing invariant, not exact HNSW recall (#773)#774

Merged
mosuka merged 1 commit into
mainfrom
test/773-field-routing-flaky-recall
Jun 3, 2026
Merged

test(vector): assert routing invariant, not exact HNSW recall (#773)#774
mosuka merged 1 commit into
mainfrom
test/773-field-routing-flaky-recall

Conversation

@mosuka
Copy link
Copy Markdown
Owner

@mosuka mosuka commented Jun 3, 2026

Summary

vector_field_routing_test.rs asserted the presence of every expected doc (e.g. ids.contains(&1) && ids.contains(&2)) on a randomized HNSW graph. That is an exact-recall assertion, which is environment-sensitively flaky: the Test laurus (ubuntu-latest, x86_64-unknown-linux-gnu, stable) CI job intermittently returned only the nearest doc ({1}) and failed at vector_field_routing_test.rs:138, while the aarch64 and Windows jobs of the same run passed. It is not reproducible locally (75/75 passes here), consistent with HNSW build-RNG/connectivity non-determinism — the anti-pattern lessons.md (2026-05-29) already documents.

All five tests in the file shared this pattern, so all were latent flakes; only test_query_vector_fields_routes happened to fire.

Change

These are routing tests. The deterministic guarantee is that a query routed to a field returns only that field's docs (no cross-field leakage) — not that approximate search recalls every one. So:

  • Single-field routing tests now assert the result is non-empty and a subset of the targeted field's docs (drops the flaky per-doc contains, keeps the no-leak guarantee).
  • test_no_fields_searches_all asserts both fields are represented, anchored on the docs identical to the query (doc 1 in img_vec, doc 3 in txt_vec, similarity 1.0, reliably returned) instead of requiring all four docs.

The observed failure case {1} now passes deterministically ({1} ⊆ {1,2} and non-empty), so this resolves the flake without weakening the routing guarantee.

Verification

  • cargo fmt --check — clean
  • cargo clippy -p laurus --test vector_field_routing_test -- -D warnings — clean
  • Ran the test file 40× in a loop — 40/40 pass (was intermittently failing on CI).

Closes #773

The field-routing tests asserted the presence of every expected doc
(e.g. contains(&1) && contains(&2)) on a randomized HNSW graph. That is
an exact-recall assertion, which is environment-sensitively flaky: the
ubuntu-latest x86_64 CI job intermittently returned only the nearest doc
(e.g. {1}), failing the assertion, while aarch64/Windows passed.

These are routing tests: the deterministic guarantee is that a query
routed to a field returns only that field's docs (no cross-field leak),
not that approximate search recalls every one. Assert that invariant
instead — non-empty result that is a subset of the targeted field — and,
for the search-all case, that both fields are represented via the
query-identical docs that are reliably returned.

Closes #773
@mosuka mosuka merged commit 58c2952 into main Jun 3, 2026
50 of 52 checks passed
@mosuka mosuka deleted the test/773-field-routing-flaky-recall branch June 3, 2026 11:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test(vector): flaky field-routing test asserts exact HNSW recall on a randomized graph

1 participant