Exact Vector Search Performance Study

Exact Vector Search Performance Study

Benchmark hardware: All performance measurements on this page were collected on an Apple M1 Max. Results on other systems will vary.

Summary

emailindex migrated its semantic retriever from a scalar SQL cosine scan to stable sqlite-vec vec0 MATCH exact K-nearest-neighbor retrieval. The change preserves exact cosine ranking and the existing hybrid Reciprocal Rank Fusion strategy; it does not introduce approximate nearest-neighbor indexing.

On the production-sized 81,264-vector archive, the implemented cosine MATCH path reduced warm unfiltered vector SQL latency from about 1.10 seconds to 50.5 milliseconds (about 21.8x faster). A date-filtered query improved from about 3.35 seconds to 96.0 milliseconds (about 34.9x faster). The measured result IDs and ordering were identical for both benchmark cases.

Why This Change

The previous query joined every vector to its email, called vec_distance_cosine() once per row, sorted the complete result set, and then retained the top K:

SELECT e.id, vec_distance_cosine(ev.embedding, ?) AS distance
FROM emails e
JOIN email_vectors ev ON e.id = ev.email_id
ORDER BY distance ASC
LIMIT 10;

Stable sqlite-vec 0.1.9 can perform the same exhaustive search internally:

SELECT email_rowid, distance
FROM email_vectors_v2
WHERE embedding MATCH ?
  AND k = 10
  AND searchable = 1;

MATCH remains exact and exhaustive in stable 0.1.9. Its speed comes from compact vec0 storage, SIMD distance calculations, and internal top-K selection—not reduced recall or an ANN index.

Prior Baseline (August 14, 2026)

The original study used an Apple M1 Max, SQLite 3.53.3, sqlite-vec v0.1.9, and 81,264 BGE-small 384-dimensional vectors. Three consecutive runs produced:

Retrieval path	Run 1	Run 2	Run 3
Scalar cosine + email join + SQL sort	1.9946 s	1.2083 s	1.4718 s
Existing vec0 MATCH (default L2 table)	0.0402 s	0.0290 s	0.0289 s

That comparison established the potential of MATCH, but the existing vec0 table defaulted to L2 and lacked filter metadata. The implementation therefore did not simply switch queries; it created a new explicitly cosine-configured index.

Implemented Architecture

The new search index is derived from the canonical embeddings in emails.embedding:

CREATE VIRTUAL TABLE email_vectors_v2 USING vec0(
    email_rowid INTEGER PRIMARY KEY,
    embedding FLOAT[384] distance_metric=cosine,
    searchable BOOLEAN,
    timestamp TEXT,
    from_address TEXT,
    is_outbound BOOLEAN,
    has_attachments BOOLEAN
);

Key decisions:

• Explicit cosine distance preserves the old retriever's metric.
• Integer email rowids avoid the documented cost of long text metadata keys.
• Quoted-reply exclusion and common exact filters execute inside KNN before top-K selection.
• Tag, sender-name substring, recipient substring, and relevance-cursor searches retain scalar cosine fallback because stable vec0 metadata does not support their LIKE predicates.
• Hybrid search still retrieves top 50 vector and FTS candidates and fuses them with RRF (k=60, vector weight 0.6, FTS weight 0.4). Only the vector candidate-generation implementation changed.
• Query-time BGE-small embedding generation runs on CPU. Bulk ingestion retains automatic MPS/CUDA/CPU selection and its separately benchmarked batch-size optimization.

Current Benchmark (August 22, 2026)

Environment

• Apple M1 Max
• macOS
• SQLite 3.53.4
• sqlite-vec v0.1.9
• 81,264 vectors
• BAAI/bge-small-en-v1.5, 384-dimensional float32 embeddings
• Migrated copy of the production database
• Five runs per query; warm result is the median of runs 2-5

Results

Query	First run	Warm median	Speedup vs scalar
Legacy scalar cosine, unfiltered	1.1087 s	1.1011 s	baseline
New exact cosine MATCH, unfiltered	0.0549 s	0.0505 s	21.8x
Legacy scalar cosine, timestamp >= 2020-01-01	3.3268 s	3.3515 s	baseline
New exact cosine MATCH, same date filter	0.1459 s	0.0960 s	34.9x

The earlier default-L2 MATCH table was faster (about 29 ms warm) than the final cosine table (about 50 ms warm). The final implementation intentionally accepts that difference to make cosine semantics explicit and robust across future model or embedding changes.

Correctness

• Unfiltered top-10 IDs: identical order.
• Date-filtered top-10 IDs: identical order.
• Separate top-50 comparison: 50/50 overlap and identical order.
• Adversarial unit test confirms the new table follows cosine rather than L2 ranking.
• Filter-before-KNN regression test confirms the nearest allowed result is returned even when the global nearest vector is excluded.
• Semantic, hybrid, similar-email, migration, device-selection, and MCP tests passed (78 selected tests in the final regression run).
• Production index validation found 81,264 embedded emails, 81,264 vectors, zero missing rows, and zero orphans.

Hybrid Search Compatibility

The hybrid strategy was chosen because keyword search is precise but brittle while semantic search catches paraphrases but can be noisy. RRF merges ranks instead of incompatible BM25 and cosine scores. That design is unchanged.

RRF depends on vector candidate membership and order, not on whether the vector retriever uses scalar SQL or vec0's internal top-K implementation. The new path preserves:

• Cosine metric
• Filter-before-ranking semantics
• Top-50 vector pool
• Independent FTS/BM25 pool
• RRF smoothing constant and weights
• fts_rank, vec_rank, and retrieval_method response fields

Unsupported filter forms deliberately remain on the old exact scalar path rather than filtering a global MATCH result afterward, which would lose valid candidates and alter RRF results.

Reproduction

Create the index on a database copy:

.venv/bin/python ingest.py --migrate-vector-index

Run the benchmark:

.venv/bin/python benchmarks/benchmark_vector_search.py db/emails.db --runs 5

Run focused validation:

.venv/bin/python -m pytest \
  tests/test_vec0_match.py \
  tests/test_query_extensions.py \
  tests/test_embedding_offline.py \
  tests/test_parallel_ingestion.py \
  tests/test_mcp_tools.py

.venv/bin/python tests/validate_issue4.py --verbose

Limitations

• Stable MATCH remains exhaustive and therefore still scales linearly with corpus size.
• Searches using tag substrings, partial sender names, recipient substrings, or the existing relevance cursor use scalar fallback and do not receive the MATCH speedup.
• The current semantic/hybrid timestamp cursor does not represent relevance order correctly; it remains a separate API concern.
• Full unified validation still reports the pre-existing Maildir/DB population mismatch (1,800 files versus 81,264 database records). Vector-specific validation passes; the mismatch is unrelated to the new index.
• ANN support exists only in sqlite-vec 0.1.10-alpha and was intentionally excluded from this production migration.