qvcache-go

Go implementation of QVCache — a query-vector cache for approximate nearest neighbor (ANN) search. Sits in front of any vector database, exploits temporal-semantic query locality to cut tail latency and increase throughput while preserving recall.

This is a Go port of the C++ reference implementation that accompanies the paper. Same algorithm, same on-disk formats, same benchmark workload generation — built so the cache can be embedded in Go services that already speak to a vector database directly.

⚠️ Disclaimer ⚠️

This repository is an independent, exploratory Go reimplementation of ideas presented in QVCache: … (arXiv:2602.02057)

The design and core techniques implemented here belong to the paper's authors — this project does not claim any original research contribution. It exists to help me (and hopefully others) understand the paper by working through an implementation in Go.

This is not the reference implementation. The authors' own implementation is at qvcache/qvcache-vldb26 and is the source of truth for correctness, performance, and fidelity to the paper.

This is not production code. This code was written for exploration and learning.

If you're the paper's author and have any concerns about this repo please get in touch.

Background

Original paper: QVCache: … (arXiv:2602.02057)

Reference C++ implementation: iangregson/qvcache-vldb26 (fork of the authors' artifact)

The cache works by maintaining a small in-memory index (Vamana mini-indexes with LRU eviction) of recently-fetched result vectors. For each incoming query, it computes a regional theta threshold (PCA-bucketed), checks whether the cached top-K results are "good enough" (cache_kth_dist ≤ (1+D)·θ), and only falls through to the backend on a miss. See paper §3-§4 for the full design.

Status

• All paper-faithful metrics implemented: regional theta with EMA updates, PCA-projected region keys, four-way mini-index LRU rotation, async insert pool, four search strategies (stop_first_hit, all, adaptive, parallel).
• Backends shipped: brute force, Vamana (in-memory), Weaviate.
• Validated end-to-end against the C++ reference over a shared Weaviate backend (3 datasets × 2 modes). Recall preservation matches the paper's reported 2-5% drop; steady-state cache speedup is 7-14× over the backend. See Benchmarks and PLAN.md for the optimization write-up.

Install

go install github.com/iangregson/qvcache-go/cmd/benchmark@latest

Or as a library:

go get github.com/iangregson/qvcache-go

Quickstart — using the cache in code

import (
    "github.com/iangregson/qvcache-go"
    "github.com/iangregson/qvcache-go/backends"
)

// 1. Connect to the underlying vector DB.
backend, err := backends.NewWeaviateBackend(ctx, backends.WeaviateConfig{
    Host:       "localhost",
    HTTPPort:   8080,
    GRPCPort:   50051,
    Collection: "MyVectors",
    Metric:     qvcache.MetricL2,
})
if err != nil {
    log.Fatal(err)
}

// 2. Wrap it in QVCache.
cache := qvcache.NewQVCache(qvcache.Config{
    Dim:                  128,
    R:                    64,
    MemoryL:              128,
    Alpha:                1.2,
    P:                    0.9,
    DeviationFactor:      0.25,
    MemoryIndexMaxPoints: 100_000,
    PCADim:               16,
    BucketsPerDim:        8,
    NumberOfMiniIndexes:  4,
    PCASampleSize:        10_000,
    SearchStrategy:       qvcache.StrategyAdaptive,
    Metric:               qvcache.MetricL2,
}, backend)

// 3. Query.
tags, dists, hit := cache.Search(query, 10)

The cache invariants: same input → same output as a direct backend call, but with sub-ms p50 hit latency and ~5-15× QPS speedup once warm.

Backends

backend	use case
backends.BruteForceBackend	small in-memory base set, exact NN
backends.VamanaBackend	DiskANN-style graph, in-memory
backends.WeaviateBackend	external Weaviate instance over gRPC

Implementing a new backend is ~50 lines: implement qvcache.Backend (Search, FetchVectorsByIDs, NumVectors, Dim).

Build & test

go vet ./...
go test ./...
go test -bench=. ./...

Run the benchmark CLI

The benchmark harness produces JSONL window_metrics records suitable for the same analysis pipeline as the C++ reference:

go run ./cmd/benchmark \
    --base-path  data/sift-128-euclidean/sift-128-euclidean_base.bin \
    --query-path data/sift-128-euclidean/sift-128-euclidean_query.bin \
    --gt-path    data/sift-128-euclidean/sift-128-euclidean_groundtruth.bin \
    --metric l2 --k 10 \
    --backend weaviate --weaviate-collection Sift128 \
    --R 64 --memory-L 128 --alpha 1.2 --p 0.9 --deviation-factor 0.25 \
    --memory-index-max-points 100000 --pca-dim 16 --buckets-per-dim 8 \
    --number-of-mini-indexes 4 --pca-sample-size 10000 \
    --search-strategy adaptive \
    --window-size 4 --n-repeat 3 --stride 1 --n-round 1 \
    --n-split 10 --n-split-repeat 20

Each window emits one JSON line with qps, recall_all, hit_ratio, tail_latency_ms{p50,p90,p95,p99}, memory_active_vectors, pca_active_regions, etc.

Benchmarks

Over a shared Weaviate backend, using the paper's exact noisy workload (n_split=10, n_split_repeat=20, window=4, n_repeat=3, stride=1, noise_ratio=0.01) and per-dataset deviation factor (0.25 for SIFT/GloVe-as-L2, 0.075 for GIST). Numbers are steady-state (window ≥ 3, repeat > 0) means on an Apple M3 Max, after the optimization pass described in PLAN.md (search_strategy=parallel).

Absolute numbers (warm steady-state)

dataset	QPS	recall	p50 (ms)	speedup vs backend
SIFT-128	4300	0.9834	0.23	7.4×
GIST-960	3084	0.9514	0.33	14.1×
GloVe-100	3613	0.9532	0.27	9.7×

speedup vs backend = qvcache QPS ÷ Weaviate-direct QPS, i.e. the gain from the cache. P50 hit latency is sub-millisecond on every dataset.

Recall delta (qvcache − backend)

dataset	Go
SIFT-128	-0.016
GIST-960	-0.024
GloVe-100	-0.006

All within the paper's reported 2-5% drop range. The cache returns the right vectors — recall is gated on the merged top-k across all mini-indexes, so a cache hit's result is the best of everything cached.

P50 latency reduction (backend ÷ qvcache)

dataset	Go
SIFT-128	7.5×
GIST-960	14.0×
GloVe-100	10.0×

vs the C++ reference

The within-implementation speedup ratio (cache vs own backend) now matches the C++ reference on SIFT/GloVe (cpp 7.9× / 9.8×) and exceeds it on GIST (cpp 5.5×), and beats the Rust port on all three. Absolute cross-impl QPS isn't hardware-comparable: in the harness the C++ reference runs QEMU-emulated (linux/amd64 on Apple Silicon) while Go and Rust run native arm64. The full 3-impl × 3-dataset × 2-mode matrix, methodology, and caveats live at iangregson/qvcache-exploration under analysis/ (see analysis/summary.md).

Takeaway: for services already in Go, embedding qvcache-go directly avoids cgo + a separate process and delivers a 7–14× throughput gain over hitting the vector DB on every query, with sub-ms p50 hit latency and recall within the paper's range.

Conventions

• Float32 throughout (consistent with reference impl)
• Vectors are flat: vector i occupies data[i*dim:(i+1)*dim]
• Tags are 1-based on output (matches reference convention; subtract 1 for 0-based groundtruth comparison)
• Errors returned for I/O; panics for human errors (dimension mismatches)

Citation

Original paper:

@article{qvcache2026,
  title   = {QVCache: A Query-Aware Vector Cache},
  author  = {Anıl Eren Göçer, Ioanna Tsakalidou, Hamish Nicholson, Kyoungmin Kim, Anastasia Ailamaki},
  journal = {arXiv preprint arXiv:2602.02057},
  year    = {2026},
  url     = {https://arxiv.org/pdf/2602.02057}
}

Acknowledgements

• The QVCache paper authors for the algorithm and reference implementation.
• The reference C++ impl at qvcache/qvcache-vldb26 (forked with Weaviate backend at iangregson/qvcache-vldb26).
• Claude Code