memrust v0.5.2
SQ8 quantization is now the default for vectors ≥ 1024 dims.
At that width and above, quantized search measures as fast as — or faster than — f32, because memory bandwidth dominates the extra decode arithmetic. So the 4× memory saving comes for free. Measured on an M-series laptop (n=6k, clustered, memrust bench):
| Dim | f32 QPS | SQ8 QPS | f32 recall@10 | SQ8 recall@10 | Vectors |
|---|---|---|---|---|---|
| 384 | 6,241 | 5,468 | 1.000 | 0.998 | 8.8 → 2.2 MB |
| 1024 | 1,562 | 1,645 | 1.000 | 0.992 | 23.4 → 5.9 MB |
| 1536 | 1,001 | 1,006 | 1.000 | 0.996 | 35.2 → 8.8 MB |
Below 1024 dims f32 is faster and stays the default. Since the vector width isn't known until the first insert, the storage mode settles then (while the store is still empty) and is fixed for the life of the collection. Force it with --quantize / --no-quantize.
Checkpoints written by earlier versions still load — the old boolean quantize: false maps to Never, so an existing collection never silently changes representation.
pip install memrust # Python SDK
memrust serve # dashboard + HTTP API on :7700