Skip to content

memrust v0.5.2

Choose a tag to compare

@AIAnytime AIAnytime released this 27 Jul 17:50

SQ8 quantization is now the default for vectors ≥ 1024 dims.

At that width and above, quantized search measures as fast as — or faster than — f32, because memory bandwidth dominates the extra decode arithmetic. So the 4× memory saving comes for free. Measured on an M-series laptop (n=6k, clustered, memrust bench):

Dim f32 QPS SQ8 QPS f32 recall@10 SQ8 recall@10 Vectors
384 6,241 5,468 1.000 0.998 8.8 → 2.2 MB
1024 1,562 1,645 1.000 0.992 23.4 → 5.9 MB
1536 1,001 1,006 1.000 0.996 35.2 → 8.8 MB

Below 1024 dims f32 is faster and stays the default. Since the vector width isn't known until the first insert, the storage mode settles then (while the store is still empty) and is fixed for the life of the collection. Force it with --quantize / --no-quantize.

Checkpoints written by earlier versions still load — the old boolean quantize: false maps to Never, so an existing collection never silently changes representation.

pip install memrust      # Python SDK
memrust serve            # dashboard + HTTP API on :7700