feat(core): parameterize quantizer bit-width (5/6-bit) — dominates residual mode on size AND speed

[ajianaz]

Motivation

The residual mode (#23, #38) buys recall at 2x storage (514→1028 B/vec) and +43% scan cost. Diagnostic (real EmbeddingGemma + faithful-port clustered set, release, aarch64) shows a plain higher-bit single-pass mode dominates it at these operating points:

Real EmbeddingGemma (n=2000+100, d=768, padded 1024)

mode	B/vec	compression	recall@1	recall@10	scan
plain 4-bit	514	5.98x	0.910	0.961	1.00x
plain 5-bit	642	4.79x	0.980	0.983	~1x
plain 6-bit	770	3.99x	0.990	0.984	~1x
4-bit+residual	1028	2.99x	1.000	0.991	1.43x

Clustered adversarial (d=384, vecq-bench generator port)

mode	B/vec	recall@1	recall@10
plain 4-bit	258	0.830	0.882
plain 5-bit	322	0.935	0.942
plain 6-bit	386	0.960	0.970
4-bit+residual	516	0.920	0.939

6-bit plain matches or beats residual recall on BOTH datasets at 25% less storage and no second-pass penalty. 5-bit is the sweet spot for size-sensitive profiles (r@10 0.983 real at ~half residual's size). Residual keeps one niche: perfect recall@1 on real data.

Proposal

• Add quantizer width to the index config: 4-bit (default, unchanged), 5-bit, 6-bit.
• Lloyd-Max tables for 32/64 levels: precomputed constants (same offline procedure as 4-bit).
• Packing: bit-packing (5/6 bits/code) replacing nibble-packing for the non-4-bit paths; format header gains a width field (v1.5, readers accept v1–v4).
• Scoring: LUT gather like the 4-bit NEON path, wider tables (32/64 entries → two/four 16-byte table segments + select); bit-identity tests scalar vs NEON as usual.
• Keep residual as-is (opt-in); document the operating points honestly in BENCHMARK.md: 5-bit for max compression-per-recall, 6-bit for recall ≈ residual at less storage, residual for max recall@1.

Acceptance

• Bit-identity across scalar/NEON for all widths.
• recall@10 within noise of the diagnostic numbers above on the real dataset test (real_residual_validation.rs extended per width).
• Plain 4-bit path untouched (bit-identical baseline re-verified per merge policy).

Numbers reproducible by re-running the same procedure: Lloyd iteration on a dense N(0,1) grid at 32/64 levels, real RHDH rotation, exact-norm residual reference (post-#38 estimator).

[ajianaz]

Design locked (green-light @ajianaz)

Width becomes an index config option; default = 5-bit (the sweet spot: 642 B/vec, 4.79x, recall@10 0.983 real).

width	role
4-bit	selectable, max compression (5.98x) — code path unchanged, bit-identical
5-bit	DEFAULT — compression/recall sweet spot
6-bit	opt-in, recall ≈ residual at 25% less storage
residual (opt-in, 4-bit base)	unchanged, max recall@1 niche

Consequences (accepted):

• README/BENCHMARK headline moves from "5.98x @ 0.961" to "4.79x @ 0.983" for the default profile; all numbers updated in the same commit per repo policy.
• Existing v1.4-and-older files are implicitly 4-bit and stay readable (format v1.5 carries an explicit width field).
• 5/6-bit ship with the scalar scan path first (NEON gather is a fast-follow); honest perf labeling until then.
• residual stays 4-bit-base only for now (width==4 assert).