v1.0.0 — Stable API

iqdb-quantize v1.0.0 — Stable

The quantization layer is stable. v1.0.0 commits the public API of iqdb-quantize under SemVer for the entire 1.x series: no breaking changes until 2.0. The three schemes every iQDB deployment dials between — scalar, product, binary, behind one Quantizer trait — are now a fixed point the index crates can build on without churn. Nothing in the public surface changed since 0.5.0; this release adds the consumer-simulation soak, runnable examples, and the stability commitment.

What is iqdb-quantize?

The memory-efficiency layer of the iQDB vector database. It compresses f32 embedding vectors into compact codes that preserve similarity-search quality — a million 768-dim vectors drop from ~3 GB to as little as ~96 MB. Three schemes share one Quantizer trait: scalar (SQ8, ~4×, every metric), product (PQ, up to ~192×, with batch-ADC scoring for IVF-PQ), and binary (BQ, ~32×, Hamming). Distance is asymmetric — the database is compressed, the query stays f32 — and the recommended quality path is to search quantized, then rerank the shortlist with full precision.

use iqdb_quantize::{Quantizer, ScalarQuantizer};
use iqdb_types::DistanceMetric;

let mut sq = ScalarQuantizer::new();
sq.train(&[&[0.0_f32, 1.0, 2.0][..], &[1.0_f32, 0.0, 1.0][..]]).unwrap();

let code = sq.quantize(&[0.5_f32, 0.5, 1.5]).unwrap();       // 3 bytes from 12
let d = sq.distance(&[0.5_f32, 0.5, 1.5], &code, DistanceMetric::Cosine).unwrap();
assert!(d.is_finite());

Why 1.0 now

The road here was deliberate, one verified phase at a time:

• 0.2.0 — Scalar quantization (SQ8) + the Quantizer trait. The first scheme behind the trait every quantizer implements, with per-dimension affine calibration and asymmetric distance through iqdb-distance.
• 0.3.0 — Product quantization. k-means codebooks per subvector and the PqAdcTables batch-ADC primitive — the path IVF-PQ scores through.
• 0.4.0 — Binary quantization + feature freeze. The third scheme and the declaration that the public surface is complete.
• 0.5.0 — Recall validation + API freeze. End-to-end recall measured against full-f32 baselines, tracing instrumentation, the criterion bench harness, and the surface locked.

By 0.5.0 every Definition-of-Done criterion was met. The 0.6–0.9 RC phase exists to soak the API against live consumers; under the spine-first ordering the real consumer (iqdb-ivf) is not yet published against this surface, so its intent was met by a consumer-simulation built at the exact shape IVF-PQ uses. The crate proceeds to 1.0 on a fully-satisfied checklist rather than a calendar — the same path iqdb-types and iqdb-distance took.

What 1.0.0 adds

Consumer-simulation soak

tests/consumer_simulation.rs is a mini IVF-PQ index built only on the public surface: it partitions a corpus into coarse clusters, stores each member as a PqCode, builds the ADC tables once per query with build_query_tables, and scans the probed clusters through PqAdcTables::distance — exactly the intra-cluster scan iqdb-ivf performs. It asserts the contracts the consumer relies on:

• Batch ADC equals the single-shot path bit-for-bit, for every code and every supported metric. If PqAdcTables::distance ever drifted from ProductQuantizer::distance, this fails.
• The index recovers the right neighbourhood — PQ cluster purity ≥ 0.9, and a PQ-shortlist + f32-rerank recovers the exact top-10 (overlap ≥ 0.9); an SQ8 flat index preserves the exact top-10 directly (≥ 0.9).
• The boundary is safe — foreign code shapes and unsupported metrics return typed errors, never panic.

Runnable examples

Five documented examples (cargo run --example <name>):

• scalar_quantization — SQ8 train / quantize / asymmetric distance / decode.
• product_quantization — PQ with build_query_tables batch scoring.
• binary_quantization — BQ and the Hamming-only contract.
• rerank — the search-quantized-then-rerank quality path, matching the exact answer.
• compression — the three schemes' code sizes side by side.

The 1.x compatibility promise

• The public surface recorded in dev/ROADMAP.md is frozen until 2.0: the Quantizer trait, the three quantizers (ScalarQuantizer, BinaryQuantizer, ProductQuantizer), the three code types (Sq8Code, BqCode, PqCode), the PqAdcTables batch-ADC primitive, and VERSION.
• Additive, non-breaking changes remain allowed within 1.x.
• A call with a DistanceMetric a scheme does not support returns IqdbError::InvalidMetric rather than panicking — the forward-compatible handling for the #[non_exhaustive] enum.

Breaking changes

None. No public API changed from 0.5.0; 1.0.0 is the SemVer stability commitment.

Performance

Compression is exact and deterministic — the same seed + data yield byte-identical PQ codes on every platform:

Scheme	Code (768-dim)	Compression	Metrics
SQ8	768 bytes	4×	every metric
BQ	96 bytes	32×	Hamming
PQ (M = 16)	16 bytes	192×	Euclidean / DotProduct / Manhattan

Per-vector throughput, benchmarked on Windows x86_64 at 768 dims (criterion medians):

Operation	Median
SQ8 quantize	~1.56 µs
SQ8 asymmetric Cosine distance	~0.95 µs
BQ quantize	~0.64 µs
BQ Hamming distance	~0.68 µs

The f32 distance SQ8 and PQ delegate to is SIMD-accelerated transparently through iqdb-distance (AVX2 / NEON), so quantized search inherits those kernels without a feature flag here.

Verification

cargo fmt --all -- --check
cargo clippy --all-targets -- -D warnings
cargo clippy --all-targets --all-features -- -D warnings
cargo test --all-features
RUSTDOCFLAGS="-D warnings" cargo doc --no-deps --all-features
cargo deny check
cargo audit

All green. 119 tests across unit, edge-case, property (proptest), recall, determinism, tracing, smoke, consumer-simulation, and doctest suites, plus five runnable examples. cargo deny check and cargo audit are clean. There is zero unsafe in the crate. The loom Definition-of-Done item is N/A by design — the quantizers own their calibration by value with no interior mutability and no lock-free path, so there is no concurrent protocol to model (recorded in dev/ROADMAP.md).

What's next

iqdb-quantize is done. Its consumer builds on it: iqdb-ivf scores in-cluster codes through this stable 1.x surface (PqAdcTables). Anything that consumer surfaces as a genuinely useful addition will be an additive 1.0.x release.

Installation

[dependencies]
iqdb-quantize = "1.0"

MSRV: Rust 1.87.

Documentation

• README
• API reference
• ROADMAP (frozen API record)
• Standards (REPS)
• CHANGELOG
• docs.rs/iqdb-quantize

---

Full diff: v0.5.0...v1.0.0.
Changelog: CHANGELOG.md.

v0.5.0 — Recall Validation + API Freeze

iqdb-quantize v0.5.0 — Recall Validation + API Freeze

Measured, instrumented, locked. v0.5.0 closes the 0.x design series. The three schemes from 0.2–0.4 are now validated end to end against full-precision baselines on synthetic corpora, the training paths emit structured tracing events, a criterion bench harness tracks the hot paths, and the public surface is frozen for the 1.x series.

What is iqdb-quantize?

The memory-efficiency layer of the iQDB vector database — scalar (SQ8), product (PQ), and binary (BQ) quantization behind one Quantizer trait, with distance computed directly on the compressed form and full-precision re-ranking as the recommended quality path.

What's new in 0.5.0

Recall validation against full-f32 baselines

A recall suite (tests/recall.rs) runs each scheme over Gaussian-cluster synthetic data and compares against the full-precision baseline:

• SQ8 — top-10 index overlap ≥ 0.9.
• BQ — top-10 cluster purity ≥ 0.7.
• PQ — top-k overlap against the Euclidean f32 baseline.

Thresholds are taken from values actually measured on the seeded corpus, with margin — they assert the schemes preserve search quality, not an aspirational target.

Tracing instrumentation

Training boundaries across all three quantizers are instrumented at info level (per-vector encode/decode/distance stays hot-path only). When a fallible training call fails it emits a structured error event carrying error-forge's kind() and caption(). tests/tracing.rs verifies the events and fields flow through using an inlined recording subscriber, so nothing is asserted by installing a global one.

use iqdb_quantize::{Quantizer, ScalarQuantizer};
use iqdb_types::IqdbError;

// A failed train still returns a typed error (and emits a tracing event) — never a panic.
let mut sq = ScalarQuantizer::new();
let err = sq.train(&[]).unwrap_err();
assert!(matches!(err, IqdbError::InvalidConfig { .. }));

Criterion bench harness

benches/quantize.rs tracks SQ8 quantize, SQ8 asymmetric distance versus raw f32 distance, BQ quantize, and BQ Hamming throughput, so any hot-path regression shows up against a recorded baseline.

API freeze

The public surface declared complete at the 0.4.0 feature freeze is now locked for the 1.x series — only additive, non-breaking changes land before 2.0. The frozen item list is recorded in dev/ROADMAP.md. cargo audit and cargo deny check are clean.

Breaking changes

Pre-1.0 API churn. This release adds the recall suite, tracing test, and bench harness; the public surface is unchanged from 0.4.0.

Verification

The full suite — unit, edge-case, property, recall, determinism, tracing, and doctests — passes on the CI matrix (Linux, macOS, Windows) on stable and the 1.87 MSRV:

cargo fmt --all -- --check
cargo clippy --all-targets --all-features -- -D warnings
cargo test --all-features
RUSTDOCFLAGS="-D warnings" cargo doc --no-deps --all-features

MSRV: Rust 1.87.

What's next

• v0.6.0 → v0.9.x — integration + RC. Exercise the surface against its real consumer, iqdb-ivf (IVF-PQ scores in-cluster codes through PqAdcTables), settle final benchmarks, and polish docs ahead of a stable v1.0.0.

Installation

[dependencies]
iqdb-quantize = "0.5"

Documentation

• README
• API reference
• ROADMAP
• Standards (REPS)
• CHANGELOG

---

Changelog: CHANGELOG.md.