Skip to content

v0.5.0 — Recall Validation + API Freeze

Pre-release
Pre-release

Choose a tag to compare

View all tags
@jamesgober jamesgober released this 06 Jun 05:03
· 1 commit to main since this release
v0.5.0
66f6c20

iqdb-quantize v0.5.0 — Recall Validation + API Freeze

Measured, instrumented, locked. v0.5.0 closes the 0.x design series. The three schemes from 0.2–0.4 are now validated end to end against full-precision baselines on synthetic corpora, the training paths emit structured tracing events, a criterion bench harness tracks the hot paths, and the public surface is frozen for the 1.x series.

What is iqdb-quantize?

The memory-efficiency layer of the iQDB vector database — scalar (SQ8), product (PQ), and binary (BQ) quantization behind one Quantizer trait, with distance computed directly on the compressed form and full-precision re-ranking as the recommended quality path.

What's new in 0.5.0

Recall validation against full-f32 baselines

A recall suite (tests/recall.rs) runs each scheme over Gaussian-cluster synthetic data and compares against the full-precision baseline:

  • SQ8 — top-10 index overlap ≥ 0.9.
  • BQ — top-10 cluster purity ≥ 0.7.
  • PQ — top-k overlap against the Euclidean f32 baseline.

Thresholds are taken from values actually measured on the seeded corpus, with margin — they assert the schemes preserve search quality, not an aspirational target.

Tracing instrumentation

Training boundaries across all three quantizers are instrumented at info level (per-vector encode/decode/distance stays hot-path only). When a fallible training call fails it emits a structured error event carrying error-forge's kind() and caption(). tests/tracing.rs verifies the events and fields flow through using an inlined recording subscriber, so nothing is asserted by installing a global one.

use iqdb_quantize::{Quantizer, ScalarQuantizer};
use iqdb_types::IqdbError;

// A failed train still returns a typed error (and emits a tracing event) — never a panic.
let mut sq = ScalarQuantizer::new();
let err = sq.train(&[]).unwrap_err();
assert!(matches!(err, IqdbError::InvalidConfig { .. }));

Criterion bench harness

benches/quantize.rs tracks SQ8 quantize, SQ8 asymmetric distance versus raw f32 distance, BQ quantize, and BQ Hamming throughput, so any hot-path regression shows up against a recorded baseline.

API freeze

The public surface declared complete at the 0.4.0 feature freeze is now locked for the 1.x series — only additive, non-breaking changes land before 2.0. The frozen item list is recorded in dev/ROADMAP.md. cargo audit and cargo deny check are clean.

Breaking changes

Pre-1.0 API churn. This release adds the recall suite, tracing test, and bench harness; the public surface is unchanged from 0.4.0.

Verification

The full suite — unit, edge-case, property, recall, determinism, tracing, and doctests — passes on the CI matrix (Linux, macOS, Windows) on stable and the 1.87 MSRV:

cargo fmt --all -- --check
cargo clippy --all-targets --all-features -- -D warnings
cargo test --all-features
RUSTDOCFLAGS="-D warnings" cargo doc --no-deps --all-features

MSRV: Rust 1.87.

What's next

  • v0.6.0 → v0.9.x — integration + RC. Exercise the surface against its real consumer, iqdb-ivf (IVF-PQ scores in-cluster codes through PqAdcTables), settle final benchmarks, and polish docs ahead of a stable v1.0.0.

Installation

[dependencies]
iqdb-quantize = "0.5"

Documentation

Changelog: CHANGELOG.md.

Assets 2
Loading