DiskANN v0.61.0

DiskANN v0.61.0

Highlights

• Quantization: Added 3-bit spherical quantization with AVX2 acceleration, faster
  spherical encoding, and NEON float32 MinMax kernels.
• In-memory indexing (experimental): Added product quantization and 1-8-bit scalar
  quantization, unified quantized representations, and AArch64 prefetching.
• Multi-vector search: Added i8 x i8 and MinMax8 x MinMax4 MaxSim kernels.
• Fixes: Preserve virtual start-point out-edges when writing disk indexes, correctly
  round-trip scalar quantization configurations without standard deviation, and support
  zero-column mutable matrix iterators.

Breaking changes and upgrading

Changes are concentrated in quantization APIs and experimental/tooling crates.

• PQ training (#1467) - Training now returns BasicTable. SimplePivots was removed.
  Replace flatten() with view_pivots().as_slice() and use pivots_for(chunk) for
  per-chunk access.
• PQ distance tables (#1458) - Change ProcessInto bounds to ProcessInto<f32> and calls
  from process_into::<Metric>(...) to process_into::<Metric, f32>(...).
• MaxSim kernels (#1429) - Generic kernel/erasure code now requires T: MaxSimElement and
  score buffers of T::Score (f32 for floats, i32 for i8). Update build_max_sim
  error handling from NotSupported to BuildMaxSimError.
• diskann-inmem (#1485) - Replace repr::Spherical with repr::Quantized. Wrap the
  quantizer in repr::quantized::spherical::Spherical::new(...) when configuring it.
  Configuration and Rerank types now live under repr::quantized.
• diskann-benchmark-runner (#1475) - input::Registered and public Registry::input were
  removed. Use the concrete Input implementation's tag() and example() methods instead.

All Changes

• Fix serialization round trip for QuantizationType::SQ without standard deviation by Alex Razumov (@arrayka) in #1461
• i8 x i8 MaxSim Kernels by Suryansh Gupta (@suri-kumkaran) in #1429
• Adding subview_mut to MatrixMut by Aditya Krishnan (@arkrishn94) in #1463
• Add InsertResult::SuccessUpdate by Jack Moffitt (@metajack) in #1449
• [diskann-quantization] PQ Infrastructure for diskann-inmem by Mark Hildebrand (@hildebrandmw) in #1458
• [diskann-quantization] Teach PQ training to return a BasicTable. by Mark Hildebrand (@hildebrandmw) in #1467
• [deps] Run cargo-update by Mark Hildebrand (@hildebrandmw) in #1452
• Support zero-column mutable matrix iterators by Alex Razumov (@arrayka) in #1462
• [diskann-quantization] Use f32::NAN instead of core::f32::NAN. by Mark Hildebrand (@hildebrandmw) in #1471
• [diskann-wide] Add V4::new_checked_miri_uncached. by Mark Hildebrand (@hildebrandmw) in #1480
• [diskann-wide] Add maddubs intrinsic wrappers to V3 and V4 by juchen-ms (@partychen) in #1479
• Bump the github-actions group across 1 directory with 5 updates by Dependabot (@dependabot)[bot] in #1482
• [benchmark-runner] Simplify Input representation by Mark Hildebrand (@hildebrandmw) in #1475
• [diskann-quantization] Optimize spherical encoding scale search by xinyuwen2 in #1478
• [diskann-garnet] Implement XVCREATE, XVIMPORT by Jack Moffitt (@metajack) in #1455
• [diskann-garnet] Support configurable start points and allow importing start points by Jack Moffitt (@metajack) in #1464
• [diskann-garnet] Add spherical 2-bit and 4-bit quantization by Jack Moffitt (@metajack) in #1465
• [deps] Address depandabot issue 32 by Mark Hildebrand (@hildebrandmw) in #1496
• [diskann-disk] Preserve virtual start point out-edges when writing the disk index by Dongliang Wu (@doliawu) in #1483
• [diskann-quantization] Add Neon F32 Minmax Kernels by pfoxARM in #1431
• [diskann-garnet] Add macOS arm64 support to NuGet publishing by Jack Moffitt (@metajack) in #1495
• [diskann-disk] Require start points at index initialization. by Mark Hildebrand (@hildebrandmw) in #1497
• Expose metric-aware quantizer centroid distance by Namrata Mohanty (@cherryrhea) in #1395
• [diskann-inmem] Add support for Product Quantization by Mark Hildebrand (@hildebrandmw) in #1469
• Adds aarch64 prefetching into diskann-vector and diskann-inmem by pfoxARM in #1408
• [diskann-inmem] Add scalar quantization by Mark Hildebrand (@hildebrandmw) in #1484
• Bump the github-actions group with 3 updates by Dependabot (@dependabot)[bot] in #1505
• [diskann-inmem] Spherical Quantization Round 2 by Mark Hildebrand (@hildebrandmw) in #1485
• [diskann-quantization] Add SphericalQ 3-bit support and AVX2 kernel by xinyuwen2 in #1474
• [muilt-vetor] Add standalone MinMax8 x MinMax4 MaxSim kernel by juchen-ms (@partychen) in #1494

New Contributors

• Namrata Mohanty (@cherryrhea) made their first contribution in #1395

Full Changelog: v0.60.0...v0.61.0