DiskANN v0.53.0

DiskANN v0.53.0 Release Notes

Breaking Changes

An AI generated, human reviewed list of changes is summarized below.

Paged search overhauled — channel-based API (#1078)

PagedSearchState and its 'static-bound pause/resume model have been replaced with an async, channel-based interface. The recommended way to drive paged search is now via a tokio::sync::mpsc channel, with the searcher embedded in an otherwise-'static future. See the rendered RFC for the new shape. Callers wired against PagedSearchState must migrate to the channel API.

Users of paged search via wrapped_async::DiskANNIndex that know their inner futures will never suspend can use the new wrapped_async::DiskANNIndex::paged_search_no_await; this will efficiently run paged searches with minimal synchronization overhead.

DiskANNIndex::flat_search removed (#1076)

DiskANNIndex::flat_search and the IdIterator trait have been removed from the diskann crate. Equivalent functionality lives on the new inherent method DiskIndexSearcher::flat_search in diskann-disk. This unblocks the experimental directions in #1067 and #983.

// Before
diskann_index.flat_search(query, ...)?;

// After
disk_index_searcher.flat_search(query, ...).await?;

DiskIndexSearcher::flat_search now batched (#1097)

The new DiskIndexSearcher::flat_search uses the bulk pq_distances path instead of one-vector-at-a-time Accessor::build_query_computer + evaluate_similarity. Downstream behavior is equivalent but tighter resource bounds apply.

centroid removed from PQ interfaces (#1010)

The dataset-centroid argument has been removed from FixedChunkPQTable construction, populate, and most other PQ APIs. The shift only ever worked for L2 distance and was silently ignored for inner-product / cosine, so passing it was a footgun. When an L2 shift is required, fold it into the PQ pivots instead (the library now does this internally).

// Before
let table = FixedChunkPQTable::new(.., centroid, ..);

// After — drop the centroid argument
let table = FixedChunkPQTable::new(.., ..);

Flat search interface (#983)

A new flat module in diskann adds a provider-agnostic brute-force search surface, mirroring the shape of graph search. Backends implement a single trait, DistancesUnordered<C> (in flat/strategy.rs), which fuses iteration and distance computation, allowing any backend (in-memory, quantized, disk, remote) to plug into a shared algorithm. See the rendered RFC. This is additive but is the new canonical surface — direct ad-hoc flat-search call sites should migrate.

bf_tree extracted into diskann-bftree crate (#1020)

The bf_tree provider has been moved out of diskann-providers (previously at diskann-providers/src/model/graph/provider/async_/bf_tree/) into a new standalone diskann-bftree crate. Along with the move:

• Switched from PQ to spherical quantization.
• Dropped dependencies on DeletionCheck, AsDeletionCheck, and RemoveDeletedIdsAndCopy.
• Simplified generics.

Consumers must update their Cargo.toml to depend on diskann-bftree and update import paths.

direct_distance_impl and inner_product_raw re-exposed (#1081)

direct_distance_impl (free function) and FixedChunkPQTable::inner_product_raw are pub again after being privatized in #1044. Restored to unblock a downstream user. Not breaking in the typical direction — this restores previously available API surface.

MinMax recompress takes a grid-scale parameter (#1109)

The MinMax recompress API now accepts a grid-scale parameter.

New Features

• SIMD-optimized L2-squared norm (#1107)
• Significantly faster bitmap computation (#1099)
• Large speedup on the bitmap construction path used by filtered search.
• LLVM IR bloat regression check in CI (#1083)
• CI now flags regressions in generated LLVM IR size, helping catch unintended monomorphization blow-ups.
• Recall computation fix for under-k groundtruth (#1069)

Full List of Changes

• [CI] Try to fix publishing step by @hildebrandmw in #1057
• Revise README for DiskANN3 by @harsha-simhadri in #1046
• [benchmark] Remove DispatchRule by @hildebrandmw in #1064
• [benchmark] Automatic Input Registration by @hildebrandmw in #1066
• Remove centroid from most PQ interfaces. by @hildebrandmw in #1010
• [diskann/disk] Remove flat_search from DiskANNIndex by @hildebrandmw in #1076
• macos build and miri check to nightly by @harsha-simhadri in #1058
• [API] Make some methods public again. by @hildebrandmw in #1081
• [benchmark] Simply Inputs more. by @hildebrandmw in #1077
• Turn on stack protection for the diskann-garnet NuGet build by @metajack in #1082
• Fix options for diskann-garnet nuget pipeline by @metajack in #1091
• [CI] add LLVM IR bloat regression check by @arrayka in #1083
• Bump openssl from 0.10.79 to 0.10.80 by @dependabot[bot] in #1093
• [Disk CI benchmarks] Use 1ES.Pool=diskann-github by @arrayka in #869
• Fix recall computation for fewer than k groundtruth results by @magdalendobson in #1069
• bf_tree migration away from diskann-providers by @JordanMaples in #1020
• [RFC/diskann] Overhaul paged search by @hildebrandmw in #1078
• Remove unsafe code from compute_vec_l2sq by @arrayka in #1094
• Remove direct accessor call in diskann-garnet by @hildebrandmw in #1098
• Refactor DiskIndexSearcher::flat_search to use batching by @hildebrandmw in #1097
• [flat index] Flat Search Interface by @arkrishn94 in #983
• migrating multi-hop tests from diskann-providers to diskann by @JordanMaples in #928
• Significantly speed up bitmap computation by @magdalendobson in #1099
• compute_vecs_l2sq: Replace scalar L2 Squared norm with SIMD-optimized FastL2NormSquared by @arrayka in #1107
• [minmax] Add grid scaling to recompress API by @arkrishn94 in #1109
• Bump version to 0.53.0 by @arkrishn94 in #1111

New Contributors

• @magdalendobson made their first contribution in #1069

Full Changelog: v0.52.0...v0.53.0