Skip to content

DiskANN v0.55.0

Choose a tag to compare

@hildebrandmw hildebrandmw released this 10 Jul 15:05
924ecbf

Breaking changes since 0.54.0

Removed diskann-platform; decoupled diskann-providers (#1205)

The diskann-platform crate is deleted — its perf counters become a private
detail of Timer, and its io primitives (io_uring / Windows IOCP / handles)
move next to their only consumer, aligned_file_reader, in diskann-disk.
Timer relocates from diskann-providers to
diskann-disk::utils::instrumentation::Timer. diskann-providers loses its
opentelemetry / diskann-platform dependencies and its perf_test feature;
diskann-disk loses its unused virtual_storage feature and
Timer::{elapsed_seconds, elapsed_seconds_for_step}.

  • Upgrade: Drop any dependency on diskann-platform. Reference Timer at
    diskann_disk::utils::instrumentation::Timer (or use std::time::Instant
    directly, as the providers/tools call sites now do). Remove references to the
    diskann-providers perf_test feature and the diskann-disk
    virtual_storage feature.

Removed dead pub code; dropped diskann-toolsdiskann-disk dependency (#1185)

Removed the dead search_disk_index / build_disk_index modules in
diskann-tools, the unwired AssociatedDataFilter /
default_associated_data_filter in diskann-disk, and unreferenced
test_utils size constants. Ground-truth utilities and build_pq are no longer
generic over diskann-disk's GraphDataType.

  • Upgrade: Replace uses of the removed modules with the live disk build/search
    paths in diskann-benchmark, and the removed AssociatedDataFilter with the
    live VectorFilter / default_vector_filter. Where you relied on the
    GraphDataType generic, switch to a plain V: VectorRepr (plus an
    associated-data generic where needed).

FilteredAccessor introduced; QueryVisitDecision removed (#1141, #1192)

A new filter-aware FilteredAccessor trait replaces the pattern of passing a
QueryLabelProvider to every call site; multi-hop search is rewritten on top of
it. The SearchAccessor bound on SearchStrategy::SearchAccessor is relaxed to
let the disjoint FilteredAccessor participate. QueryVisitDecision is removed,
and QueryLabelProvider moves to diskann::graph::ext::labeled (as a
compatibility layer). Follow-up #1192
changes the expand_beam_accept_only bound from
P: HybridPredicate<Accept<Self::Id>> to
P: Predicate<Self::Id> + PredicateMut<Accept<Self::Id>>.

  • Upgrade: Implement/consume FilteredAccessor for filtered search; move
    QueryLabelProvider imports to diskann::graph::ext::labeled. Remove any use
    of QueryVisitDecision. Update expand_beam_accept_only predicate bounds to
    the split Predicate + PredicateMut<Accept<_>> form.

Disk-path filtered search: search() signatures gained adaptive_l (#1173)

DiskIndexSearcher::search / search_internal now take
adaptive_l: Option<AdaptiveL>. None preserves today's behavior; Some(_)
routes the graph path through the new filter_search() (backed by
InlineFilterSearch + the new PredicateLabelProvider adapting
&dyn Fn(&u32) -> bool to QueryLabelProvider<u32>).

  • Upgrade: Add the new adaptive_l argument at disk-search call sites (pass
    None for unfiltered search).

Flat index: SearchStrategy / DistancesUnordered trait changes (#1160)

FlatIndex::knn_search gains post-processing support via
diskann::glue::SearchPostProcess. With the VectorId scalar-conversion
constraint gone, DistancesUnordered now has HasId as a super-trait and
SearchStrategy no longer carries an Id associated type.

  • Upgrade: Add a HasId impl to your DistancesUnordered types and remove the
    Id associated type from SearchStrategy implementations; tie the flat
    index's ID type to the underlying DataProvider::InternalId.

Benchmark::try_match / description signatures changed (#1231)

try_match changes from
fn try_match(&self, input: &Self::Input) -> Result<MatchScore, FailureScore>
to fn try_match(&self, input: &Self::Input, context: &MatchContext) -> Score,
and description drops its input: Option<&Self::Input> argument.

  • Upgrade: Update Benchmark implementors to accumulate into the new Score
    type (recording a reason on every failure via MatchContext) and to the
    simplified description(&self, f) signature.

SampleableForStart now requires WithApproximateNorm (#1223)

WithApproximateNorm is now a super-trait of SampleableForStart (required by
StartPointStrategy::compute).

  • Upgrade: Ensure SampleableForStart implementors also implement
    WithApproximateNorm.

bf-tree upgraded 0.4 → 0.5; snapshot API migration (#1183)

Migrates to bf-tree 0.5's CPR snapshot API: snapshot*
cpr_snapshot(&path) / BfTree::new_from_cpr_snapshot(); save_bftree is no
longer async; BfTreeParams::to_config and copy_snapshot_if_needed are
removed. A new use_snapshot: bool is added to BfTreeProvider,
BfTreeProviderParameters, and SavedParams;
BfTreeStoreConfig::use_snapshot is required in benchmark JSON input
(SavedParams::use_snapshot defaults for backwards compatibility with existing
saved-param files).

  • Upgrade: Set use_snapshot in BfTreeStoreConfig JSON; migrate any direct
    snapshot calls to the CPR API; treat save_bftree as synchronous.

diskann-garnet FFI: inline filtering, bumped to 3.0.0 (#1162)

Garnet replaces its postfilter with inline filtering (~50x QPS improvement in
the PR's measurement) and is bumped to 3.0.0.

  • Upgrade: Garnet consumers must adopt the 3.0.0 FFI surface with inline
    filtering.

diskann-tools CLI standardized to kebab-case; range-search bin removed (#1217, #1213, #1212)

All diskann-tools command-line parameters now use kebab-case, variable names
are standardized, and the old (disabled) disk-index range-search binary is
removed in favor of the new range-groundtruth utility. The
vector_filters_file option is renamed to reflect that it takes a
pre-calculated bitmap.

  • Upgrade: Update scripts to the kebab-case flag names and the renamed
    bitmap-input option; use the new range-groundtruth utility instead of the
    removed binary.

Notable fixes & features (non-breaking)

  • Serialization: New foundational diskann-record crate for versioned
    save/load of DiskANN indexes (Disk + Memory backends, Save/Load traits,
    save_fields/load_fields macros) (#1188).
  • Distance functions: New ProjectedEigen distance for multi-vector re-ranking
    in diskann-quantization (#1203).
  • Diversity search: Determinant-diversity post-processing wired as an optional
    plugin for async full-precision topk and disk-index search, with new example
    configs (#1011).
  • Benchmarks: New end-to-end flat-index benchmark for full-precision in-memory
    search (#1170); consolidated
    Benchmark::try_match logic reduces implementor boilerplate (~700 lines)
    (#1231); directory checking
    added to diskann-benchmark-runner (unifies file/dir discovery)
    (#1227).
  • bftree concurrency: Striped RwLock on NeighborProvider eliminates lost
    edges under concurrent mutation (11–51% edge loss → 0), safe dual-store write
    ordering, and quant-store cleanup on delete (#1158).
  • bftree correctness: Validate record sizes at construction and force insert
    errors to be handled, fixing misleading "vector not found" errors on
    oversized records (#1166,
    #1194).
  • Benchmark recall: Fixed floating-point precision loss in the recall
    calculation (exact integer accumulation grouped by denominator)
    (#1171 / #1241).
  • Filtered search: Simplified the expand_beam_accept_only pre-filter
    predicate, restoring multi-hop filtered-search recall (#1192).