Skip to content

DiskANN v0.47.0

Choose a tag to compare

@hildebrandmw hildebrandmw released this 23 Feb 17:51
· 167 commits to main since this release
9ac3e08

DiskANN v0.47.0

Summary

  • This version contains a major breaking change to the search interface of DiskANNIndex. Please read the upgrade instructions below.
  • An Aarch64 Neon has been added to diskann-wide.
  • Various bug-fixes and code-quality improvements.

Changes to Search

The search interface has been unified around a single index.search() entry point using the Search trait.
The old per-search-type methods on DiskANNIndex (search, search_recorded, range_search, multihop_search) have been removed and replaced by typed parameter structs that carry their own search logic.

What Changed

Removed Replacement
SearchParams struct diskann::graph::search::Knn
RangeSearchParams struct diskann::graph::search::Range
SearchParamsError diskann::graph::KnnSearchError
RangeSearchParamsError diskann::graph::RangeSearchError
index.search(&strategy, &ctx, &query, &params, &mut out) index.search(knn, &strategy, &ctx, &query, &mut out)
index.search_recorded(..., &mut recorder) index.search(RecordedKnn::new(knn, &mut recorder), ...)
index.range_search(&strategy, &ctx, &query, &params) index.search(range, &strategy, &ctx, &query, &mut ())
index.multihop_search(..., &label_eval) index.search(MultihopSearch::new(knn, &label_eval), ...)
index.diverse_search(...) index.search(Diverse::new(knn, diverse_params), ...)

flat_search remains an inherent method on DiskANNIndex
Its search_params argument changed from &SearchParams to &Knn.

Upgrade Instructions

1. k-NN Search (search)

Before:

use diskann::graph::SearchParams;

let params = SearchParams::new(10, 100, None)?;
let stats = index.search(&strategy, &ctx, &query, &params, &mut output).await?;

After:

use diskann::graph::{Search, search::Knn};

let params = Knn::new(10, 100, None)?;
// Note: params is now the FIRST argument (moved before strategy)
let stats = index.search(params, &strategy, &ctx, &query, &mut output).await?;

Key differences:

  • SearchParams -> Knn (import from diskann::graph::search::Knn)
  • SearchParamsError -> KnnSearchError (import from diskann::graph::KnnSearchError)
  • Search params moved to the first argument of index.search()
  • k_value, l_value fields are now private; use .k_value(), .l_value() accessors (return NonZeroUsize)

2. Recorded/Debug Search (search_recorded)

Before:

use diskann::graph::SearchParams;

let params = SearchParams::new(10, 100, None)?;
let stats = index
   .search_recorded(&strategy, &ctx, &query, &params, &mut output, &mut recorder)
   .await?;

After:

use diskann::graph::{Search, search::{Knn, RecordedKnn}};

let params = Knn::new(10, 100, None)?;
let recorded = RecordedKnn::new(params, &mut recorder);
let stats = index.search(recorded, &strategy, &ctx, &query, &mut output).await?;

3. Range Search (range_search)

Before:

use diskann::graph::RangeSearchParams;

let params = RangeSearchParams::new(None, 100, None, 0.5, None, 1.0, 1.0)?;
let (stats, ids, distances) = index
   .range_search(&strategy, &ctx, &query, &params)
   .await?;

After:

use diskann::graph::{
    Search,
    search::Range,
    RangeSearchOutput,
};

// Simple form:
let params = Range::new(100, 0.5)?;
// Or full options form:
let params = Range::with_options(None, 100, None, 0.5, None, 1.0, 1.0)?;

// Note: output buffer is `&mut ()` — results come back in the return type
let result: RangeSearchOutput<_> = index
   .search(params, &strategy, &ctx, &query, &mut ())
   .await?;

// Access results:
let stats = result.stats;
let ids = result.ids;           // Vec<O>
let distances = result.distances; // Vec<f32>

Key differences:

  • RangeSearchParams -> Range (import from diskann::graph::search::Range)
  • RangeSearchParamsError -> RangeSearchError (import from diskann::graph::RangeSearchError)
  • Return type changed from (SearchStats, Vec<O>, Vec<f32>) to RangeSearchOutput<O> (a struct with .stats, .ids, .distances fields)
  • Pass &mut () as the output buffer
  • Field starting_l_value -> constructor arg starting_l (accessor: .starting_l())
  • Field initial_search_slack -> constructor arg initial_slack (accessor: .initial_slack())
  • Field range_search_slack -> constructor arg range_slack (accessor: .range_slack())

4. Multihop / Label-Filtered Search (multihop_search)

Before:

use diskann::graph::SearchParams;

let params = SearchParams::new(10, 100, None)?;
let stats = index
   .multihop_search(&strategy, &ctx, &query, &params, &mut output, &label_eval)
   .await?;

After:

use diskann::graph::{Search, search::{Knn, MultihopSearch}};

let knn = Knn::new(10, 100, None)?;
let params = MultihopSearch::new(knn, &label_eval);
let stats = index.search(params, &strategy, &ctx, &query, &mut output).await?;

Key differences:

  • MultihopSearch wraps a Knn -> label evaluator into a single params object
  • The label evaluator is part of the params, not a separate argument

5. Flat Search (unchanged method, new param type)

Before:

use diskann::graph::SearchParams;

let params = SearchParams::new(10, 100, None)?;
index.flat_search(&strategy, &ctx, &query, &filter, &params, &mut output).await?;

After:

use diskann::graph::search::Knn;

let params = Knn::new(10, 100, None)?;
index.flat_search(&strategy, &ctx, &query, &filter, &params, &mut output).await?;

Only the parameter type changed (SearchParams -> Knn).

Import Path Changes

Old New
diskann::graph::SearchParams diskann::graph::search::Knn
diskann::graph::RangeSearchParams diskann::graph::search::Range
diskann::graph::SearchParamsError diskann::graph::KnnSearchError
diskann::graph::RangeSearchParamsError diskann::graph::RangeSearchError
diskann::graph::search::MultihopSearch (new)
diskann::graph::search::RecordedKnn (new)
diskann::graph::search::Diverse (new, feature-gated)
diskann::graph::Search (trait, re-exported)
diskann::graph::RangeSearchOutput (re-exported)

Change List

  • copy bftrees from the snapshot location to the save location by @backurs in #783
  • (RFC) Refactor search interface with unified SearchDispatch trait by @narendatha in #773
  • Make queue.closest_notvisited() safe and update call sites by @arrayka in #787
  • git ignore: Ignore local settings for claude code AI agent by @arrayka in #789
  • Enabling flag support in codecov by @arrayka in #790
  • Increase unit test coverage for diskann-tools crate by @Copilot in #763
  • Neon MVP by @hildebrandmw in #777
  • Adding GraphParams to be able to save graph parameters of index to SavedParams by @backurs in #786

New Contributors

Full Changelog: 0.46.0...v0.47.0