granne (graph-based retrieval of approximate nearest neighbors) is a Rust library for ANN-search based on Hierarchical Navigable Small World (HNSW) graphs and is used in Cliqz Search. For some background and motivation behind granne, please read Indexing Billions of Text Vectors.
Note: granne is still under active development. A more stable release (with documentation) is coming soon.
- Multithreaded index creation
- Extensible indexes (add elements to an already built index)
- Python bindings
int8elements (cosine distance)
granne is dependent on
BLAS (https://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms) for some computations. This applies both to the rust and python versions. On Debian/Ubuntu both
libopenblas-dev should work, with the latter being significantly faster.
On Mac OS there seems to be some issue (maybe
with the default
BLAS library. A workaround is to install e.g.
openblas and link to that instead.
You will also need
Rust installed. This can be done by calling:
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
Or by visiting https://rustup.rs/ and following the instructions there.
# build cargo build --release # test cargo test # bench cargo +nightly bench
To quickly install:
pip install setuptools_rust pip install .
To build python wheels for python 2.7, 3.4, 3.5 and 3.6 (requires docker).
docker build -t granne_manylinux docker/manylinux/ docker run -v $(pwd):/granne/ granne_manylinux /opt/build_wheels.sh
The output is written to
wheels/ and can be installed by
pip install granne --no-index -f wheels/
*granne is Swedish and means neighbor