You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
(1) ANN search on the dense vector itself - indexing, retrieval, etc. This will require a dependency on faiss, hnsw, etc.
(2) query encoding. This will require a dependency on hgf, PyTorch, Tensorflow, etc.
(3) document encoding. This may be the same as (2) or may be different.
(4) code for learning the encoders. Obviously, this will have lots of dependencies.
My proposal is that (1) and (2) go directly into pyserini, (3) and (4) go into another repo. The rationale is that pyserini is a retrieval toolkit, and should only have the minimal amount of code necessary to perform dense retrieval, here, with "pre-built" vector encodings.
This means that pyserini will be burden with a longer dependency chain, but I don't really see a way around that...
The text was updated successfully, but these errors were encountered:
There are four components we need to worry about:
My proposal is that (1) and (2) go directly into pyserini, (3) and (4) go into another repo. The rationale is that
pyserini
is a retrieval toolkit, and should only have the minimal amount of code necessary to perform dense retrieval, here, with "pre-built" vector encodings.This means that pyserini will be burden with a longer dependency chain, but I don't really see a way around that...
The text was updated successfully, but these errors were encountered: