Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fully support index-free encoding and querying #38

Closed
bclavie opened this issue Jan 12, 2024 · 0 comments
Closed

Fully support index-free encoding and querying #38

bclavie opened this issue Jan 12, 2024 · 0 comments
Labels
enhancement New feature or request ongoing Feature is currently being worked on

Comments

@bclavie
Copy link
Owner

bclavie commented Jan 12, 2024

Most of the necessary functions are currently present, but not fully implemented.

While it can run slower and is memory intensive, there's nothing stopping us from querying smaller collections on-device, by encoding the documents and performing the computation without building an index.

The goal here would for RAGPretrainedModel/the ColBERT model class to support an additional .index_free_encode() and .index_free_search() (or better naming) functions. The former would encode docs and store their representation in-memory, while the latter would query it.

Functionally very similar to rerank(), except encoding & searching are performed at different stages and the encodings are stored, rather than on-the-fly for rerank.

@bclavie bclavie added enhancement New feature or request ongoing Feature is currently being worked on labels Jan 12, 2024
@bclavie bclavie closed this as completed Jan 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request ongoing Feature is currently being worked on
Projects
None yet
Development

No branches or pull requests

1 participant