Fix update_embeddings() for FAISSDocumentStore #978
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There are three main use cases for
update_embeddings()
:Creating embeddings in a new document store after writing the documents. In this case, embeddings are created for all the documents in the document store. The default parameters for update_embeddings() are sufficient for this case.
Incremental update after adding more documents in an existing index. In this case, update_embeddings() can be called with
update_existing_embeddings
set to False to prevent recomputation of existing embeddings.Create/Update embeddings for a subset of documents in a document store using the
filters
parameter. ForFAISSDocumentStore
,filters
cannot be used in conjunction with theupdate_existing_embeddings
set toTrue
.This PR resolves #885 where
update_embeddings()
could result in more embeddings than the number of documents and adds a new methodget_embedding_count()
for all document stores.