Handling Empty Sparse Embeddings for Certain Queries in Hybrid Search

[shalini0311]

Hi everyone,

I’m using Milvus with hybrid search and have fitted my documents for sparse embeddings. However, I’m encountering an issue where, for certain queries, the sparse embeddings are coming up empty. This happens with specific queries, and I’m not sure how to handle these cases effectively. I suspect that this may be due to the query being out of the corpus on which the sparse embeddings were fitted.

This is a sample code I'm using:

from milvus_model.sparse import BM25EmbeddingFunction
class MyBM25SparseEmbedding(BM25SparseEmbedding):
    def __init__(self, corpus: List[str] = None):
        if corpus is not None:
            super().__init__(corpus=corpus)

    def load(self, save_path: str):
        self.bm25_ef = BM25EmbeddingFunction()
        self.bm25_ef.load(save_path)

    def save(self, save_path: str):
        self.bm25_ef.save(save_path)

For search

sparse_search_params = {"metric_type": "IP"}
dense_search_params = {"metric_type": "IP", "params": {}}

sparse_embedding_func = MyBM25SparseEmbedding()
sparse_embedding_func.load(bm25_store_file)
print(sparse_embedding_func.bm25_ef.idf)

collection = Collection(collection_name)
retriever = MilvusCollectionHybridSearchRetriever(
    collection=collection,
    rerank=WeightedRanker(0.5, 0.5),
    anns_fields=[dense_field, sparse_field],
    field_embeddings=[dense_embedding_func, sparse_embedding_func],
    field_search_params=[dense_search_params, sparse_search_params],
    top_k=3,
    text_field=text_field,
)

question = "How to start learning programming?"
print(question)
documents = retriever.get_relevant_documents(question)
print(f'Similar docs: {len(documents)}')
for document in documents:
    print(document)

1. How can I handle the scenario where sparse embeddings return empty to ensure hybrid search works correctly?
2. Is there a way to detect or prevent empty sparse embeddings before running the search?

Looking forward to hearing your thoughts or any suggestions on how to resolve this!

Thanks in advance!

[yhmo]

Which version of your milvus server?

The recent versions already support empty sparse embeddings.
This is an example to show empty sparse embeddings are inserted and queried. This example works well with milvus v2.5.4.

import random
import numpy as np

from pymilvus import (
    utility,
    FieldSchema, CollectionSchema, DataType,
    Collection, AnnSearchRequest, RRFRanker, connections,
)

dense_dim = 128

connections.connect("default", host="localhost", port="19530")

fields = [
    FieldSchema(name="pk", dtype=DataType.VARCHAR,
                is_primary=True, auto_id=True, max_length=100),
    FieldSchema(name="text", dtype=DataType.VARCHAR, max_length=512),
    FieldSchema(name="sparse_vector", dtype=DataType.SPARSE_FLOAT_VECTOR),
    FieldSchema(name="dense_vector", dtype=DataType.FLOAT_VECTOR, dim=dense_dim),
]
schema = CollectionSchema(fields, "")
col_name = 'hybrid_demo'
utility.drop_collection(collection_name=col_name)
col = Collection(col_name, schema, consistency_level="Strong")

sparse_index = {"index_type": "SPARSE_INVERTED_INDEX", "metric_type": "IP"}
col.create_index("sparse_vector", sparse_index)
dense_index = {"index_type": "FLAT", "metric_type": "IP"}
col.create_index("dense_vector", dense_index)
col.load()
print("collection created")

rng = np.random.default_rng()
# note that the sparse vectors are empty
col.insert({"text": "milvus is a vector database", "sparse_vector": {}, "dense_vector": [random.random() for _ in range(dense_dim)]})
col.insert({"text": "today is sunday", "sparse_vector": {}, "dense_vector": [random.random() for _ in range(dense_dim)]})
col.insert({"text": "the sky is blue", "sparse_vector": {}, "dense_vector": [random.random() for _ in range(dense_dim)]})
col.flush()
print("data inserted")

k = 1
sparse_search_params = {"metric_type": "IP"}
# note that the sparse vectors are empty
sparse_req = AnnSearchRequest(data=[{}],
                              anns_field="sparse_vector",
                              param=sparse_search_params,
                              limit=k)
dense_search_params = {"metric_type": "IP"}
dense_req = AnnSearchRequest(data=[[random.random() for _ in range(dense_dim)]],
                             anns_field="dense_vector",
                             param=dense_search_params,
                             limit=k)

res = col.hybrid_search(reqs=[sparse_req, dense_req],
                        rerank=RRFRanker(),
                        limit=k,
                        output_fields=['text'])
print("hybrid search results:")
res = res[0]
for hit in res:
    print(f'text: {hit.fields["text"]} distance {hit.distance}')

[shalini0311]

Using the above command, I see that my server version is - v2.4.11. Let me try upgrading it and I'll get back to you then.

[yhmo]

Milvus Lite is not a full-function product.
The Milvus Standalone installed by docker-compose is recommended: https://milvus.io/docs/install_standalone-docker-compose.md , the latest version is v2.5.4

[shalini0311]

@yhmo , there's some organizational challenges in upgrading the Milvus server to 2.5x. Is there any workaround do you suggest to handle this in 2.4x (<=2.4.11)?

[yhmo]

If you cannot upgrade milvus, you need to clearly know how the BM25SparseEmbedding works to avoid empty sparse embedding.

BM25SparseEmbedding is a class of langchain.milvus: https://github.com/langchain-ai/langchain-milvus/blob/main/libs/milvus/langchain_milvus/utils/sparse.py#L20
Two interfaces in this class: embed_query() and embed_documents()
It uses BM25EmbeddingFunction to do bm25 work.

BM25EmbeddingFunction is a class of milvus-model: https://github.com/milvus-io/milvus-model/blob/ac5bdcd35c96b5a8a740b7f7689d96b05bfe0008/src/pymilvus/model/sparse/bm25/bm25.py#L39
BM25EmbeddingFunction construct bm25 idf by corpus: https://github.com/milvus-io/milvus-model/blob/ac5bdcd35c96b5a8a740b7f7689d96b05bfe0008/src/pymilvus/model/sparse/bm25/bm25.py#L125
load()/save() is to load/save idf data from a json file: https://github.com/milvus-io/milvus-model/blob/ac5bdcd35c96b5a8a740b7f7689d96b05bfe0008/src/pymilvus/model/sparse/bm25/bm25.py#L190

When you call MilvusCollectionHybridSearchRetriever.get_relevant_documents(), the BM25SparseEmbedding.embed_query() is called to convert the "question" to a sparse embedding.

retriever = MilvusCollectionHybridSearchRetriever(
    collection=collection,
    rerank=WeightedRanker(0.5, 0.5),
    anns_fields=[dense_field, sparse_field],
    field_embeddings=[dense_embedding_func, sparse_embedding_func],
    field_search_params=[dense_search_params, sparse_search_params],
    top_k=3,
    text_field=text_field,
)
documents = retriever.get_relevant_documents(question)

Since the "question" doesn't contain any words of the idf, it returns an empty sparse embedding, which means this "question" doesn't have any relevance to the documents. You can skip the search if the returned sparse embedding is empty.

retriever = MilvusCollectionHybridSearchRetriever(
    collection=collection,
    rerank=WeightedRanker(0.5, 0.5),
    anns_fields=[dense_field, sparse_field],
    field_embeddings=[dense_embedding_func, sparse_embedding_func],
    field_search_params=[dense_search_params, sparse_search_params],
    top_k=3,
    text_field=text_field,
)
# do search if the sparse embedding is not empty, otherwise skip
if len(sparse_embedding_func.embed_query(question)) > 0:
  documents = retriever.get_relevant_documents(question)

[shalini0311]

@yhmo , Thank you, can we perform dense search with such queries, instead of skipping it?