Re-ranking based on metadata

[inmagalvez]

Hello! Is there a way to re-rank the retrieved chunks based on the value of a metadata field? I mean, first retrieving all the chunks with the semantic search and then re-ranking all of them based on that metadata value. I believe with the hybrid search this is not possible.

[yhmo]

1. If you want to rerank results by int/float type metadata, you can use "output_fields" to fetch the metadata and reorder them by yourself
   Example:

import random

from pymilvus import (
MilvusClient,DataType,CollectionSchema,FieldSchema
)

client = MilvusClient(uri="http://localhost:19530")

collection_name = "AAA"
dim = 128
client.drop_collection(collection_name=collection_name)


fields = [
    FieldSchema(name="id", dtype=DataType.INT64, is_primary=True),
    FieldSchema(name="vector", dtype=DataType.FLOAT_VECTOR, dim=dim),
    FieldSchema(name="flag", dtype=DataType.INT64),
]

schema = CollectionSchema(fields)
client.create_collection(
    collection_name=collection_name,
    schema=schema,
)
index_params = client.prepare_index_params()
index_params.add_index(field_name="vector", index_type="FLAT", metric_type="COSINE")
client.create_index(collection_name, index_params)
client.load_collection(collection_name=collection_name)
print(collection_name, "created")

data = [{"id": k, "vector": [random.random() for _ in range(dim)], "flag": k%100} for k in range(1000)]
client.insert(collection_name=collection_name, data=data)


results = client.search(collection_name=collection_name,
                    data=[[random.random() for _ in range(dim)]],
                    limit=10,
                    anns_field="vector",
                    output_fields=["flag"],
                    consistency_level="Strong",
                    )
for res in results:
    print("before sort")
    for r in res:
        print(r)
    res.sort(key=lambda x:x["entity"]["flag"])
    print("after sort")
    for r in res:
        print(r)

Output:

AAA created
before sort
{'id': 269, 'distance': 0.8211848139762878, 'entity': {'flag': 69}}
{'id': 737, 'distance': 0.816531240940094, 'entity': {'flag': 37}}
{'id': 875, 'distance': 0.8149145245552063, 'entity': {'flag': 75}}
{'id': 247, 'distance': 0.812071681022644, 'entity': {'flag': 47}}
{'id': 951, 'distance': 0.8093258738517761, 'entity': {'flag': 51}}
{'id': 405, 'distance': 0.8049411773681641, 'entity': {'flag': 5}}
{'id': 880, 'distance': 0.8037360310554504, 'entity': {'flag': 80}}
{'id': 861, 'distance': 0.803280234336853, 'entity': {'flag': 61}}
{'id': 223, 'distance': 0.8000259399414062, 'entity': {'flag': 23}}
{'id': 673, 'distance': 0.7990730404853821, 'entity': {'flag': 73}}
after sort
{'id': 405, 'distance': 0.8049411773681641, 'entity': {'flag': 5}}
{'id': 223, 'distance': 0.8000259399414062, 'entity': {'flag': 23}}
{'id': 737, 'distance': 0.816531240940094, 'entity': {'flag': 37}}
{'id': 247, 'distance': 0.812071681022644, 'entity': {'flag': 47}}
{'id': 951, 'distance': 0.8093258738517761, 'entity': {'flag': 51}}
{'id': 861, 'distance': 0.803280234336853, 'entity': {'flag': 61}}
{'id': 269, 'distance': 0.8211848139762878, 'entity': {'flag': 69}}
{'id': 673, 'distance': 0.7990730404853821, 'entity': {'flag': 73}}
{'id': 875, 'distance': 0.8149145245552063, 'entity': {'flag': 75}}
{'id': 880, 'distance': 0.8037360310554504, 'entity': {'flag': 80}}

2. If you want to rerank results by semantics of text, you can use the post-search rerankers: https://milvus.io/docs/rerankers-overview.md

[inmagalvez]

Thank you for you answer!
However, with the proposed approach (the one detailed in 1, by first retrieving the most semantically similar records and then reordering those according to the metadata field) I would be only reranking 10 records out of 1000 (in the detailed example). What I need is to give a weight to the semantic similarity metric and another weight to the metadata field, so the reranking is applied to all the records present in the collection.

[yhmo]

Currently, milvus doesn't support hybrid search between vector fields and metadata fields. I think it is a new requirement.
Seems it is not a general requirement since I didn't see similar questions from other users before.

[inmagalvez]

Alright, thank you very much for your answer!

[xiaofan-luan]

this is actually on our roadmap.

It could happens on the rerank stage where you can pull some scalar data out rerank together with similarity score.

have a issue opened here #37066