Milvus vector search with scalar filtering returns different results between v2.3.12 and v2.4.17

[kietheros]

I want to ask about vector search with scalar filtering.
Collection schema: (id, vector, user_id) Vector index: IVF_SQ8, IP, nlist 128

# pymilvus 2.4.15
search_params = {
    "metric_type": "IP",
    "params": {"nprobe": 10}
}
top_k = 1
results = collection.search(
    data=[input_vector],
    anns_field="vector",
    param=search_params,
    limit=top_k,
    expr=f"user_id in [11927]",
    output_fields=["id", "user_id"],
    round_decimal=3
)
print(results)

I tested with two Milvus server (same data).
I’m sure the collection contains 3 documents with user_id = 11927.

1. Milvus 2.3.12 (standalone)

results

data: ['["id: 13996, distance: -0.038, entity: {'user_id': 11927, 'id': 13996}"]']

2. Milvus 2.4.17 (cluster)

results

data: ['[]']
It seems that the vectors with user_id = 11927 might have a very small distance (-0.038) to the input_vector, so Milvus returns an empty result. Is that possible?

My questions:

1. When performing a query with scalar filtering, how does Milvus process it?

• Does Milvus apply the filter first to get a subset of vectors, then search within that subset?
• Or does it search the vectors first, then filter them, and if fewer than top_k remain, keep searching until top_k results are found?
• Does the execution strategy depend on the type of vector index (IVF, HNSW, etc.)?

2. What is the difference between the two Milvus versions above that could cause such different results?

3. How to optimize the above query, list of user IDs can have 1–2 thousand elements?

[xiaofan-luan]

could you try latest 2.5.16 and verify the results?

[kietheros]

@xiaofan-luan Currently, I only can test with Milvus 2.3 and 2.4. In my test, Milvus 2.4 holds more data than milvus 2.3.
From what I understand, when using the IVF_SQ8 index, Milvus first selects nprobe nearest clusters . It then scans the vectors in those clusters, applies the user_id filter, and returns the highest-scoring match. This means that if none of the vectors in the selected clusters meet the user_id condition, the search result can end up empty.
Please correct me if I am wrong

[xiaofan-luan]

milvus will usually apply filter first, and then send the bitset to vector index https://milvus.io/docs/bitset.md.

There are maybe several possibilities:

1. there is a bug here in 2.4.x. that's why you may want to verify if it's the same on latest 2.5.16
2. the target vector is not in the nprobe bucket. You can enlarge your nprobe same to nlist(which fallback to a brute force search) and see if you can get the right result

[kietheros]

@xiaofan-luan Is HNSW the index that provides the best recall when searching combined with filtering?

[xiaofan-luan]

it depends on your filter ratio, but most likely yes

[kietheros]

@xiaofan-luan Thank you for your support. I have one more question: is my understanding of how Milvus search works, as I described above, correct?

I understand that Milvus will first create a bitset for filtering. But with the index IVF, the first step is to select nprobe clusters, and this step does not use the bitset, so there may be cases where the target vector is not in those nprobe clusters

[xiaofan-luan]

@xiaofan-luan Thank you for your support. I have one more question: is my understanding of how Milvus search works, as I described above, correct?

I understand that Milvus will first create a bitset for filtering. But with the index IVF, the first step is to select nprobe clusters, and this step does not use the bitset, so there may be cases where the target vector is not in those nprobe clusters

yes, you are correct