[Bug]: [Nightly] The accuracy of range search using FLOAT16_VECTOR is lower than expected

[NicoYuan1986]

Is there an existing issue for this?

• I have searched the existing issues

Environment

- Milvus version: 23dedc2
- Deployment mode(standalone or cluster):cluster
- MQ type(rocksmq, pulsar or kafka):    pulsar
- SDK version(e.g. pymilvus v2.0.0rc2):
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

The accuracy of range search using FLOAT16_VECTOR is lower than expected.

[2024-05-30T19:43:22.211Z]             hit_rate = round(len(set(ids).intersection(set(range_ids))) / len(set(ids)), 2)
[2024-05-30T19:43:22.211Z]             log.debug(f"range search results with growing {bool(i % 2)} hit rate: {hit_rate}")
[2024-05-30T19:43:22.211Z] >           assert hit_rate >= 0.2    # issue #32630 to improve the accuracy
[2024-05-30T19:43:22.211Z] E           assert 0.18 >= 0.2

Expected Behavior

improve the accuracy

Steps To Reproduce

1. create collection, insert 8000 vectors,
2. search with topk=1000
3. range search from the 30th-330th distance as filter
4. verified the range search results is same as the search results in the range

Milvus Log

1. link: https://jenkins.milvus.io:18080/blue/organizations/jenkins/Milvus%20Nightly%20CI/detail/master/758/pipeline/243/
2. log: artifacts-milvus-distributed-pulsar-nightly-758-pymilvus-e2e-logs.tar.gz
3. collection name: search_collection_EWTw9Xmm
4. failed time: [2024-05-30T18:50:58.290Z] [gw1] [ 64%] FAILED testcases/test_search.py::TestCollectionRangeSearch::test_range_search_default[IVF_PQ-L2-FLOAT16_VECTOR]

Anything else?

No response

[xiaofan-luan]

hit rate 0.2 is too low.
why the test assert hit rate > 0.2?

[xiaofan-luan]

and I don't think this is related to fp16.
Either this is not a valid test, or this is a bug of range search, not fp16

[yanliang567]

I agree that this is an issue about range search #32630
/assign @liliu-z