[Bug]: Insert into FLOAT16_VECTOR field with DISKANN index fails with <ParamError: (code=1, message=Collection field dim is 1024, but entities field dim is 512)>
#31724
Assignees
Labels
kind/bug
Issues or changes related a bug
stale
indicates no udpates for 30 days
triage/accepted
Indicates an issue or PR is ready to be actively worked on.
Milestone
Comments
wilsonzlin
added
kind/bug
|
@liliu-z |
|
/assign @liliu-z |
yanliang567
added
triage/accepted
|
diskann index actaully don't support BF16/Float16 index, we are still work on it. |
|
Thanks for the update, yes I agree that a better check/message would be more debuggable, and perhaps also a mention in the docs. |
|
Hi! I faced a similar issue but only with FLOAT16_VECTOR without any indices at all. How to reproduce: from pymilvus import Collection, CollectionSchema, DataType, FieldSchema, connections
connections.connect()
work_id = FieldSchema(
name="id",
dtype=DataType.VARCHAR,
max_length=33,
description="work_id",
is_primary=True,
)
embedding = FieldSchema(
name="embedding",
dtype=DataType.FLOAT16_VECTOR,
descrition="title + abstract embedding",
dim=1024,
is_primary=False,
)
year = FieldSchema(
name="year",
dtype=DataType.INT64,
description="year",
is_primary=False,
)
concept_id_lvl_0 = FieldSchema(
name="concept_id_lvl_0",
dtype=DataType.VARCHAR,
max_length=64,
description="concept_id_lvl_0",
is_primary=False,
)
concept_id_lvl_1 = FieldSchema(
name="concept_id_lvl_1",
dtype=DataType.VARCHAR,
max_length=64,
description="concept_id_lvl_1",
is_primary=False,
)
schema = CollectionSchema(
fields=[
work_id,
embedding,
year,
concept_id_lvl_0,
concept_id_lvl_1,
],
description="works collection",
)
collection = Collection(name='some_name', schema=schema)
collection.insert(
data=[['W1234'], [[0] * 1024], [2010], [['C418']], [['C418']]],
partition_name=str(2010),
)
collection.flush()Traceback: RPC error: [batch_insert], <ParamError: (code=1, message=Collection field dim is 1024, but entities field dim is 512)>, <Time:{'RPC start': '2024-04-13 13:03:23.231383', 'RPC error': '2024-04-13 13:03:23.231460'}>
Traceback (most recent call last):
File "/home/nmyakishev-93/milvus_fp16.py", line 53, in <module>
collection.insert(
File "/home/nmyakishev-93/.pyenv/versions/3.11.7/envs/proj_venv/lib/python3.11/site-packages/pymilvus/orm/collection.py", line 500, in insert
return conn.batch_insert(
^^^^^^^^^^^^^^^^^^
File "/home/nmyakishev-93/.pyenv/versions/3.11.7/envs/proj_venv/lib/python3.11/site-packages/pymilvus/decorators.py", line 147, in handler
raise e from e
File "/home/nmyakishev-93/.pyenv/versions/3.11.7/envs/proj_venv/lib/python3.11/site-packages/pymilvus/decorators.py", line 143, in handler
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/nmyakishev-93/.pyenv/versions/3.11.7/envs/proj_venv/lib/python3.11/site-packages/pymilvus/decorators.py", line 182, in handler
return func(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/nmyakishev-93/.pyenv/versions/3.11.7/envs/proj_venv/lib/python3.11/site-packages/pymilvus/decorators.py", line 122, in handler
raise e from e
File "/home/nmyakishev-93/.pyenv/versions/3.11.7/envs/proj_venv/lib/python3.11/site-packages/pymilvus/decorators.py", line 87, in handler
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/nmyakishev-93/.pyenv/versions/3.11.7/envs/proj_venv/lib/python3.11/site-packages/pymilvus/client/grpc_handler.py", line 575, in batch_insert
raise err from err
File "/home/nmyakishev-93/.pyenv/versions/3.11.7/envs/proj_venv/lib/python3.11/site-packages/pymilvus/client/grpc_handler.py", line 558, in batch_insert
request = self._prepare_batch_insert_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/nmyakishev-93/.pyenv/versions/3.11.7/envs/proj_venv/lib/python3.11/site-packages/pymilvus/client/grpc_handler.py", line 542, in _prepare_batch_insert_request
else Prepare.batch_insert_param(collection_name, entities, partition_name, fields_info)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/nmyakishev-93/.pyenv/versions/3.11.7/envs/proj_venv/lib/python3.11/site-packages/pymilvus/client/prepare.py", line 527, in batch_insert_param
location = cls._pre_batch_check(entities, fields_info)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/nmyakishev-93/.pyenv/versions/3.11.7/envs/proj_venv/lib/python3.11/site-packages/pymilvus/client/prepare.py", line 472, in _pre_batch_check
location, primary_key_loc, auto_id_loc = traverse_info(fields_info, entities)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/nmyakishev-93/.pyenv/versions/3.11.7/envs/proj_venv/lib/python3.11/site-packages/pymilvus/client/utils.py", line 320, in traverse_info
raise ParamError(
pymilvus.exceptions.ParamError: <ParamError: (code=1, message=Collection field dim is 1024, but entities field dim is 512)>pymilvus version: |
|
this will be fixed in 2.4 |
|
/assign @yanliang567 |
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Is there an existing issue for this?
Environment
Current Behavior
When I create a simple collection with a
vecFLOAT16_VECTOR field, and a DISKANN index for that field, I get an insert error that seems to imply the insert vector's dimensions is incorrect, when it is correct. The reported dimension is exactly half, which I suspect is related to the FLOAT16_VECTOR field type. If I use FLOAT_VECTOR as the type, it works fine.Expected Behavior
It should insert without failing.
Steps To Reproduce
Set up a standard Milvus Docker Compose stack, then run the following script:
Milvus Log
The text was updated successfully, but these errors were encountered: