fix(milvus): make metric type configurable to prevent mismatch errors#6199
fix(milvus): make metric type configurable to prevent mismatch errors#6199octo-patch wants to merge 1 commit intoFlowiseAI:mainfrom
Conversation
…fixes FlowiseAI#6033) Previously the metric type was hardcoded to L2 when creating a new Milvus index in addVectors(), causing metric type mismatch errors for collections using COSINE or IP. Added a configurable Metric Type input parameter (L2, COSINE, IP) and wired it through indexCreateParams so upsert operations use the correct metric type.
There was a problem hiding this comment.
Code Review
This pull request updates the Milvus vector store component to version 2.2 and introduces a configurable 'Metric Type' input for similarity searches, supporting L2, COSINE, and IP metrics. While the metric type is correctly implemented for upsert operations, it is currently missing from the 'init' method. This omission will cause retrieval and search operations to ignore the user-selected metric, potentially leading to incorrect results or runtime errors when accessing index parameters.
| }, | ||
| { | ||
| label: 'Metric Type', | ||
| name: 'milvusMetricType', |
There was a problem hiding this comment.
The new milvusMetricType input is correctly utilized in the upsert method, but it is missing from the init method (lines 266-347). This is a significant oversight because when the node is used for retrieval (the retriever output) or to connect to an existing collection, it will ignore the user-selected metric type.
This leads to two issues:
- Potential Crash:
similaritySearchVectorWithScore(line 387) and the score normalization logic (line 420) both accessvectorStore.indexCreateParams.metric_type. IfindexCreateParamsis not provided ininit, this may throw an error or result in incorrect search behavior. - Metric Mismatch: Search operations will use the default metric type instead of the one configured by the user, which is the exact problem this PR aims to solve for upserts.
The init method needs to extract milvusMetricType and include it in milVusArgs.indexCreateParams similarly to the upsert implementation.
References
- When a feature requires a specific configuration, it is preferable to throw an error if the configuration is missing rather than silently falling back to a different implementation.
Fixes #6033
Problem
The Milvus node hardcoded
MetricType.L2when creating a new index inMilvusUpsert.addVectors(). Users with collections configured to useCOSINEorIPmetric types encountered metric type mismatch errors during vector insertion, because the index was always created with L2 regardless of the collection's actual metric type.Solution
L2(default, backward-compatible),COSINE, andIPindexCreateParamsinMilvusLibArgsduring upsertMilvusUpsert.addVectors()now usesthis.indexCreateParams?.metric_type ?? MetricType.L2instead of the hardcodedMetricType.L2, ensuring the index is created with the correct metric type2.1to2.2Testing
L2for full backward compatibility with existing flowsvectorStore.indexCreateParams.metric_typecorrectly — this fix aligns upsert to do the same