You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently we only set the ef_search parameter when we init the index, usually on creation of a Collection. However, this parameter is actually per-query at the level of HNSW.
Generally we only need to set this parameter once since in principle there is a 'best' setting for the average case on any collection, but there are cases where we may want to let the user set it dynamically, for example to conduct a parameter sweep against a particular dataset.
Additionally, EF should be at least as large as the number of elements we want to return, which means it may have to be dynamic anyway.
I took a look at how voyager does this and I think it's buggy, because they just call set_ef on the index. What happens if you have concurrent queries asking for different EF's into the same index?
As far as I understand the code, the algo picks max(k,ef), so maybe we can trick it by passing per query ef that doesn't change the default behavior of the index if such is not provided (making it possible to parallelize queries).
Describe the problem
Currently we only set the
ef_search
parameter when we init the index, usually on creation of aCollection
. However, this parameter is actually per-query at the level of HNSW.Generally we only need to set this parameter once since in principle there is a 'best' setting for the average case on any collection, but there are cases where we may want to let the user set it dynamically, for example to conduct a parameter sweep against a particular dataset.
Additionally, EF should be at least as large as the number of elements we want to return, which means it may have to be dynamic anyway.
Describe the proposed solution
Allow EF search to be set on a per-query basis.
Alternatives considered
No response
Importance
would make my life easier
Additional Information
This is somewhat of a pain in the ass. Besides piping in this parameter on query (which now creates the need for a
query_parameters
argument...) the way the HNSW implementation handles this is, also, bad, since it's set as a property of the index: https://github.com/chroma-core/hnswlib/blob/d52dd85106c5a1955a7dbd745a76ba2a8259451a/hnswlib/hnswalg.h#L33even though it's only a property of each query.
To do this 'right' (for example, to handle concurrency correctly), it would take a refactor of our HNSW impl too.
The text was updated successfully, but these errors were encountered: