feat: migrate multi-vector query and reranker logic to C++ by chinaux · Pull Request #405 · alibaba/zvec

chinaux · 2026-05-14T11:41:04Z

refact:

Add Reranker base class with RrfReRanker and WeightedReRanker implementations
Add Collection::MultiQuery interface for multi-vector queries with reranking
Add MultiVectorQuery struct in doc.h with forward declaration for Reranker
Add C API bindings for reranker and MultiQuery (zvec_reranker_, zvec_multi_vector_query_, zvec_collection_multi_query)
Add Python binding for reranker classes with py::function bridge for callback

- Add Reranker base class with RrfReRanker and WeightedReRanker implementations - Add Collection::MultiQuery interface for multi-vector queries with reranking - Add MultiVectorQuery struct in doc.h with forward declaration for Reranker - Add C API bindings for reranker and MultiQuery (zvec_reranker_*, zvec_multi_vector_query_*, zvec_collection_multi_query) - Add Python binding for reranker classes with py::function bridge for callback - Validate duplicate field names in multi-vector queries (C++ and Python consistent) - Remove TODO comment about concurrent execution (SQLEngine is not thread-safe) - Update collection.h MultiQuery doc comment from concurrently to sequentially - Add C++ collection tests (6 MultiQuery test cases) - Add C API tests (reranker functions + multi_vector_query end-to-end) - Implement Python test cases (11 previously skipped tests now active) - Simplify Python query_executor validation for unified duplicate field check

…idate_and_sanitize)

…se2, sparse2)

…ction._get_object

- Register _SubVectorQuery in pybind11 with from_vector_query() factory - Convert _VectorQuery to _SubVectorQuery in MultiVectorQueryExecutor - Relax RRF/Weighted score assertion tolerance from 1e-10 to 1e-6 - Fix WeightedReRanker test metric to IP (matching HnswIndexParam default)

…eclaration

- import Callable from collections.abc instead of typing (UP035) - remove redundant quotes around MetricType annotations (UP037)

…directly, and use insert return value to avoid duplicate set lookup

…PI callback reranker

egolearner

LGTM

Adapt FTS to the query-model refactor from alibaba#428/alibaba#405 on main: - VectorQuery is replaced by SearchQuery + QueryTarget (variant<VectorClause, FtsClause>) - Fold FTS validation into SearchQuery::validate_and_sanitize - c_api / python binding read & write FTS via FtsClause and target_.clause_ - Add QueryTarget::get_fts_clause() helper, replacing raw std::get_if calls - Migrate FTS tests to the new model; drop the vector/fts mutual-exclusion case that no longer applies under the variant

chinaux requested review from Cuiyus, feihongxu0824 and zhourrr as code owners May 14, 2026 11:41

github-actions Bot assigned chinaux May 14, 2026

zhourrr reviewed May 14, 2026

View reviewed changes

Comment thread src/include/zvec/db/collection.h Outdated

zhourrr reviewed May 15, 2026

View reviewed changes

Comment thread src/db/collection.cc

zhourrr reviewed May 15, 2026

View reviewed changes

Comment thread src/include/zvec/db/doc.h Outdated

zhourrr reviewed May 15, 2026

View reviewed changes

Comment thread src/include/zvec/db/doc.h Outdated

zhourrr reviewed May 15, 2026

View reviewed changes

Comment thread src/include/zvec/db/doc.h Outdated

zhourrr reviewed May 15, 2026

View reviewed changes

Comment thread src/include/zvec/db/reranker.h Outdated

zhourrr reviewed May 15, 2026

View reviewed changes

Comment thread src/db/reranker/reranker.cc

zhourrr reviewed May 15, 2026

View reviewed changes

Comment thread src/binding/c/c_api.cc

zhourrr reviewed May 15, 2026

View reviewed changes

Comment thread src/include/zvec/c_api.h

zhourrr reviewed May 19, 2026

View reviewed changes

Comment thread src/include/zvec/db/query.h Outdated

zhourrr reviewed May 19, 2026

View reviewed changes

Comment thread src/include/zvec/db/query.h Outdated

zhourrr force-pushed the feat/refact_py_to_cpp branch from 9f92f1b to 3d46458 Compare May 25, 2026 02:47

zhourrr requested a review from egolearner May 25, 2026 05:44

zhourrr reviewed May 25, 2026

View reviewed changes

Comment thread src/include/zvec/db/reranker.h Outdated

egolearner reviewed May 26, 2026

View reviewed changes

Comment thread src/include/zvec/db/reranker.h Outdated

Comment thread src/include/zvec/db/reranker.h Outdated

Comment thread src/include/zvec/db/reranker.h

Comment thread src/db/reranker/reranker.cc Outdated

egolearner reviewed May 26, 2026

View reviewed changes

Comment thread src/include/zvec/db/query.h Outdated

chinaux added 10 commits May 28, 2026 09:54

style: format Python files with ruff

2b15287

fix: adapt to main branch API changes (VectorQuery->Query rename, val…

2822e05

…idate_and_sanitize)

fix: multi_vector tests now use multiple same-type vector fields (den…

fb5fc1b

…se2, sparse2)

fix: suppress RET501 for intentional default return None in RerankFun…

e5fe8e0

…ction._get_object

style: ruff format test_collection.py

7507f71

refact multi vector query

ed69888

format code

3025df4

style: ruff format query_executor.py

5b341f6

chinaux added 6 commits May 28, 2026 09:54

fix: define _USE_MATH_DEFINES for M_PI on Windows (MSVC)

eaf11f8

refactor: include reranker.h directly in query.h instead of forward d…

e05e13e

…eclaration

refactor(reranker): move topn from member variable to rerank() parameter

84d832f

refact code

abcefef

style(python): fix ruff UP035/UP037 in multi_vector_reranker

567ada3

- import Callable from collections.abc instead of typing (UP035) - remove redundant quotes around MetricType annotations (UP037)

chore: trigger PR sync

b4c0cb3

chinaux force-pushed the feat/refact_py_to_cpp branch from 1871946 to b4c0cb3 Compare May 28, 2026 01:55

zhourrr reviewed May 28, 2026

View reviewed changes

Comment thread src/db/reranker/reranker.cc Outdated

zhourrr reviewed May 28, 2026

View reviewed changes

Comment thread src/include/zvec/c_api.h Outdated

egolearner reviewed May 28, 2026

View reviewed changes

Comment thread src/db/reranker/reranker.cc Outdated

Comment thread src/db/reranker/reranker.cc Outdated

Comment thread src/db/reranker/reranker.cc Outdated

Comment thread src/include/zvec/db/query.h Outdated

Comment thread src/db/collection.cc Outdated

egolearner reviewed May 28, 2026

View reviewed changes

Comment thread src/include/zvec/db/reranker.h Outdated

Cuiyus reviewed May 28, 2026

View reviewed changes

Comment thread src/include/zvec/db/reranker.h Outdated

Comment thread src/include/zvec/db/query.h

Cuiyus reviewed May 28, 2026

View reviewed changes

Comment thread src/db/collection.cc

Cuiyus reviewed May 28, 2026

View reviewed changes

Comment thread src/db/collection.cc Outdated

refact code

b187572

chinaux force-pushed the feat/refact_py_to_cpp branch from c838e3a to b187572 Compare May 28, 2026 08:25

egolearner reviewed May 28, 2026

View reviewed changes

Comment thread examples/c/CMakeLists.txt Outdated

fix(examples): restore CMakeLists.txt formatting broken by clang-format

8c943e8

egolearner reviewed May 28, 2026

View reviewed changes

Comment thread src/db/reranker/reranker.cc Outdated

Comment thread src/db/collection.cc Outdated

refactor(reranker): remove redundant metrics_ map by querying schema …

26dc158

…directly, and use insert return value to avoid duplicate set lookup

egolearner reviewed May 28, 2026

View reviewed changes

Comment thread src/include/zvec/db/reranker.h Outdated

egolearner reviewed May 28, 2026

View reviewed changes

Comment thread src/binding/python/model/param/python_param.cc

egolearner reviewed May 28, 2026

View reviewed changes

Comment thread src/binding/python/model/python_collection.cc Outdated

egolearner reviewed May 28, 2026

View reviewed changes

Comment thread src/include/zvec/c_api.h Outdated

egolearner reviewed May 28, 2026

View reviewed changes

Comment thread src/include/zvec/c_api.h Outdated

egolearner reviewed May 28, 2026

View reviewed changes

Comment thread src/binding/python/model/python_reranker.cc Outdated

refactor(reranker): defer schema binding to query time and remove C A…

2acda85

…PI callback reranker

egolearner approved these changes May 28, 2026

View reviewed changes

chinaux merged commit f539580 into alibaba:main May 29, 2026
24 of 25 checks passed

Conversation

chinaux commented May 14, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

egolearner left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants