fix(fts): sum all boolean must scores - #8292
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
✅ Gate recommendation: approve.
Summing all required scores at the conjunction and fallback boundaries restores order-independent ranking and matches the existing MemWAL contract. Keeping multi-child competitive floors at the combined node preserves correctness while retaining conservative block pruning.
Xuanwo
approved these changes
Aug 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is the bug?
Boolean FTS queries over committed indices only used the first
MUSTclause as a scoring clause. LaterMUSTclauses filtered membership but did not contribute to_score.This made boosts on later clauses ineffective, allowed clause order to change rankings, and disagreed with the active MemWAL path, which already sums every
MUSTscore.What issues does this cause?
How does this PR fix the problem?
_scorecolumn produced by the committed-index Boolean fallback.MUSTas a required scoring clause across Rust, Python, Java, and the FTS guide.MUSTqueries, real top-k truncation, same-column composable execution, cross-column fallback execution, and active MemWAL scoring.Validation
cargo fmt --all -- --checkcargo clippy --all --tests --benches -- -D warningscargo test -p lance test_boolean_must_onlycargo test -p lance test_boolean_must_scores_sum_across_execution_pathscargo test -p lance-index boolean_sums_all_matching_clause_scorescargo test -p lance-index required_conjunction_uses_all_must_scores_for_competitive_boundscd python && uv run make lintcd java && ./mvnw spotless:checkcould not be run because this environment has no JDK/JRE installed.Scope
This is a correctness prerequisite for OSS-1696, not the complete ticket. It does not implement the specialized WAND/MAXSCORE, ReqOpt, or Phrase strategies, instrumentation, or the 10M-row MMLB benchmark, and it should not close OSS-1696.
Base-plus-active structured Boolean execution also remains unsupported by the MemWAL planner and is not expanded in this focused fix. There are no index-format or public API changes, and this PR makes no performance claim.