API: Implement notStartsWith bounds check in StrictMetricsEvaluator#15883
Open
bharos wants to merge 3 commits intoapache:mainfrom
Open
API: Implement notStartsWith bounds check in StrictMetricsEvaluator#15883bharos wants to merge 3 commits intoapache:mainfrom
bharos wants to merge 3 commits intoapache:mainfrom
Conversation
When column bounds are entirely outside the prefix range, all rows must satisfy notStartsWith. Previously this always returned ROWS_MIGHT_NOT_MATCH regardless of bounds, missing an optimization opportunity for file-level pruning. Now returns ROWS_MUST_MATCH when: - Lower bound truncated to prefix length > prefix (all values above) - Upper bound truncated to prefix length < prefix (all values below) - Column contains only null values (nulls satisfy NOT predicates) Follows the same truncation pattern used in InclusiveMetricsEvaluator.startsWith and the null-handling pattern from StrictMetricsEvaluator.notEq.
bharos
force-pushed
the
perf/strict-metrics-not-starts-with-bounds
branch
from
432bb3e to
bbb3deb
Compare
anoopj
approved these changes
Apr 7, 2026
| // TODO: Handle cases that definitely cannot match, such as notStartsWith("x") when the bounds | ||
| // are ["a", "b"]. | ||
| int id = ref.fieldId(); | ||
| if (isNestedColumn(id)) { |
Contributor
There was a problem hiding this comment.
Consider adding a test for nested column?
Contributor
Author
There was a problem hiding this comment.
Thanks @anoopj , added a nested string field "nested_string_col" in test and a test case
PTAL
Contributor
|
The change looks reasonable to me. The only callout is the null handling: if there are null values, the implementation will return ROWS_MUST_MATCH for them as well. This doesn't follow SQL's 3-valued semantics, but is consistent with the current implementation of notEq, so I think the change is reasonable. Please get this reviewed by a committer. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Implements bounds-based evaluation for
notStartsWithinStrictMetricsEvaluator, replacing the existing TODO with actual logic.Previously,
notStartsWithalways returnedROWS_MIGHT_NOT_MATCH,which prevented the engine from eliminating the residual predicate even
when file-level column bounds made it provable that no value could start
with the given prefix.
Changes
StrictMetricsEvaluator.notStartsWith: Added checks for nestedcolumns, all-nulls columns, and lower/upper bound comparisons against
the prefix. Returns
ROWS_MUST_MATCHwhen bounds prove the prefix isentirely outside the value range.
TestStrictMetricsEvaluator: Added 8 test methods covering:all-nulls, bounds above/below/overlapping the prefix, wider ranges,
missing stats, some-nulls with bounds outside prefix, and prefix
longer than bounds.
How it works
For
NOT STARTS WITH <prefix>:min(prefixLen, boundLen)) isstrictly greater than the prefix, all values are above the prefix
range →
ROWS_MUST_MATCHmin(prefixLen, boundLen)) isstrictly less than the prefix, all values are below the prefix range
→
ROWS_MUST_MATCHROWS_MIGHT_NOT_MATCH(conservative)This follows the same pattern used by
notEqandnotInin thisclass, including the null-handling convention.
Closes #15882