use PeekableIntIterator for OR filter "partial index" value matchers #16300
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR switches the
OR
filter "partial index" value matchers to usePeekableIntIterator
instead ofIntIterator
which can dramatically improve performance when these are used on top of an index offset which has a small number of set bits (since we can useadvanceIfNeeded
instead of looping to checknext
until we seek the correct offset). These matchers are used when some sub-filters support indexes we make a synthetic value matcher that checks the index instead of actually evaluating the matchers, which can be quite beneficial, but in certain scenarios can also be rather expensive.The added benchmark is one such expensive scenario, where the OR filter is nested under an
AND
filter. The equality clause of the AND is rather selective, but the first clause of theOR
matches all rows, so the while loop of the previous code needs to callhasNext
/next
quite a lot to seek to the next offset.before:
after:
I should've known to do this in the first place given #8822 and done this as part of #15838, but the second best time is today or something.
This PR has: