fix order by optimization #2251
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
max-hoffman
approved these changes
Jan 10, 2024
added 2 commits
January 10, 2024 15:54
This was referenced Jan 11, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This hopefully fixes 24 sqllogictests.
We introduced an optimization where we would drop
Sortnodes for queries that includedORDER BYs if there wereIndexLookups that were a matching prefix over the sortfields. The idea was that since indexes are already in order there was no need to sort.However, there is a case when that isn't necessarily true. If the index is created from a filter over multiple columns, specifically when that filter contains an
ORexpression, it is possible to iterate over columns in a non-sorted order. During analysis, the filters are converted into several non-overlapping ranges; it is possible for a range expression in the range to overlap with another range's range expression, but for the two ranges to not overlap. It's clearer to look at the test.We didn't catch this bug earlier, as it only affects queries in dolt. We iterate over rows in the in-memory tables in such a way that the rows will still appear in order.