You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In #9593, @suremarc added a way to reorganize input files in a ListingTable to avoid a merge, if the sort key ranges do not overlap
This feature is behind a feature flag, split_file_groups_by_statistics which defaults to false as I think there needs to be some more tests in place before we turn it on
Describe the solution you'd like
Add additional tests and then enable split_file_groups_by_statistics by default
Describe alternatives you've considered
No response
Additional context
No response
The text was updated successfully, but these errors were encountered:
I think a good place to start would be to write some sqllogic level tests to cover the important cases
Perhaos for the first test:
Create files: file1.parquet, file2.parquet both sorted on a but file 1 has the columns in the order a, b, c and file has the columns in the order c, b, a. The keyranges of values of a should be non overlapping
Create an external table a, b, c with explicit order by a, and then query SELECT ... ORDER BY a and make sure the output plan doesn't use sort preserving merge
Is your feature request related to a problem or challenge?
Part of #10313
In #9593, @suremarc added a way to reorganize input files in a ListingTable to avoid a merge, if the sort key ranges do not overlap
This feature is behind a feature flag,
split_file_groups_by_statistics
which defaults tofalse
as I think there needs to be some more tests in place before we turn it onDescribe the solution you'd like
Add additional tests and then enable
split_file_groups_by_statistics
by defaultDescribe alternatives you've considered
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: