Child issue of EPIC #22395.
Consider this query
select a, binned
from my_range_partitioned_table
group by a, date_bin(1 hour, timestamp, 0) as binned
on a table which is Range partitioned on timestamp by day.
This query should not have a partial aggregate -> repartition -> final aggregate pipeline because
- 1 hour <= 1 day
- there are no incomplete intervals (ie. there's no 1 hour window which overlaps a split point; in other words, there's no 1 hour window that is partially in one partition and partially in another)
If the query were like this, then you cannot remove the repartition since 27 hours spans multiple partitions.
select a, binned
from my_range_partitioned_table
group by a, date_bin(27 hours, timestamp, 0) as binned
Due to subset satisfaction, both of these partitioning schemes should allow this query to avoid repartitions.
Range partitioned on timestamp by day
Range partitioned on timestamp by day and a by some arbitrary range.
Related
Child issue of EPIC #22395.
Consider this query
on a table which is
Rangepartitioned ontimestampby day.This query should not have a partial aggregate -> repartition -> final aggregate pipeline because
If the query were like this, then you cannot remove the repartition since 27 hours spans multiple partitions.
Due to subset satisfaction, both of these partitioning schemes should allow this query to avoid repartitions.
Rangepartitioned ontimestampby dayRangepartitioned ontimestampby day andaby some arbitrary range.Related
Partitioning::Rangeto satisfy windowDistribution::KeyPartitionedrequirements #23289