Skip to content

Support functions date_bin and date_trunc in range partition satisfaction #23569

Description

@jayshrivastava

Child issue of EPIC #22395.

Consider this query

select a, binned
from my_range_partitioned_table
group by a, date_bin(1 hour, timestamp, 0) as binned

on a table which is Range partitioned on timestamp by day.

This query should not have a partial aggregate -> repartition -> final aggregate pipeline because

  • 1 hour <= 1 day
  • there are no incomplete intervals (ie. there's no 1 hour window which overlaps a split point; in other words, there's no 1 hour window that is partially in one partition and partially in another)

If the query were like this, then you cannot remove the repartition since 27 hours spans multiple partitions.

select a, binned
from my_range_partitioned_table
group by a, date_bin(27 hours, timestamp, 0) as binned

Due to subset satisfaction, both of these partitioning schemes should allow this query to avoid repartitions.

  1. Range partitioned on timestamp by day
  2. Range partitioned on timestamp by day and a by some arbitrary range.

Related

Metadata

Metadata

Labels

enhancementNew feature or request

Fields

No fields configured for Feature.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions