Is your feature request related to a problem or challenge?
For every batch, BoundedWindowAggExec in Linear mode does two operations that are O(n) in the number of live partitions:
aggregate_evaluate_stateful / evaluate_stateful iterate all of PartitionBatches and probe PartitionWindowAggStates for each partition, per window expressionupdate_partition_batch calls set_most_recent_row on every buffered partition
This is pretty inefficient when the # of partitions significantly exceeds the batch size; most partitions won't receive a row in the current batch, so we shouldn't need to touch them.
Describe the solution you'd like
No response
Describe alternatives you've considered
No response
Additional context
No response
Is your feature request related to a problem or challenge?
For every batch,
BoundedWindowAggExecinLinearmode does two operations that areO(n)in the number of live partitions:aggregate_evaluate_stateful/evaluate_statefuliterate all ofPartitionBatchesand probePartitionWindowAggStatesfor each partition, per window expressionupdate_partition_batchcallsset_most_recent_rowon every buffered partitionThis is pretty inefficient when the # of partitions significantly exceeds the batch size; most partitions won't receive a row in the current batch, so we shouldn't need to touch them.
Describe the solution you'd like
No response
Describe alternatives you've considered
No response
Additional context
No response