Cannot combine adjacent partial and final AggregateExecs into a single AggregateExec

[haruband]

Describe the bug

It seems a optimizer cannot combine adjacent partial and final AggregateExecs into a single AggregateExec. Because a EnforceDistribution rule which runs before a CombinePartialFinalAggregate rule inserts a RepartitionExec for hashing between adjacent partial and final AggregateExecs.

To Reproduce

No response

Expected behavior

A optimizer should combine adjacent partial and final AggregateExecs, if possible.

Additional context

No response

[alamb]

A optimizer should combine adjacent partial and final AggregateExecs, if possible.

I agree it should combine the adjacent aggregators if possible. One reason it may not be combining them is to increase parallelism as the initial partial AggregateExec runs in parallel on the raw data stream and then the second final AggregateExec runs after the initial pass has run.

This is partly illustrated here
https://docs.rs/datafusion/latest/datafusion/logical_expr/trait.Accumulator.html#tymethod.state

If you can share an EXPLAIN PLAN showing what you are seeing and we can perhaps help diagnose if there is an improvement to make in DataFusion or if something else is going on

[lewiszlw]

I think this is expected behavior. Closing it.
Feel free to reopen if you have further info.