Skip to content

FilterPushdown will generate duplicate exprs #3914

@waynexia

Description

@waynexia

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
The plan used to (before #3841) be like

https://github.com/apache/arrow-datafusion/blob/a9ee67471d1ab0ad11f12d024315b3a824d6101f/benchmarks/expected-plans/q21.txt#L15

Which contains some duplicate exprs, see the file diff for highlights.

And in #3841 @Dandandan run some optimize rules one more time and got the duplicate exprs optimized out. But in #3861 the duplicate exprs will cause another problem when combine with CommonSubexpressionEliminate rule.

Describe the solution you'd like

Besides let other rules to do further optimization (this is useful to make more optimizations), I think it's also important to generate a "good enough" plan from a rule itself (i.e., remove duplications like this in FilterPushdown). So we can cut the dependent graph among individual plans (which is hard to maintain...).

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions