Optimize the evaluation of `DATE_TRUNC(<col>) == <constant>)` when pushed down

[alamb]

Is your feature request related to a problem or challenge?

Filing on behalf of @colinmarc and @drin so we can get more community help

It is much easier for table providers such as the build in parquet one as well as iceberg to push down <col> op <constant> expressions. They often don't push the predicates down if they have a scalar function wrapping them

For example using the predicate

date_trunc(part, column) <= constant_rhs 

Will not typically be able to be pushed down

However, this equivalent expression can be

column <= date_trunc(part, date_add(constant_rhs, INTERVAL 1 part)

Describe the solution you'd like

It would be good if DataFusion could do this type of rewrite

Describe alternatives you've considered

This seems to fit well into the existing simply expressions framework:
https://github.com/apache/datafusion/tree/main/datafusion/optimizer/src/simplify_expressions

And we already do something very similar for casts in unwrap casts: https://github.com/apache/datafusion/blob/e12ef3ae90677fe4b1bc548feea2b3082eecdaa2/datafusion/optimizer/src/simplify_expressions/unwrap_cast.rs#

For example

CAST(x AS float) = 5

is rewritten to

x = CAST(5 as float)

The code for that is here:

Perhaps we can follow a similar model for date_trunc

Note There is already a ScalarUDFImpl::simplify for simplifying functions, however that method doesn't get any part of the larger expression (the = in the above code) so we might have to extend the API somewhat

Additional context

No response

[alamb]

It appears this PR to duckdb added the same optimization:

• Add date_trunc() simplification rules duckdb/duckdb#18457

[UBarney]

take

[drin]

Hi @UBarney, I have an implementation that seems to work, but I wrote it as a custom optimizer rule and i'm still new to rust and datafusion. I can share the implementation in a PR and maybe you can help me to refactor the pieces into the correct places? I am not going to be working on this today, so I'll add soon to unblock you (in case you do)

[drin]

I apologize for how messy this is, but I'm out of time until the afternoon. Here's a branch I pushed and a diff: https://github.com/apache/datafusion/compare/main...drin:datafusion:octalene.feat-optimize-datetrunc?expand=1

I don't want to open a PR until I have the time to clean up the description. If you don't want to start with this as context then feel free to ignore it for now.

[UBarney]

Rewriting date_trunc(part, column) <= constant_rhs as column <= date_trunc(part, date_add(constant_rhs, INTERVAL 1 part)) may lead to a bug like this

[drin]

There's a PR to fix that which we found while doing the implementation I shared in the diff above:
duckdb/duckdb#19628

[drin]

sorry, I should be able to clean things up a bit and share a reference PR today if it'd be useful. I know you independently found the mentioned bug, so you may have already made good progress on a PR

[UBarney]

I know you independently found the mentioned bug, so you may have already made good progress on a PR

I've only just started, and I probably won't have time to continue working on it for the next few days.