Skip to content

Constant folding of to_timestamp uses different implementation than actual function call #766

@alamb

Description

@alamb

Describe the bug
A clear and concise description of what the bug is.
Different implementations of string --> timestamp conversions are used if to_timestamp() is called in constant folding pass or in the actual compute kernel

To Reproduce
The function string_to_timestamp_nanos was moved upstream to arrow-rs and is used in constant folding here: https://github.com/apache/arrow-datafusion/blob/ffb195c8634fc537127eff1a082811d28e8fcc2b/datafusion/src/optimizer/constant_folding.rs

use arrow::compute::kernels::cast_utils::string_to_timestamp_nanos;

However, there is still an older version of string_to_timestamp_nanos that is in the datafusion codebase:
https://github.com/apache/arrow-datafusion/blob/5900b4c6829b0bdbe69e1f95fb74e935bc8f33d4/datafusion/src/physical_plan/datetime_expressions.rs#L269

Expected behavior
datafusion should use the same function to convert between string -> timestamps in both places

Additional context
It appears the functions were moved in apache/arrow-rs@db030cd but did not quite clean up the datafusion implementation

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions