Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Return plan error when adding utf8 and timestamp #5696

Merged
merged 1 commit into from
Mar 23, 2023

Conversation

Weijun-H
Copy link
Contributor

Which issue does this PR close?

Closes #5650

Rationale for this change

What changes are included in this PR?

Return plan error when performing binary operations between timestamp and utf8

Are these changes tested?

Are there any user-facing changes?

@github-actions github-actions bot added core Core datafusion crate physical-expr Physical Expressions sqllogictest labels Mar 22, 2023
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick fix @Weijun-H -- while the behavior is still not ideal I think this PR is an improvement (as the error is no longer internal) 👍

Longer term I think we will need to improve interval arithmetic but that will be a larger project (one I hope to sketch out more later today)

@@ -1213,6 +1214,13 @@ pub fn binary(
) -> Result<Arc<dyn PhysicalExpr>> {
let lhs_type = &lhs.data_type(input_schema)?;
let rhs_type = &rhs.data_type(input_schema)?;
if (is_utf8_or_large_utf8(lhs_type) && is_timestamp(rhs_type))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Weijun-H -- this does indeed fix the bug as stated.

I was trying to figure out how this code even executes (and was not caught earlier)

It seems like maybe this could be related to

https://github.com/apache/arrow-datafusion/blob/d8925bebc1f5382b044ce80f78bc465a9145e425/datafusion/expr/src/type_coercion/binary.rs#L80-L85

🤔

Maybe it is time to figure out a better coercion story

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 sometime ago we agreed to that logical planner will be a better place for coercions and coercion checks

@alamb alamb merged commit b6fa286 into apache:main Mar 23, 2023
@Weijun-H Weijun-H deleted the plan-error-for-utf8-timestamp branch March 23, 2023 23:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Core datafusion crate physical-expr Physical Expressions sqllogictest
Projects
None yet
3 participants