Skip to content

Add Utf8->Binary type coercion for comparison #7080

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
alamb merged 4 commits into from
Jul 25, 2023
Merged

Add Utf8->Binary type coercion for comparison #7080

alamb merged 4 commits into from
Jul 25, 2023

Conversation

@jonahgao
Copy link
Member

@jonahgao jonahgao commented Jul 25, 2023 •
edited
Loading

Which issue does this PR close?

Partial fix for #7039.

Closes #7039

Rationale for this change

The following casts should be safe:

  • Utf8 -> Binary
  • Utf8 | LargetUtf8 | Binary -> LargeBinary

What changes are included in this PR?

Are these changes tested?

Yes

Are there any user-facing changes?

No

@github-actions github-actions bot added the logical-expr Logical plan and expressions label Jul 25, 2023
@jonahgao jonahgao mentioned this pull request Jul 25, 2023
Closed
@github-actions github-actions bot added core Core DataFusion crate sqllogictest SQL Logic Tests (.slt) labels Jul 25, 2023
alamb
alamb approved these changes Jul 25, 2023
View reviewed changes
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jonahgao ❤️

I took the liberty of merging up from main to get a fix for CI failures and pushed a small sql level test in sqllogictests

jackwener
jackwener approved these changes Jul 25, 2023
View reviewed changes
Copy link
Member

@jackwener jackwener left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @jonahgao

jackwener
jackwener reviewed Jul 25, 2023
View reviewed changes
datafusion/expr/src/type_coercion/binary.rs Outdated
Comment on lines 776 to 782
(LargeBinary, LargeBinary)
| (Binary, LargeBinary)
| (LargeBinary, Binary)
| (Utf8, LargeBinary)
| (LargeBinary, Utf8)
| (LargeUtf8, LargeBinary)
| (LargeBinary, LargeUtf8) => Some(LargeBinary),
Copy link
Member

@jackwener jackwener Jul 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can use

(LargeBinary, Utf8 | LargeUtf8 | Binary | LargeBinary) | (Utf8 | LargeUtf8 | Binary | LargeBinary, LargeBinary)

Copy link
Member Author

@jonahgao jonahgao Jul 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated. Thank you @jackwener ❤️

@jonahgao
Copy link
Member Author

jonahgao commented Jul 25, 2023

Thanks @jonahgao ❤️

I took the liberty of merging up from main to get a fix for CI failures and pushed a small sql level test in sqllogictests

Thank you @alamb ❤️

@alamb alamb merged commit 52cf58b into apache:main Jul 25, 2023
@jonahgao jonahgao deleted the binary_type_coercion branch July 26, 2023 01:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jackwener jackwener jackwener approved these changes

@alamb alamb alamb approved these changes

Assignees

No one assigned

Labels

core Core DataFusion crate logical-expr Logical plan and expressions sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Error with partitioned dataset: Cannot infer common argument type for comparison operation Binary != Utf8

3 participants

@jonahgao @alamb @jackwener