Fix IN expr for NaN #7378
Merged
Fix IN expr for NaN #7378
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sarutak
commented
Aug 22, 2023
| let batch = RecordBatch::try_new(Arc::new(schema.clone()), vec![Arc::new(a)])?; | ||
|
|
||
| // expression: "a in (0.0, 0.2)" | ||
| // expression: "a in (0.0, 0.1)" |
Member
Author
There was a problem hiding this comment.
Some comments are wrong and it's a tiny issue. So I'll fix them within this PR.
viirya
reviewed
Aug 22, 2023
viirya
reviewed
Aug 22, 2023
| where | ||
| T: ArrayAccessor, | ||
| T::Item: PartialEq + HashValue, | ||
| T::Item: IsEqual + HashValue, |
viirya
reviewed
Aug 22, 2023
Member
viirya
left a comment
viirya
left a comment
There was a problem hiding this comment.
Can we add end-to-end test for this too?
viirya
reviewed
Aug 23, 2023
viirya
reviewed
Aug 23, 2023
| ---- | ||
| 1.2 | ||
| NaN | ||
| NaN |
Member
Author
There was a problem hiding this comment.
It's actually -NaN but just shown as NaN.
I've already noticed this issue and have plan to fix it, but it's a separate issue.
viirya
approved these changes
Aug 23, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes #7377
Rationale for this change
This PR fixes an issue that
'NaN'::double in ('NaN'::double)is evaluated asfalse, which is inconsistent with the result of'NaN'::double = 'NaN'::double.What changes are included in this PR?
The root cause is that equality is evaluated using
PartialEq::eq. So the fix is usingto_bitsfor equality check for float values like asScalarValue::Float64does.NOTE: This issue doesn't happen when
SimplifiersimplifiesINexpr likeAre these changes tested?
Modified an existing test.
Are there any user-facing changes?
Yes, but this change is for correctness.