Skip to content
This repository has been archived by the owner on Feb 18, 2024. It is now read-only.

Added operators that include validities in comparisons #846

Merged
merged 3 commits into from Feb 16, 2022

Conversation

ritchie46
Copy link
Collaborator

This PR adds equality kernels that allow you to include validities in comparisons. Rationale for this is to be able to have the same behavior as numpy:

>>> np.array([1.0, np.nan, 2.0]) == np.array([1.0, 1.0, 2.0])
array([ True, False,  True])

The default kernels propagate the null values and would return [true, null, true].

@codecov
Copy link

codecov bot commented Feb 16, 2022

Codecov Report

Merging #846 (b04fa7e) into main (1f9c94d) will decrease coverage by 0.09%.
The diff coverage is 56.48%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #846      +/-   ##
==========================================
- Coverage   71.53%   71.44%   -0.10%     
==========================================
  Files         336      336              
  Lines       17800    17931     +131     
==========================================
+ Hits        12734    12811      +77     
- Misses       5066     5120      +54     
Impacted Files Coverage Δ
src/compute/comparison/binary.rs 44.26% <0.00%> (-28.72%) ⬇️
src/compute/comparison/boolean.rs 77.63% <51.85%> (-14.21%) ⬇️
src/compute/comparison/primitive.rs 88.37% <58.33%> (-11.63%) ⬇️
src/compute/comparison/mod.rs 55.78% <68.75%> (+6.58%) ⬆️
src/compute/comparison/utf8.rs 83.60% <100.00%> (+10.63%) ⬆️
src/array/utf8/mutable.rs 81.60% <0.00%> (+0.57%) ⬆️
src/compute/arithmetics/time.rs 26.60% <0.00%> (+0.91%) ⬆️
src/bitmap/utils/slice_iterator.rs 87.93% <0.00%> (+1.72%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1f9c94d...b04fa7e. Read the comment docs.

@ritchie46 ritchie46 force-pushed the no_null_propagate_comp branch 2 times, most recently from bbf5b7c to 89185f0 Compare February 16, 2022 12:45
@jorgecarleitao jorgecarleitao merged commit 6ccb154 into jorgecarleitao:main Feb 16, 2022
@jorgecarleitao jorgecarleitao changed the title include validities in comparisons Added operators that include validities in comparisons Feb 16, 2022
@jorgecarleitao jorgecarleitao added the feature A new feature label Feb 16, 2022
dexterduck pushed a commit to mindx/arrow2 that referenced this pull request Mar 2, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
feature A new feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants