Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix 876, allow null array to be cast from and to other types #884

Merged
merged 1 commit into from
Nov 1, 2021
Merged

Fix 876, allow null array to be cast from and to other types #884

merged 1 commit into from
Nov 1, 2021

Conversation

jimexist
Copy link
Member

@jimexist jimexist commented Oct 30, 2021 •
edited
Loading

Which issue does this PR close?

Closes #876
Depends on #591

Rationale for this change

What changes are included in this PR?

Are there any user-facing changes?

@jimexist jimexist changed the title Fix 876 Fix 876, allow null array to be cast to other types Oct 30, 2021
@jimexist jimexist requested a review from alamb October 30, 2021 10:14
@github-actions github-actions bot added the arrow Changes to the arrow crate label Oct 30, 2021
@jimexist jimexist changed the title Fix 876, allow null array to be cast to other types Fix 876, allow null array to be cast from and to other types Oct 30, 2021
alamb
alamb approved these changes Oct 30, 2021
View reviewed changes
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks really nice @jimexist -- thank you

arrow/src/compute/kernels/cast.rs
@@ -946,10 +965,6 @@ pub fn cast_with_options(
}
}
}

// null to primitive/flat types
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this particular special case was added in apache/arrow#8484 by @nevi-me -- I don't think there was any fundamental reason not to include other types, just that it wasn't needed for the particular issue he was dealing with

arrow/src/compute/kernels/cast.rs
let array = Arc::new(NullArray::new(6)) as ArrayRef;
fn test_cast_null_array_from_and_to_others() {
macro_rules! typed_test {
($ARR_TYPE:ident, $DATATYPE:ident, $TYPE:tt) => {{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

looks good

@alamb
Copy link
Contributor

alamb commented Oct 30, 2021

Looks like the tests are failing with dictionaries:

Test casting Dictionary(Int8, Int32) --> Null
thread 'compute::kernels::cast::tests::test_can_cast_types' panicked at 'not implemented: Take not supported for data type Null', arrow/src/compute/kernels/take.rs:283:14


failures:
    compute::kernels::cast::tests::test_can_cast_types

@codecov-commenter
Copy link

Codecov Report

Merging #884 (5143f63) into master (898924f) will increase coverage by 0.02%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #884      +/-   ##
==========================================
+ Coverage   82.45%   82.47%   +0.02%     
==========================================
  Files         168      168              
  Lines       48231    48232       +1     
==========================================
+ Hits        39767    39778      +11     
+ Misses       8464     8454      -10
Impacted Files Coverage Δ
arrow/src/compute/kernels/cast.rs 94.63% <100.00%> (+<0.01%) ⬆️
arrow/src/datatypes/datatype.rs 65.36% <0.00%> (-0.44%) ⬇️
parquet/src/arrow/array_reader.rs 77.72% <0.00%> (-0.09%) ⬇️
arrow/src/ipc/writer.rs 84.28% <0.00%> (+0.18%) ⬆️
arrow/src/array/data.rs 74.93% <0.00%> (+0.81%) ⬆️
arrow/src/array/array.rs 85.71% <0.00%> (+2.33%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 898924f...5143f63. Read the comment docs.

@jimexist jimexist merged commit 81ffa24 into apache:master Nov 1, 2021
@jimexist jimexist deleted the fix-876 branch November 1, 2021 01:14
@alamb alamb mentioned this pull request Nov 2, 2021
Closed
alamb pushed a commit that referenced this pull request Nov 4, 2021
@jimexist @alamb
allow null array to be cased to all other types (#884)
863cc98
@alamb alamb mentioned this pull request Nov 4, 2021
Merged
alamb added a commit that referenced this pull request Nov 5, 2021
@alamb @jimexist
allow null array to be cased to all other types (#884) (#917)
34c5eab 
Co-authored-by: Jiayu Liu <Jimexist@users.noreply.github.com>
@flaneur2020 flaneur2020 mentioned this pull request Nov 9, 2021
Merged
@liukun4515 liukun4515 mentioned this pull request Jun 23, 2022
Merged
mcheshkov pushed a commit to cube-js/arrow-rs that referenced this pull request Aug 26, 2024
@jimexist @mcheshkov
allow null array to be cased to all other types (apache#884)
d257bf2 
Can drop this after rebase on commit 81ffa240, first released in 7.0.0
mcheshkov pushed a commit to cube-js/arrow-rs that referenced this pull request Aug 26, 2024
@jimexist @mcheshkov
allow null array to be cased to all other types (apache#884)
aecac9a 
Can drop this after rebase on commit 81ffa240, first released in 7.0.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alamb alamb alamb approved these changes

Assignees
No one assigned
Labels
arrow Changes to the arrow crate
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Support casting to/from DataType::Null/ NullArray and other types
3 participants
@jimexist @alamb @codecov-commenter