Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cherry pick feat(ipc): Support writing dictionaries nested in structs and unions to active_release #915

Merged
merged 1 commit into from
Nov 5, 2021

Conversation

alamb
Copy link
Contributor

@alamb alamb commented Nov 4, 2021

Automatic cherry-pick of c7cf8f7

…870)

* feat(ipc): Support for writing dictionaries nested in structs and unions

Dictionaries are lost when serializing a RecordBatch for IPC, producing
invalid arrow data. This PR changes encoded_batch to recursively find
all dictionary fields within the schema (currently only in structs and
unions) so nested dictionaries are properly serialized.

* address lint and clippy
@github-actions github-actions bot added the arrow Changes to the arrow crate label Nov 4, 2021
@codecov-commenter
Copy link

codecov-commenter commented Nov 4, 2021

Codecov Report

Merging #915 (86c85db) into active_release (03d95e6) will decrease coverage by 0.15%.
The diff coverage is 71.42%.

❗ Current head 86c85db differs from pull request most recent head f8215f9. Consider uploading reports for the commit f8215f9 to get more accurate results
Impacted file tree graph

@@                Coverage Diff                 @@
##           active_release     #915      +/-   ##
==================================================
- Coverage           82.45%   82.30%   -0.16%     
==================================================
  Files                 168      168              
  Lines               48175    48147      -28     
==================================================
- Hits                39725    39627      -98     
- Misses               8450     8520      +70     
Impacted Files Coverage Δ
arrow/src/array/cast.rs 85.71% <ø> (ø)
arrow/src/ipc/writer.rs 84.12% <71.42%> (-1.90%) ⬇️
arrow/src/array/transform/fixed_binary.rs 75.00% <0.00%> (-3.95%) ⬇️
parquet/src/schema/visitor.rs 67.10% <0.00%> (-2.64%) ⬇️
arrow/src/compute/kernels/cast_utils.rs 87.50% <0.00%> (-2.50%) ⬇️
parquet/src/encodings/decoding.rs 90.45% <0.00%> (-2.02%) ⬇️
arrow/src/datatypes/ffi.rs 70.85% <0.00%> (-2.02%) ⬇️
parquet/src/arrow/arrow_reader.rs 89.32% <0.00%> (-1.95%) ⬇️
arrow/src/ffi.rs 84.69% <0.00%> (-1.89%) ⬇️
arrow/src/csv/reader.rs 90.00% <0.00%> (-1.78%) ⬇️
... and 71 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 03d95e6...f8215f9. Read the comment docs.

@alamb alamb merged commit 8540214 into active_release Nov 5, 2021
@alamb alamb deleted the cherry_pick_c7cf8f77 branch November 5, 2021 10:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arrow Changes to the arrow crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants