Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARROW-5821: [Java] Support compact fixed-width vectors #4789

Closed
wants to merge 1 commit into from

Conversation

tianchen92
Copy link
Contributor

Related to ARROW-5821.
In shuffle stage of some applications, FixedWitdhVectors may have very little non-null data.
In this case, directly serialize vectors is not a good choice, generally we can compact the vector make it only holding non-null value and create a BitVector to trace the indices for non-null values so that it could be deserialized properly.

@codecov-io
Copy link

Codecov Report

Merging #4789 into master will increase coverage by 17.33%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #4789       +/-   ##
==========================================
+ Coverage   72.26%   89.6%   +17.33%     
==========================================
  Files         822     662      -160     
  Lines      107713   96352    -11361     
  Branches     1418       0     -1418     
==========================================
+ Hits        77844   86332     +8488     
+ Misses      29507   10020    -19487     
+ Partials      362       0      -362
Impacted Files Coverage Δ
cpp/src/arrow/testing/gtest_util.h 97.36% <0%> (-2.64%) ⬇️
cpp/src/arrow/compute/kernel.h 61.16% <0%> (-1.53%) ⬇️
cpp/src/arrow/result.h 91.3% <0%> (-0.37%) ⬇️
cpp/src/arrow/flight/server.h 100% <0%> (ø) ⬆️
cpp/src/gandiva/expression_registry.h 100% <0%> (ø) ⬆️
cpp/src/arrow/csv/column-builder.h 100% <0%> (ø) ⬆️
cpp/src/arrow/table_builder.h 100% <0%> (ø) ⬆️
cpp/src/plasma/plasma.h 100% <0%> (ø) ⬆️
cpp/src/gandiva/configuration.h 100% <0%> (ø) ⬆️
cpp/src/arrow/compute/kernels/cast.h 100% <0%> (ø) ⬆️
... and 746 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9ffb9cd...a1ef69c. Read the comment docs.

@emkornfield
Copy link
Contributor

We should maybe close this until we decide if this should be support as part of the encoding/compression discussions?

@tianchen92 tianchen92 closed this Jul 16, 2019
@tianchen92
Copy link
Contributor Author

We should maybe close this until we decide if this should be support as part of the encoding/compression discussions?

Sure, closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants