-
Notifications
You must be signed in to change notification settings - Fork 682
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Expose column index and offset index in ColumnChunkMetadata
#1318
Expose column index and offset index in ColumnChunkMetadata
#1318
Conversation
Codecov Report
@@ Coverage Diff @@
## master #1318 +/- ##
==========================================
- Coverage 83.01% 82.98% -0.04%
==========================================
Files 180 180
Lines 52810 52866 +56
==========================================
+ Hits 43840 43870 +30
- Misses 8970 8996 +26
Continue to review full report at Codecov.
|
Will sync with |
#1320 has been merged FYI |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me -- thanks @shanisolomon
cc @sunchao (mostly for visibility that the Rust implementation is beginning down the path of using more advanced parquet metadata features)
Thanks, @alamb! Could you help with merging please? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @alamb @shanisolomon ! yes it certainly will be VERY useful if parquet-rs
has these advanced predicate pushdown features!
Just merged to master! |
ColumnChunkMetadata
Which issue does this PR close?
Closes #1317.
Exposing the column index and offset index offsets and lengths so parquet engines could optimize their reads.