Skip to content

Add some missing optional fields to the Parquet RowGroup metadata - #15421

Merged
rapids-bot[bot] merged 25 commits into
NVIDIA:branch-24.06from
etseidl:rowgroup_fields
Apr 23, 2024
Merged

Add some missing optional fields to the Parquet RowGroup metadata#15421
rapids-bot[bot] merged 25 commits into
NVIDIA:branch-24.06from
etseidl:rowgroup_fields

Conversation

@etseidl

@etseidl etseidl commented Apr 1, 2024

Copy link
Copy Markdown
Contributor

Description

This PR adds the sorting_columns, file_offset, total_compressed_size, and ordinal optional fields to the Parquet RowGroup metadata object.

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@etseidl
etseidl requested a review from a team as a code owner April 1, 2024 20:48
@copy-pr-bot

copy-pr-bot Bot commented Apr 1, 2024

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions github-actions Bot added the libcudf Affects libcudf (C++/CUDA) code. label Apr 1, 2024

@hyperbolic2346 hyperbolic2346 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good. Some questions/comments.

Comment thread cpp/include/cudf/io/parquet.hpp
Comment thread cpp/include/cudf/io/parquet.hpp Outdated
Comment thread cpp/src/io/parquet/parquet.hpp
Comment thread cpp/src/io/parquet/parquet.hpp
Comment thread cpp/src/io/parquet/writer_impl.cu Outdated
@hyperbolic2346 hyperbolic2346 added 3 - Ready for Review Ready for review by team cuIO cuIO issue non-breaking Non-breaking change labels Apr 2, 2024
@hyperbolic2346

Copy link
Copy Markdown
Contributor

/ok to test

@hyperbolic2346 hyperbolic2346 added the feature request New feature or request label Apr 2, 2024
@hyperbolic2346

Copy link
Copy Markdown
Contributor

/ok to test

@hyperbolic2346

Copy link
Copy Markdown
Contributor

/ok to test

@hyperbolic2346

Copy link
Copy Markdown
Contributor

/ok to test

@hyperbolic2346

Copy link
Copy Markdown
Contributor

/ok to test

@etseidl

etseidl commented Apr 23, 2024

Copy link
Copy Markdown
Contributor Author

@vuule could you take a look at this one? 🙏 Mike's gonna get tired of "ok to test"ing it 🤣

@vuule
vuule self-requested a review April 23, 2024 21:22

@vuule vuule left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a sad day for us pickers... a nit-less PR

@vuule

vuule commented Apr 23, 2024

Copy link
Copy Markdown
Collaborator

/merge

@rapids-bot
rapids-bot Bot merged commit 6780e59 into NVIDIA:branch-24.06 Apr 23, 2024
@etseidl
etseidl deleted the rowgroup_fields branch April 23, 2024 21:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

3 - Ready for Review Ready for review by team cuIO cuIO issue feature request New feature or request libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants