Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[C++][Parquet] ColumnChunkMetadata: using encoding_stats to build encodings #35757

Closed
mapleFU opened this issue May 25, 2023 · 0 comments · Fixed by #35758
Closed

[C++][Parquet] ColumnChunkMetadata: using encoding_stats to build encodings #35757

mapleFU opened this issue May 25, 2023 · 0 comments · Fixed by #35758
Assignees
@mapleFU
Labels
Component: C++ Component: Parquet Type: enhancement
Milestone
13.0.0

Comments

@mapleFU
Copy link
Member

mapleFU commented May 25, 2023

Describe the enhancement requested

Currently, the logic of building ColumnChunkMetadata is complex, introduce so much unneccessary logics.

Here I'd like to using encoding_stats to build the content of encodings.

Component(s)

C++, Parquet
@github-actions github-actions bot mentioned this issue May 25, 2023
Merged
pitrou added a commit that referenced this issue May 31, 2023
@mapleFU @pitrou @wgtmac
GH-35757: [C++][Parquet] using page-encoding-stats to build encodings (…
fee8bf9 
…#35758)

### Rationale for this change

This patch uses `encoding_stats` to build the encodings. Currently, `ColumnChunk.encodings` is repeated ( May has multiple `PLAIN` ) and having some wired logic by getting `WriterProperties`. And if later we want to support Fallback to non "PLAIN" encoding, `encoding_stats` would be right, but `encoding` would be "PLAIN". So this patch using `encoding_stats` to build the `encoding`, and force them to be consistent.

### What changes are included in this PR?

### Are these changes tested?

Already has test

### Are there any user-facing changes?

Some behavior would be changed, but I think my change is right :)

* Closes: #35757

Lead-authored-by: mwish <maplewish117@gmail.com>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
Co-authored-by: Gang Wu <ustcwg@gmail.com>
Signed-off-by: Antoine Pitrou <antoine@python.org>
@pitrou pitrou added this to the 13.0.0 milestone May 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mapleFU mapleFU

Labels
Component: C++ Component: Parquet Type: enhancement
Projects
None yet
Milestone
13.0.0
Development

Successfully merging a pull request may close this issue.

GH-35757: [C++][Parquet] Build encodings metadata from actual encodings mapleFU/arrow
2 participants
@pitrou @mapleFU