Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[C++][Parquet] Support encryption for page index #34950

Closed
wgtmac opened this issue Apr 7, 2023 · 0 comments · Fixed by #36574
Closed

[C++][Parquet] Support encryption for page index #34950

wgtmac opened this issue Apr 7, 2023 · 0 comments · Fixed by #36574
Assignees
@wgtmac
Labels
Component: C++ Component: Parquet Type: enhancement
Milestone
14.0.0

Comments

@wgtmac
Copy link
Member

wgtmac commented Apr 7, 2023 •
edited
Loading

Describe the enhancement requested

Page index supports modular encryption as advised by parquet specs. Once #34054 is merged, we should support encryption so page index can be used in a wider scope.

Component(s)

C++, Parquet
@wgtmac wgtmac changed the title [C++][Parquet] [C++][Parquet] Do not write page statistics once page index is enabled Apr 7, 2023
@wgtmac wgtmac changed the title [C++][Parquet] Do not write page statistics once page index is enabled [C++][Parquet] Support encryption for page index Apr 7, 2023
wgtmac added a commit to wgtmac/arrow that referenced this issue Jul 9, 2023
@wgtmac
apacheGH-34950: [C++][Parquet] Support encryption for page index
f071970
@github-actions github-actions bot mentioned this issue Jul 9, 2023
Merged
wgtmac added a commit to wgtmac/arrow that referenced this issue Jul 9, 2023
@wgtmac
apacheGH-34950: [C++][Parquet] Support encryption for page index
63592a5
wgtmac added a commit to wgtmac/arrow that referenced this issue Jul 16, 2023
@wgtmac
apacheGH-34950: [C++][Parquet] Support encryption for page index
be9dddc
wgtmac added a commit to wgtmac/arrow that referenced this issue Aug 15, 2023
@wgtmac
apacheGH-34950: [C++][Parquet] Support encryption for page index
6741c72
wgtmac added a commit to wgtmac/arrow that referenced this issue Aug 31, 2023
@wgtmac
apacheGH-34950: [C++][Parquet] Support encryption for page index
92fe19a
wgtmac added a commit to wgtmac/arrow that referenced this issue Sep 20, 2023
@wgtmac
apacheGH-34950: [C++][Parquet] Support encryption for page index
eba79d5
wgtmac added a commit to wgtmac/arrow that referenced this issue Sep 28, 2023
@wgtmac
apacheGH-34950: [C++][Parquet] Support encryption for page index
c6a9821
pitrou pushed a commit that referenced this issue Sep 28, 2023
@wgtmac
GH-34950: [C++][Parquet] Support encryption for page index (#36574)
853d849 
### Rationale for this change

Parquet modular encryption requires page index to be encrypted if the column chunk is encrypted. This feature is missing for now.

### What changes are included in this PR?

Support both encryption and decryption for parquet page index.

### Are these changes tested?

Added round trip tests in write_configurations_test.cc and read_configurations_test.cc.

### Are there any user-facing changes?

NO.
* Closes: #34950

Authored-by: Gang Wu <ustcwg@gmail.com>
Signed-off-by: Antoine Pitrou <antoine@python.org>
@pitrou pitrou added this to the 14.0.0 milestone Sep 28, 2023
etseidl pushed a commit to etseidl/arrow that referenced this issue Sep 28, 2023
@wgtmac @etseidl
apacheGH-34950: [C++][Parquet] Support encryption for page index (apa…
51e83dd 
…che#36574)

### Rationale for this change

Parquet modular encryption requires page index to be encrypted if the column chunk is encrypted. This feature is missing for now.

### What changes are included in this PR?

Support both encryption and decryption for parquet page index.

### Are these changes tested?

Added round trip tests in write_configurations_test.cc and read_configurations_test.cc.

### Are there any user-facing changes?

NO.
* Closes: apache#34950

Authored-by: Gang Wu <ustcwg@gmail.com>
Signed-off-by: Antoine Pitrou <antoine@python.org>
JerAguilon pushed a commit to JerAguilon/arrow that referenced this issue Oct 23, 2023
@wgtmac
apacheGH-34950: [C++][Parquet] Support encryption for page index (apa…
511ee52 
…che#36574)

### Rationale for this change

Parquet modular encryption requires page index to be encrypted if the column chunk is encrypted. This feature is missing for now.

### What changes are included in this PR?

Support both encryption and decryption for parquet page index.

### Are these changes tested?

Added round trip tests in write_configurations_test.cc and read_configurations_test.cc.

### Are there any user-facing changes?

NO.
* Closes: apache#34950

Authored-by: Gang Wu <ustcwg@gmail.com>
Signed-off-by: Antoine Pitrou <antoine@python.org>
JerAguilon pushed a commit to JerAguilon/arrow that referenced this issue Oct 23, 2023
@wgtmac
apacheGH-34950: [C++][Parquet] Support encryption for page index (apa…
12a5ef3 
…che#36574)

### Rationale for this change

Parquet modular encryption requires page index to be encrypted if the column chunk is encrypted. This feature is missing for now.

### What changes are included in this PR?

Support both encryption and decryption for parquet page index.

### Are these changes tested?

Added round trip tests in write_configurations_test.cc and read_configurations_test.cc.

### Are there any user-facing changes?

NO.
* Closes: apache#34950

Authored-by: Gang Wu <ustcwg@gmail.com>
Signed-off-by: Antoine Pitrou <antoine@python.org>
loicalleyne pushed a commit to loicalleyne/arrow that referenced this issue Nov 13, 2023
@wgtmac @loicalleyne
apacheGH-34950: [C++][Parquet] Support encryption for page index (apa…
c472e10 
…che#36574)

### Rationale for this change

Parquet modular encryption requires page index to be encrypted if the column chunk is encrypted. This feature is missing for now.

### What changes are included in this PR?

Support both encryption and decryption for parquet page index.

### Are these changes tested?

Added round trip tests in write_configurations_test.cc and read_configurations_test.cc.

### Are there any user-facing changes?

NO.
* Closes: apache#34950

Authored-by: Gang Wu <ustcwg@gmail.com>
Signed-off-by: Antoine Pitrou <antoine@python.org>
dgreiss pushed a commit to dgreiss/arrow that referenced this issue Feb 19, 2024
@wgtmac @dgreiss
apacheGH-34950: [C++][Parquet] Support encryption for page index (apa…
b87cbb2 
…che#36574)

### Rationale for this change

Parquet modular encryption requires page index to be encrypted if the column chunk is encrypted. This feature is missing for now.

### What changes are included in this PR?

Support both encryption and decryption for parquet page index.

### Are these changes tested?

Added round trip tests in write_configurations_test.cc and read_configurations_test.cc.

### Are there any user-facing changes?

NO.
* Closes: apache#34950

Authored-by: Gang Wu <ustcwg@gmail.com>
Signed-off-by: Antoine Pitrou <antoine@python.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@wgtmac wgtmac

Labels
Component: C++ Component: Parquet Type: enhancement
Projects
None yet
Milestone
14.0.0
Development

Successfully merging a pull request may close this issue.

GH-34950: [C++][Parquet] Support encryption for page index wgtmac/arrow
2 participants
@pitrou @wgtmac