[C++][Parquet] Writer cannot add extra key-value metadata once opened #34888
Comments
wgtmac
added a commit
to wgtmac/arrow
that referenced
this issue
Apr 4, 2023
wgtmac
added a commit
to wgtmac/arrow
that referenced
this issue
Apr 6, 2023
wgtmac
added a commit
to wgtmac/arrow
that referenced
this issue
Apr 6, 2023
wgtmac
added a commit
to wgtmac/arrow
that referenced
this issue
Apr 11, 2023
wjones127
added a commit
that referenced
this issue
Apr 13, 2023
### Rationale for this change Parquet specs support storing key-value metadata provided by the user. However, the parquet-cpp writer can only set it via ParquetFileWriter::Open(). Sometimes user may want to add extra information to it while writing. So it is good to support adding extra key-value metadata any time before closing the file writer. ### What changes are included in this PR? Add a new interface `void AddKeyValueMetadata(std::shared_ptr<const KeyValueMetadata> key_value_metadata)` to the `ParquetFileWriter` class. User can now add more key-value metadata to the file if not closed. ### Are these changes tested? Added a new `Metadata.TestAddKeyValueMetadata` test to verify key-value metadata added before closing the writer are well preserved. ### Are there any user-facing changes? Yes, user can add custom key-value metadata whenever writer is not closed. * Closes: #34888 Lead-authored-by: Gang Wu <ustcwg@gmail.com> Co-authored-by: Will Jones <willjones127@gmail.com> Signed-off-by: Will Jones <willjones127@gmail.com>
raulcd
pushed a commit
that referenced
this issue
Apr 17, 2023
### Rationale for this change Parquet specs support storing key-value metadata provided by the user. However, the parquet-cpp writer can only set it via ParquetFileWriter::Open(). Sometimes user may want to add extra information to it while writing. So it is good to support adding extra key-value metadata any time before closing the file writer. ### What changes are included in this PR? Add a new interface `void AddKeyValueMetadata(std::shared_ptr<const KeyValueMetadata> key_value_metadata)` to the `ParquetFileWriter` class. User can now add more key-value metadata to the file if not closed. ### Are these changes tested? Added a new `Metadata.TestAddKeyValueMetadata` test to verify key-value metadata added before closing the writer are well preserved. ### Are there any user-facing changes? Yes, user can add custom key-value metadata whenever writer is not closed. * Closes: #34888 Lead-authored-by: Gang Wu <ustcwg@gmail.com> Co-authored-by: Will Jones <willjones127@gmail.com> Signed-off-by: Will Jones <willjones127@gmail.com>
liujiacheng777
pushed a commit
to LoongArch-Python/arrow
that referenced
this issue
May 11, 2023
…pache#34889) ### Rationale for this change Parquet specs support storing key-value metadata provided by the user. However, the parquet-cpp writer can only set it via ParquetFileWriter::Open(). Sometimes user may want to add extra information to it while writing. So it is good to support adding extra key-value metadata any time before closing the file writer. ### What changes are included in this PR? Add a new interface `void AddKeyValueMetadata(std::shared_ptr<const KeyValueMetadata> key_value_metadata)` to the `ParquetFileWriter` class. User can now add more key-value metadata to the file if not closed. ### Are these changes tested? Added a new `Metadata.TestAddKeyValueMetadata` test to verify key-value metadata added before closing the writer are well preserved. ### Are there any user-facing changes? Yes, user can add custom key-value metadata whenever writer is not closed. * Closes: apache#34888 Lead-authored-by: Gang Wu <ustcwg@gmail.com> Co-authored-by: Will Jones <willjones127@gmail.com> Signed-off-by: Will Jones <willjones127@gmail.com>
ArgusLi
pushed a commit
to Bit-Quill/arrow
that referenced
this issue
May 15, 2023
…pache#34889) ### Rationale for this change Parquet specs support storing key-value metadata provided by the user. However, the parquet-cpp writer can only set it via ParquetFileWriter::Open(). Sometimes user may want to add extra information to it while writing. So it is good to support adding extra key-value metadata any time before closing the file writer. ### What changes are included in this PR? Add a new interface `void AddKeyValueMetadata(std::shared_ptr<const KeyValueMetadata> key_value_metadata)` to the `ParquetFileWriter` class. User can now add more key-value metadata to the file if not closed. ### Are these changes tested? Added a new `Metadata.TestAddKeyValueMetadata` test to verify key-value metadata added before closing the writer are well preserved. ### Are there any user-facing changes? Yes, user can add custom key-value metadata whenever writer is not closed. * Closes: apache#34888 Lead-authored-by: Gang Wu <ustcwg@gmail.com> Co-authored-by: Will Jones <willjones127@gmail.com> Signed-off-by: Will Jones <willjones127@gmail.com>
rtpsw
pushed a commit
to rtpsw/arrow
that referenced
this issue
May 16, 2023
…pache#34889) ### Rationale for this change Parquet specs support storing key-value metadata provided by the user. However, the parquet-cpp writer can only set it via ParquetFileWriter::Open(). Sometimes user may want to add extra information to it while writing. So it is good to support adding extra key-value metadata any time before closing the file writer. ### What changes are included in this PR? Add a new interface `void AddKeyValueMetadata(std::shared_ptr<const KeyValueMetadata> key_value_metadata)` to the `ParquetFileWriter` class. User can now add more key-value metadata to the file if not closed. ### Are these changes tested? Added a new `Metadata.TestAddKeyValueMetadata` test to verify key-value metadata added before closing the writer are well preserved. ### Are there any user-facing changes? Yes, user can add custom key-value metadata whenever writer is not closed. * Closes: apache#34888 Lead-authored-by: Gang Wu <ustcwg@gmail.com> Co-authored-by: Will Jones <willjones127@gmail.com> Signed-off-by: Will Jones <willjones127@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the enhancement requested
Parquet specs support storing key-value metadata provided by the user. However, the parquet-cpp writer can only set it via
ParquetFileWriter::Open(). Sometimes user may want to add extra information to it while writing. So it is good to support adding extra key-value metadata any time before closing the file writer.Component(s)
C++, Parquet
The text was updated successfully, but these errors were encountered: