-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PARQUET-2022: ZstdDecompressorStream should close zstdInputStream
#889
Conversation
`ZstdDecompressorStream` should close its resource because `CompressionInputStream.close` closes only the inner stream. ``` public class ZstdDecompressorStream extends CompressionInputStream { private ZstdInputStream zstdInputStream; public ZstdDecompressorStream(InputStream stream) throws IOException { super(stream); zstdInputStream = new ZstdInputStream(stream); } } ```
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, thanks for fixing this @dongjoon-hyun
Thank you, @Fokko ! |
Could you review this, @gszadovszky ? |
Could you review this, @ggershinsky ? |
will do |
Thank you so much, @ggershinsky ! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Thank you for your review and approval, @ggershinsky ! |
Gentle ping, @gszadovszky . |
Thank you, @Fokko , @ggershinsky , @gszadovszky ! |
Thanks @dongjoon-hyun for working on this! cc @vectorijk |
Thank you, @shangxinli . |
* 'master' of https://github.com/apache/parquet-mr: (222 commits) PARQUET-2052: Integer overflow when writing huge binary using dictionary encoding (apache#910) PARQUET-2041: Add zstd to `parquet.compression` description of ParquetOutputFormat Javadoc (apache#899) PARQUET-2050: Expose repetition & definition level from ColumnIO (apache#908) PARQUET-1761: Lower Logging Level in ParquetOutputFormat (apache#745) PARQUET-2046: Upgrade Apache POM to 23 (apache#904) PARQUET-2048: Deprecate BaseRecordReader (apache#906) PARQUET-1922: Deprecate IOExceptionUtils (apache#825) PARQUET-2037: Write INT96 with parquet-avro (apache#901) PARQUET-2044: Enable ZSTD buffer pool by default (apache#903) PARQUET-2038: Upgrade Jackson version used in parquet encryption. (apache#898) Revert "[WIP] Refactor GroupReadSupport to unuse deprecated api (apache#894)" PARQUET-2027: Fix calculating directory offset for merge (apache#896) [WIP] Refactor GroupReadSupport to unuse deprecated api (apache#894) PARQUET-2030: Expose page size row check configurations to ParquetWriter.Builder (apache#895) PARQUET-2031: Upgrade to parquet-format 2.9.0 (apache#897) PARQUET-1448: Review of ParquetFileReader (apache#892) PARQUET-2020: Remove deprecated modules (apache#888) PARQUET-2025: Update Snappy version to 1.1.8.3 (apache#893) PARQUET-2022: ZstdDecompressorStream should close `zstdInputStream` (apache#889) PARQUET-1982: Random access to row groups in ParquetFileReader (apache#871) ... # Conflicts: # parquet-column/src/main/java/org/apache/parquet/example/data/simple/SimpleGroup.java # parquet-hadoop/pom.xml # parquet-hadoop/src/main/java/org/apache/parquet/format/converter/ParquetMetadataConverter.java # parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetFileReader.java
…pache#889) (cherry picked from commit 8c08403) Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
ZstdDecompressorStream
should close its resource becauseCompressionInputStream.close
closes only the inner stream.Make sure you have checked all steps below.
Jira
Tests
Use revised test case.
Commits
Documentation