-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Parquet: Release the compressor when calling close in ParquetWriter #5126
Conversation
I discovered this after switching to |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The commit message should have prefix "Parquet: " instead of "Core: ".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1. Good catch @hmit!
I tried to find a place where this could be tested, but there didn't seem to be anywhere that was already testing whether or not things were closed so I think it's fine without it.
Yeah, I didn't find a good unit-test to add this to. However, we did test it by deploying (with brotli compression) and it seems to do the trick. |
@kbendick Could you please merge this PR? |
Thanks all for the quick review and merge! |
Thanks @hmit for discovering the issue and providing a fix. Thanks @kbendick for review. Thanks @szehon-ho for review and merge! |
Sorry I couldn't merge this. I don't have that permission yet (hopefully soon). But glad this is in and appreciate the contributions 👍 |
… in Flink iceberg connector)
… in Flink iceberg connector) (#40)
No description provided.