New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use separate ValuesWriterFactory instances for non-default page options when creating ParquetWriter #22719
Conversation
presto-parquet/src/main/java/com/facebook/presto/parquet/writer/ParquetWriters.java
Outdated
Show resolved
Hide resolved
presto-parquet/src/main/java/com/facebook/presto/parquet/writer/ParquetWriters.java
Outdated
Show resolved
Hide resolved
presto-parquet/src/main/java/com/facebook/presto/parquet/writer/ParquetWriters.java
Outdated
Show resolved
Hide resolved
presto-parquet/src/main/java/com/facebook/presto/parquet/writer/ParquetWriter.java
Outdated
Show resolved
Hide resolved
8799e2b
to
09a2c91
Compare
@elharo Thanks for your suggestion, they have been fixed! Please take a look when available. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for fixing this! Just one nit.
presto-parquet/src/main/java/com/facebook/presto/parquet/writer/ParquetWriter.java
Show resolved
Hide resolved
09a2c91
to
4616db9
Compare
Looks good, but commit message is too long. Perhaps reword as follows:
|
The sharing of a single `ValuesWriterFactory` among multiple `ParquetProperties` instances with different write options is not thread-safe and may lead to inconsistency issues.
4616db9
to
65f5d92
Compare
Thanks for the suggestion, fixed! |
Fix #22130
Description
The sharing of a single
ValuesWriterFactory
among multipleParquetProperties
instances with different write options is not thread-safe and may lead to inconsistency issues.This PR set a new separate
ValuesWriterFactory
instance intoParquetProperties
on its creation when using a non-default writer options, so that it could work well in multi-thread environment.Test Plan
Contributor checklist
Release Notes