[SPARK-58869][BUILD] Upgrade Parquet to 1.18.0 - #58120
Open
iemejia wants to merge 1 commit into
Open
Conversation
Member
|
@Fokko opened a PR (#57406) for this a month ago, but has had no response or follow-up since then. In addition, I see there is a data corruption report raised in the Parquet dev mailing list https://lists.apache.org/thread/zy4ox06ocjo4c7jm76xddvbgymzcjsmt, given that, maybe we should hold this and wait for Parquet 1.18.1? |
Member
Author
|
Sorry, I hadn't noticed @Fokko's PR. My version is slightly more comprehensive, but I'm fine with using the other one. The important part is to wait for the 1.18.1 release, which contains the fix for this issue. |
uros-b
approved these changes
Aug 19, 2026
Member
|
oh I only now see the comments above, I'm fine with your alignment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
This PR upgrades Apache Parquet from
1.17.1to1.18.0. It bumps theparquet.versionproperty inpom.xmland updates the dependency manifestdev/deps/spark-deps-hadoop-3-hive-2.3accordingly.Why are the changes needed?
To keep the Parquet dependency up to date and pick up the bug fixes and
improvements shipped in the
1.18.0release.Release notes: https://github.com/apache/parquet-java/releases/tag/apache-parquet-1.18.0
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Existing tests via CI. This is a dependency version bump with the
dependency manifest updated accordingly.
Was this patch authored or co-authored using generative AI tooling?
No.