[HUDI-9273] Upgrade Parquet to 1.15.1 to address CVE-2025-30065#13315
[HUDI-9273] Upgrade Parquet to 1.15.1 to address CVE-2025-30065#13315adith-os wants to merge 2 commits intoapache:masterfrom
Conversation
|
Please review and guide on better approaches or any changes required for other modules for this update. Thanks! |
| bringing these file-formats as dependencies as well, we need to make sure that versions are | ||
| synchronized to avoid classpath ambiguity --> | ||
| <parquet.version>1.13.1</parquet.version> | ||
| <parquet.version>1.15.1</parquet.version> |
There was a problem hiding this comment.
is the version backward compatible?
There was a problem hiding this comment.
I couldn't figure out or find any mention of backward compatibility for this version with older versions. However, I noticed that adding this version does cause failures because now it is dependent on other parquet packages
There was a problem hiding this comment.
yeah, but still, thanks for the contribution, is the fix a blocker for you?
There was a problem hiding this comment.
No problem. It is not a blocker for me, I could fix errors like the below one by shading the required parquet packages for spark-bundle for my usecase. I wanted to contribute the same change but I see the same error in the tests which is a bit confusing 😅
recent failure: Lost task 0.3 in stage 12.0 (TID 26) (172.30.95.61 executor 1): java.lang.NoClassDefFoundError: org.apache.parquet.conf.HadoopParquetConfiguration
at org.apache.parquet.avro.AvroSchemaConverter.<init>(AvroSchemaConverter.java:111)
danny0405
added
version-upgrade
issue:version-compatibility
| bringing these file-formats as dependencies as well, we need to make sure that versions are | ||
| synchronized to avoid classpath ambiguity --> | ||
| <parquet.version>1.12.2</parquet.version> | ||
| <parquet.version>1.15.1</parquet.version> |
There was a problem hiding this comment.
Unfortunately this cannot be changed and the parquet version has to be aligned with What Spark uses in each Spark version. Otherwise there can be compatibility issues.
There was a problem hiding this comment.
Okay. Thank you for the review!
4 participants
Change Logs
This code change is made to update jetty version to fix CVE-2025-30065. Changing the version caused failures for application runs for Apache spark where it was manually tested due to change in the dependencies required for parquet-avro for version 1.15.1. This is currently resolved by shading the required dependencies.
Impact
Fixes the vulnerability
Risk level (write none, low medium or high below)
Low
Documentation Update
None
Contributor's checklist