Skip to content

Explicit dependencies in pxf-hdfs#122

Merged
MisterRaindrop merged 3 commits into
mainfrom
explicit-dependencies
Jul 10, 2026
Merged

Explicit dependencies in pxf-hdfs#122
MisterRaindrop merged 3 commits into
mainfrom
explicit-dependencies

Conversation

@ostinru

@ostinru ostinru commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator

I just found out that in our fork (legacy-pxf) we have broken dependences. I was curious why we don't seen an issue in tests. And, I realised that parquet-pig transitively pulls couple compressors to testRuntimeClasspath and hides the fact that pxf-hdfs doesn't have snappy (and zstd in my fork) as direct runtime dependency.

+--- org.apache.parquet:parquet-pig -> 1.15.2
    +--- org.apache.parquet:parquet-hadoop -> 1.15.2
    |    +--- org.apache.parquet:parquet-column:1.15.2 (*)
    |    +--- org.apache.parquet:parquet-format-structures:1.15.2 (*)
    |    +--- org.apache.parquet:parquet-common:1.15.2 (*)
    |    +--- org.apache.parquet:parquet-jackson:1.15.2
    |    +--- org.xerial.snappy:snappy-java:1.1.10.7
    |    +--- io.airlift:aircompressor:2.0.2
    |    +--- commons-pool:commons-pool:1.6
    |    +--- com.github.luben:zstd-jni:1.5.6-6 -> 1.5.7-6
    |    \--- org.slf4j:slf4j-api:1.7.33 -> 1.7.36

So, this PR removes transitive test-dependencies and explicitly adds runtime dependencies

@ostinru
ostinru marked this pull request as ready for review July 7, 2026 11:58
@tuhaihe
tuhaihe requested a review from MisterRaindrop July 10, 2026 02:15
@MisterRaindrop
MisterRaindrop merged commit ed7c857 into main Jul 10, 2026
52 of 58 checks passed
@MisterRaindrop
MisterRaindrop deleted the explicit-dependencies branch July 10, 2026 02:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants