Skip to content

Commit

Permalink
[SPARK-47038][BUILD] Remove shaded protobuf-java 2.6.1 dependency f…
Browse files Browse the repository at this point in the history
…rom `kinesis-asl-assembly`

### What changes were proposed in this pull request?

This PR aims to remove the shaded `protobuf-java` 2.6.1 dependency from `kinesis-asl-assembly`.

### Why are the changes needed?

Technically, this is a revert of SPARK-14421.
- At that time, Apache Spark 2.0.0 distribution contains `protobuf-java-2.5.0.jar`.
  - apache#13054
- SPARK-43489 removed `protobuf-java-2.5.0.jar` from Apache Spark 3.5+ distribution.
  - apache#41153

### Does this PR introduce _any_ user-facing change?

No, but the user can inject the required protobuf-jar from application layers.

### How was this patch tested?

Pass the CIs and manual review.

**BEFORE**
```
$ build/mvn dependency:tree -pl connector/kinesis-asl-assembly -Pkinesis-asl | grep protobuf-java
...
[INFO] +- com.google.protobuf:protobuf-java:jar:2.6.1:provided
```

**AFTER**
```
$ build/mvn dependency:tree -pl connector/kinesis-asl-assembly -Pkinesis-asl | grep protobuf-java
...
[INFO] |  |  +- com.google.protobuf:protobuf-java:jar:3.25.1:provided
```

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes apache#45096 from dongjoon-hyun/SPARK-47038.

Authored-by: Dongjoon Hyun <dhyun@apple.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
  • Loading branch information
dongjoon-hyun committed Feb 14, 2024
1 parent 7e911cd commit c1321c0
Showing 1 changed file with 0 additions and 19 deletions.
19 changes: 0 additions & 19 deletions connector/kinesis-asl-assembly/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -59,16 +59,6 @@
<artifactId>commons-lang</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
<version>2.6.1</version>
<!--
We are being explicit about version here and overriding the
spark default of 2.5.0 because KCL appears to have introduced
a dependency on protobuf 2.6.1 somewhere between KCL 1.4.0 and 1.6.1.
-->
</dependency>
<dependency>
<groupId>org.glassfish.jersey.core</groupId>
<artifactId>jersey-client</artifactId>
Expand Down Expand Up @@ -173,15 +163,6 @@
<include>*:*</include>
</includes>
</artifactSet>
<relocations>
<relocation>
<pattern>com.google.protobuf</pattern>
<shadedPattern>kinesis.protobuf</shadedPattern>
<includes>
<include>com.google.protobuf.**</include>
</includes>
</relocation>
</relocations>
<filters>
<filter>
<artifact>*:*</artifact>
Expand Down

0 comments on commit c1321c0

Please sign in to comment.