Skip to content

[CELEBORN-2218] Bump lz4-java version from 1.8.0 to 1.10.4 to resolve CVE‐2025‐12183 and CVE-2025-66566 #3555

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
SteNicholas wants to merge 5 commits into from
+50 −38
Closed

[CELEBORN-2218] Bump lz4-java version from 1.8.0 to 1.10.4 to resolve CVE‐2025‐12183 and CVE-2025-66566 #3555

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
e01149f
[CELEBORN-2218] Bump lz4-java version from 1.8.0 to 1.10.3 to resolve…
SteNicholas Dec 3, 2025
effb531
update benchmark
pan3793 Mar 2, 2026
3cc43ce
Bump lz4-java 1.10.4
pan3793 Mar 3, 2026
60b1f29
update benchmark results
pan3793 Mar 3, 2026
1ff0da1
[CELEBORN-2218] Bump lz4-java version from 1.8.0 to 1.10.4 to resolve…
SteNicholas Mar 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions client-mr/mr-shaded/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,8 +64,8 @@
<shadedPattern>${shading.prefix}.org.scala-lang</shadedPattern>
</relocation>
<relocation>
<pattern>org.lz4</pattern>
<shadedPattern>${shading.prefix}.org.lz4</shadedPattern>
<pattern>${lz4-java.group}</pattern>
<shadedPattern>${shading.prefix}.${lz4-java.group}</shadedPattern>
</relocation>
<relocation>
<pattern>org.roaringbitmap</pattern>
Expand All @@ -81,7 +81,7 @@
<include>io.netty:*</include>
<include>org.apache.commons:commons-lang3</include>
<include>org.scala-lang:scala-library</include>
<include>org.lz4:lz4-java</include>
<include>${lz4-java.group}:lz4-java</include>
<include>com.github.luben:zstd-jni</include>
<include>org.roaringbitmap:RoaringBitmap</include>
</includes>
Expand Down
2 changes: 1 addition & 1 deletion client-mr/mr-shaded/src/main/resources/META-INF/LICENSE
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,7 @@ This project bundles the following dependencies under the Apache License 2.0 (ht
Apache License 2.0
--------------------------------------

at.yawk.lz4:lz4-java
com.google.guava:failureaccess
com.google.guava:guava
io.netty:netty
Expand Down Expand Up @@ -240,7 +241,6 @@ io.netty:netty-transport-rxtx
io.netty:netty-transport-sctp
io.netty:netty-transport-udt
org.apache.commons:commons-lang3
org.lz4:lz4-java
org.roaringbitmap:RoaringBitmap
org.scala-lang:scala-library

Expand Down
2 changes: 1 addition & 1 deletion client-tez/tez-shaded/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@
<include>org.roaringbitmap:RoaringBitmap</include>
<include>org.scala-lang:scala-library</include>
<include>org.scala-lang:scala-reflect</include>
<include>org.lz4:lz4-java</include>
<include>${lz4-java.group}:lz4-java</include>
<include>io.dropwizard.metrics:metrics-core</include>
<include>com.codahale.metrics:metrics-core</include>
<include>com.github.luben:zstd-jni</include>
Expand Down
2 changes: 1 addition & 1 deletion client-tez/tez-shaded/src/main/resources/META-INF/LICENSE
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,7 @@ This project bundles the following dependencies under the Apache License 2.0 (ht
Apache License 2.0
--------------------------------------

at.yawk.lz4:lz4-java
com.google.guava:failureaccess
com.google.guava:guava
io.netty:netty
Expand Down Expand Up @@ -240,7 +241,6 @@ io.netty:netty-transport-rxtx
io.netty:netty-transport-sctp
io.netty:netty-transport-udt
org.apache.commons:commons-lang3
org.lz4:lz4-java
org.roaringbitmap:RoaringBitmap
org.scala-lang:scala-library

Expand Down
16 changes: 8 additions & 8 deletions client/benchmarks/LZ4TPCDSDataBenchmark-jdk17-results.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,48 +6,48 @@ OpenJDK 64-Bit Server VM 17.0.15+6-LTS on Linux 6.17.9-76061709-generic
Intel(R) Core(TM) i5-9500 CPU @ 3.00GHz
Compression: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------------------------------------
Compression with chunk size 65536 4 times 2131 2134 5 0.0 532707902.0 1.0X
Compression with chunk size 65536 4 times 2193 2210 24 0.0 548315522.5 1.0X

OpenJDK 64-Bit Server VM 17.0.15+6-LTS on Linux 6.17.9-76061709-generic
Intel(R) Core(TM) i5-9500 CPU @ 3.00GHz
Decompression: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------
Decompression with chunk size 65536 4 times 536 541 9 0.0 133951799.0 1.0X
Decompression with chunk size 65536 4 times 460 463 3 0.0 114986376.3 1.0X

OpenJDK 64-Bit Server VM 17.0.15+6-LTS on Linux 6.17.9-76061709-generic
Intel(R) Core(TM) i5-9500 CPU @ 3.00GHz
Compression: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
--------------------------------------------------------------------------------------------------------------------------
Compression with chunk size 262144 4 times 1754 1756 2 0.0 438523185.2 1.0X
Compression with chunk size 262144 4 times 1683 1683 0 0.0 420711475.3 1.0X

OpenJDK 64-Bit Server VM 17.0.15+6-LTS on Linux 6.17.9-76061709-generic
Intel(R) Core(TM) i5-9500 CPU @ 3.00GHz
Decompression: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
----------------------------------------------------------------------------------------------------------------------------
Decompression with chunk size 262144 4 times 436 439 4 0.0 109013659.0 1.0X
Decompression with chunk size 262144 4 times 367 369 1 0.0 91804273.0 1.0X

OpenJDK 64-Bit Server VM 17.0.15+6-LTS on Linux 6.17.9-76061709-generic
Intel(R) Core(TM) i5-9500 CPU @ 3.00GHz
Compression: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------
Compression with chunk size 1048576 4 times 1774 1780 9 0.0 443426664.3 1.0X
Compression with chunk size 1048576 4 times 1725 1726 3 0.0 431152298.8 1.0X

OpenJDK 64-Bit Server VM 17.0.15+6-LTS on Linux 6.17.9-76061709-generic
Intel(R) Core(TM) i5-9500 CPU @ 3.00GHz
Decompression: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-----------------------------------------------------------------------------------------------------------------------------
Decompression with chunk size 1048576 4 times 431 434 4 0.0 107823243.0 1.0X
Decompression with chunk size 1048576 4 times 367 368 1 0.0 91743487.5 1.0X

OpenJDK 64-Bit Server VM 17.0.15+6-LTS on Linux 6.17.9-76061709-generic
Intel(R) Core(TM) i5-9500 CPU @ 3.00GHz
Compression: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------
Compression with chunk size 4194304 4 times 1785 1791 8 0.0 446360006.8 1.0X
Compression with chunk size 4194304 4 times 1697 1702 7 0.0 424249326.5 1.0X

OpenJDK 64-Bit Server VM 17.0.15+6-LTS on Linux 6.17.9-76061709-generic
Intel(R) Core(TM) i5-9500 CPU @ 3.00GHz
Decompression: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
-----------------------------------------------------------------------------------------------------------------------------
Decompression with chunk size 4194304 4 times 436 438 1 0.0 109117732.3 1.0X
Decompression with chunk size 4194304 4 times 371 376 4 0.0 92720790.5 1.0X


2 changes: 1 addition & 1 deletion client/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@
<artifactId>guava</artifactId>
</dependency>
<dependency>
<groupId>org.lz4</groupId>
<groupId>${lz4-java.group}</groupId>
<artifactId>lz4-java</artifactId>
</dependency>
<dependency>
Expand Down
16 changes: 8 additions & 8 deletions client/src/main/java/org/apache/celeborn/client/compress/Lz4Decompressor.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,15 +26,15 @@

import com.google.common.collect.ImmutableMap;
import net.jpountz.lz4.LZ4Factory;
import net.jpountz.lz4.LZ4FastDecompressor;
import net.jpountz.lz4.LZ4SafeDecompressor;
import net.jpountz.xxhash.XXHashFactory;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class Lz4Decompressor extends Lz4Trait implements Decompressor {
private static final Logger logger = LoggerFactory.getLogger(Lz4Decompressor.class);

private final LZ4FastDecompressor decompressor;
private final LZ4SafeDecompressor decompressor;
private final Checksum checksum;

private final Map<String, Supplier<XXHashFactory>> xxHashFactories =
Expand All @@ -47,7 +47,7 @@ public class Lz4Decompressor extends Lz4Trait implements Decompressor {
XXHashFactory::unsafeInstance);

public Lz4Decompressor(Option<String> xxHashInstance) {
decompressor = LZ4Factory.fastestInstance().fastDecompressor();
decompressor = LZ4Factory.fastestInstance().safeDecompressor();
checksum = getXXHashFactory(xxHashInstance).newStreamingHash32(DEFAULT_SEED).asChecksum();
}

Expand All @@ -68,13 +68,13 @@ public int decompress(byte[] src, byte[] dst, int dstOff) throws IOException {
System.arraycopy(src, HEADER_LENGTH, dst, dstOff, originalLen);
break;
case COMPRESSION_METHOD_LZ4:
int compressedLen2 = decompressor.decompress(src, HEADER_LENGTH, dst, dstOff, originalLen);
if (compressedLen != compressedLen2) {
int originalLen2 = decompressor.decompress(src, HEADER_LENGTH, compressedLen, dst, dstOff);
if (originalLen != originalLen2) {
throw new IOException(
"Compressed length corrupted! expected: "
+ compressedLen
"Original length corrupted! expected: "
+ originalLen
+ ", actual: "
+ compressedLen2
+ originalLen2
+ ".");
}
}
Expand Down
2 changes: 1 addition & 1 deletion dev/deps/dependencies-client-flink-1.16
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ jcl-over-slf4j/1.7.36//jcl-over-slf4j-1.7.36.jar
jsr305/1.3.9//jsr305-1.3.9.jar
jul-to-slf4j/1.7.36//jul-to-slf4j-1.7.36.jar
leveldbjni-all/1.8//leveldbjni-all-1.8.jar
lz4-java/1.8.0//lz4-java-1.8.0.jar
lz4-java/1.10.4//lz4-java-1.10.4.jar
maven-jdk-tools-wrapper/0.1//maven-jdk-tools-wrapper-0.1.jar
metrics-core/4.2.25//metrics-core-4.2.25.jar
metrics-graphite/4.2.25//metrics-graphite-4.2.25.jar
Expand Down
2 changes: 1 addition & 1 deletion dev/deps/dependencies-client-flink-1.17
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ jcl-over-slf4j/1.7.36//jcl-over-slf4j-1.7.36.jar
jsr305/1.3.9//jsr305-1.3.9.jar
jul-to-slf4j/1.7.36//jul-to-slf4j-1.7.36.jar
leveldbjni-all/1.8//leveldbjni-all-1.8.jar
lz4-java/1.8.0//lz4-java-1.8.0.jar
lz4-java/1.10.4//lz4-java-1.10.4.jar
maven-jdk-tools-wrapper/0.1//maven-jdk-tools-wrapper-0.1.jar
metrics-core/4.2.25//metrics-core-4.2.25.jar
metrics-graphite/4.2.25//metrics-graphite-4.2.25.jar
Expand Down
2 changes: 1 addition & 1 deletion dev/deps/dependencies-client-flink-1.18
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ jcl-over-slf4j/1.7.36//jcl-over-slf4j-1.7.36.jar
jsr305/1.3.9//jsr305-1.3.9.jar
jul-to-slf4j/1.7.36//jul-to-slf4j-1.7.36.jar
leveldbjni-all/1.8//leveldbjni-all-1.8.jar
lz4-java/1.8.0//lz4-java-1.8.0.jar
lz4-java/1.10.4//lz4-java-1.10.4.jar
maven-jdk-tools-wrapper/0.1//maven-jdk-tools-wrapper-0.1.jar
metrics-core/4.2.25//metrics-core-4.2.25.jar
metrics-graphite/4.2.25//metrics-graphite-4.2.25.jar
Expand Down
2 changes: 1 addition & 1 deletion dev/deps/dependencies-client-flink-1.19
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ jcl-over-slf4j/1.7.36//jcl-over-slf4j-1.7.36.jar
jsr305/1.3.9//jsr305-1.3.9.jar
jul-to-slf4j/1.7.36//jul-to-slf4j-1.7.36.jar
leveldbjni-all/1.8//leveldbjni-all-1.8.jar
lz4-java/1.8.0//lz4-java-1.8.0.jar
lz4-java/1.10.4//lz4-java-1.10.4.jar
maven-jdk-tools-wrapper/0.1//maven-jdk-tools-wrapper-0.1.jar
metrics-core/4.2.25//metrics-core-4.2.25.jar
metrics-graphite/4.2.25//metrics-graphite-4.2.25.jar
Expand Down
2 changes: 1 addition & 1 deletion dev/deps/dependencies-client-flink-1.20
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ jcl-over-slf4j/1.7.36//jcl-over-slf4j-1.7.36.jar
jsr305/1.3.9//jsr305-1.3.9.jar
jul-to-slf4j/1.7.36//jul-to-slf4j-1.7.36.jar
leveldbjni-all/1.8//leveldbjni-all-1.8.jar
lz4-java/1.8.0//lz4-java-1.8.0.jar
lz4-java/1.10.4//lz4-java-1.10.4.jar
maven-jdk-tools-wrapper/0.1//maven-jdk-tools-wrapper-0.1.jar
metrics-core/4.2.25//metrics-core-4.2.25.jar
metrics-graphite/4.2.25//metrics-graphite-4.2.25.jar
Expand Down
2 changes: 1 addition & 1 deletion dev/deps/dependencies-client-flink-2.0
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ jcl-over-slf4j/1.7.36//jcl-over-slf4j-1.7.36.jar
jsr305/1.3.9//jsr305-1.3.9.jar
jul-to-slf4j/1.7.36//jul-to-slf4j-1.7.36.jar
leveldbjni-all/1.8//leveldbjni-all-1.8.jar
lz4-java/1.8.0//lz4-java-1.8.0.jar
lz4-java/1.10.4//lz4-java-1.10.4.jar
metrics-core/4.2.25//metrics-core-4.2.25.jar
metrics-graphite/4.2.25//metrics-graphite-4.2.25.jar
metrics-jvm/4.2.25//metrics-jvm-4.2.25.jar
Expand Down
2 changes: 1 addition & 1 deletion dev/deps/dependencies-client-flink-2.1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ jcl-over-slf4j/1.7.36//jcl-over-slf4j-1.7.36.jar
jsr305/1.3.9//jsr305-1.3.9.jar
jul-to-slf4j/1.7.36//jul-to-slf4j-1.7.36.jar
leveldbjni-all/1.8//leveldbjni-all-1.8.jar
lz4-java/1.8.0//lz4-java-1.8.0.jar
lz4-java/1.10.4//lz4-java-1.10.4.jar
metrics-core/4.2.25//metrics-core-4.2.25.jar
metrics-graphite/4.2.25//metrics-graphite-4.2.25.jar
metrics-jvm/4.2.25//metrics-jvm-4.2.25.jar
Expand Down
2 changes: 1 addition & 1 deletion dev/deps/dependencies-client-flink-2.2
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ jcl-over-slf4j/1.7.36//jcl-over-slf4j-1.7.36.jar
jsr305/1.3.9//jsr305-1.3.9.jar
jul-to-slf4j/1.7.36//jul-to-slf4j-1.7.36.jar
leveldbjni-all/1.8//leveldbjni-all-1.8.jar
lz4-java/1.8.0//lz4-java-1.8.0.jar
lz4-java/1.10.4//lz4-java-1.10.4.jar
metrics-core/4.2.25//metrics-core-4.2.25.jar
metrics-graphite/4.2.25//metrics-graphite-4.2.25.jar
metrics-jvm/4.2.25//metrics-jvm-4.2.25.jar
Expand Down
2 changes: 1 addition & 1 deletion dev/deps/dependencies-client-mr
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ kerby-xdr/1.0.1//kerby-xdr-1.0.1.jar
kotlin-stdlib-common/1.4.10//kotlin-stdlib-common-1.4.10.jar
kotlin-stdlib/1.4.10//kotlin-stdlib-1.4.10.jar
leveldbjni-all/1.8//leveldbjni-all-1.8.jar
lz4-java/1.8.0//lz4-java-1.8.0.jar
lz4-java/1.10.4//lz4-java-1.10.4.jar
maven-jdk-tools-wrapper/0.1//maven-jdk-tools-wrapper-0.1.jar
metrics-core/4.2.25//metrics-core-4.2.25.jar
metrics-graphite/4.2.25//metrics-graphite-4.2.25.jar
Expand Down
2 changes: 1 addition & 1 deletion dev/deps/dependencies-client-tez
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ kerby-util/1.0.1//kerby-util-1.0.1.jar
kerby-xdr/1.0.1//kerby-xdr-1.0.1.jar
leveldbjni-all/1.8//leveldbjni-all-1.8.jar
log4j/1.2.17//log4j-1.2.17.jar
lz4-java/1.8.0//lz4-java-1.8.0.jar
lz4-java/1.10.4//lz4-java-1.10.4.jar
maven-jdk-tools-wrapper/0.1//maven-jdk-tools-wrapper-0.1.jar
metrics-core/4.2.25//metrics-core-4.2.25.jar
metrics-graphite/4.2.25//metrics-graphite-4.2.25.jar
Expand Down
2 changes: 1 addition & 1 deletion dev/deps/dependencies-server
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ log4j-1.2-api/2.24.3//log4j-1.2-api-2.24.3.jar
log4j-api/2.24.3//log4j-api-2.24.3.jar
log4j-core/2.24.3//log4j-core-2.24.3.jar
log4j-slf4j-impl/2.24.3//log4j-slf4j-impl-2.24.3.jar
lz4-java/1.8.0//lz4-java-1.8.0.jar
lz4-java/1.10.4//lz4-java-1.10.4.jar
maven-jdk-tools-wrapper/0.1//maven-jdk-tools-wrapper-0.1.jar
metrics-core/4.2.25//metrics-core-4.2.25.jar
metrics-graphite/4.2.25//metrics-graphite-4.2.25.jar
Expand Down
14 changes: 12 additions & 2 deletions pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,8 @@
<leveldb.version>1.8</leveldb.version>
<log4j2.version>2.24.3</log4j2.version>
<disruptor.version>3.4.4</disruptor.version>
<lz4-java.version>1.8.0</lz4-java.version>
<lz4-java.group>at.yawk.lz4</lz4-java.group>
<lz4-java.version>1.10.4</lz4-java.version>
<mockito.version>4.11.0</mockito.version>
<mockito-scalatest.version>1.17.14</mockito-scalatest.version>
<netty.version>4.2.10.Final</netty.version>
Expand Down Expand Up @@ -527,7 +528,7 @@
<version>${leveldb.version}</version>
</dependency>
<dependency>
<groupId>org.lz4</groupId>
<groupId>${lz4-java.group}</groupId>
<artifactId>lz4-java</artifactId>
<version>${lz4-java.version}</version>
</dependency>
Expand Down Expand Up @@ -1530,6 +1531,7 @@
<module>tests/spark-it</module>
</modules>
<properties>
<lz4-java.group>org.lz4</lz4-java.group>
<lz4-java.version>1.4.0</lz4-java.version>
<scala.version>2.11.12</scala.version>
<scala.binary.version>2.11</scala.binary.version>
Expand All @@ -1549,6 +1551,7 @@
<module>tests/spark-it</module>
</modules>
<properties>
<lz4-java.group>org.lz4</lz4-java.group>
<lz4-java.version>1.7.1</lz4-java.version>
<scala.version>2.12.10</scala.version>
<scala.binary.version>2.12</scala.binary.version>
Expand All @@ -1569,6 +1572,7 @@
<module>tests/spark-it</module>
</modules>
<properties>
<lz4-java.group>org.lz4</lz4-java.group>
<lz4-java.version>1.7.1</lz4-java.version>
<scala.version>2.12.10</scala.version>
<scala.binary.version>2.12</scala.binary.version>
Expand All @@ -1589,6 +1593,7 @@
<module>tests/spark-it</module>
</modules>
<properties>
<lz4-java.group>org.lz4</lz4-java.group>
<lz4-java.version>1.7.1</lz4-java.version>
<scala.version>2.12.15</scala.version>
<scala.binary.version>2.12</scala.binary.version>
Expand All @@ -1608,6 +1613,7 @@
<module>tests/spark-it</module>
</modules>
<properties>
<lz4-java.group>org.lz4</lz4-java.group>
<lz4-java.version>1.8.0</lz4-java.version>
<scala.version>2.12.15</scala.version>
<scala.binary.version>2.12</scala.binary.version>
Expand All @@ -1627,6 +1633,7 @@
<module>tests/spark-it</module>
</modules>
<properties>
<lz4-java.group>org.lz4</lz4-java.group>
<lz4-java.version>1.8.0</lz4-java.version>
<scala.version>2.12.17</scala.version>
<scala.binary.version>2.12</scala.binary.version>
Expand All @@ -1646,6 +1653,7 @@
<module>tests/spark-it</module>
</modules>
<properties>
<lz4-java.group>org.lz4</lz4-java.group>
<lz4-java.version>1.8.0</lz4-java.version>
<scala.version>2.12.18</scala.version>
<scala.binary.version>2.12</scala.binary.version>
Expand All @@ -1665,6 +1673,7 @@
<module>tests/spark-it</module>
</modules>
<properties>
<lz4-java.group>org.lz4</lz4-java.group>
<lz4-java.version>1.8.0</lz4-java.version>
<scala.version>2.13.16</scala.version>
<scala.binary.version>2.13</scala.binary.version>
Expand All @@ -1684,6 +1693,7 @@
<module>tests/spark-it</module>
</modules>
<properties>
<lz4-java.group>org.lz4</lz4-java.group>
<lz4-java.version>1.8.0</lz4-java.version>
<scala.version>2.13.17</scala.version>
<scala.binary.version>2.13</scala.binary.version>
Expand Down
6 changes: 4 additions & 2 deletions project/CelebornBuild.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,8 @@ import CelebornCommonSettings._
object Dependencies {

val zstdJniVersion = sparkClientProjects.map(_.zstdJniVersion).getOrElse("1.5.7-1")
val lz4JavaVersion = sparkClientProjects.map(_.lz4JavaVersion).getOrElse("1.8.0")
val lz4JavaGroup = sparkClientProjects.map(_.lz4JavaGroup).getOrElse("at.yawk.lz4")
val lz4JavaVersion = sparkClientProjects.map(_.lz4JavaVersion).getOrElse("1.10.4")

// Dependent library versions
val apLoaderVersion = "4.0-10"
Expand Down Expand Up @@ -176,7 +177,7 @@ object Dependencies {
val log4j12Api = "org.apache.logging.log4j" % "log4j-1.2-api" % log4j2Version
val log4jSlf4jImpl = "org.apache.logging.log4j" % "log4j-slf4j-impl" % log4j2Version
val disruptor = "com.lmax" % "disruptor" % disruptorVersion
val lz4Java = "org.lz4" % "lz4-java" % lz4JavaVersion
val lz4Java = lz4JavaGroup % "lz4-java" % lz4JavaVersion
val protobufJava = "com.google.protobuf" % "protobuf-java" % protoVersion
val ratisClient = "org.apache.ratis" % "ratis-client" % ratisVersion
val ratisCommon = "org.apache.ratis" % "ratis-common" % ratisVersion
Expand Down Expand Up @@ -1017,6 +1018,7 @@ trait SparkClientProjects {
val sparkClientShadedProjectPath: String
val sparkClientShadedProjectName: String

val lz4JavaGroup: String = "org.lz4"
val lz4JavaVersion: String
val sparkProjectScalaVersion: String
val sparkVersion: String
Expand Down
Loading