Skip to content

Commit

Permalink
[SPARK-47695][BUILD] Upgrade AWS SDK v2 to 2.24.6
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?

This PR aims to upgrade AWS SDK v2 to 2.24.6.

### Why are the changes needed?

Like HADOOP-19082 (Apache Hadoop 3.4.1), Apache Spark 4.0.0 had better use the latest one.
- apache/hadoop#6568

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Pass the CIs.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #45821 from dongjoon-hyun/SPARK-47695.

Authored-by: Dongjoon Hyun <dhyun@apple.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
  • Loading branch information
dongjoon-hyun committed Apr 2, 2024
1 parent db0975c commit e6144e4
Show file tree
Hide file tree
Showing 3 changed files with 13 additions and 2 deletions.
2 changes: 1 addition & 1 deletion dev/deps/spark-deps-hadoop-3-hive-2.3
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ blas/3.0.3//blas-3.0.3.jar
bonecp/0.8.0.RELEASE//bonecp-0.8.0.RELEASE.jar
breeze-macros_2.13/2.1.0//breeze-macros_2.13-2.1.0.jar
breeze_2.13/2.1.0//breeze_2.13-2.1.0.jar
bundle/2.23.19//bundle-2.23.19.jar
bundle/2.24.6//bundle-2.24.6.jar
cats-kernel_2.13/2.8.0//cats-kernel_2.13-2.8.0.jar
chill-java/0.10.0//chill-java-0.10.0.jar
chill_2.13/0.10.0//chill_2.13-0.10.0.jar
Expand Down
11 changes: 11 additions & 0 deletions hadoop-cloud/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,17 @@
<version>${hadoop.version}</version>
<scope>${hadoop.deps.scope}</scope>
</dependency>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>bundle</artifactId>
<version>${aws.java.sdk.v2.version}</version>
<exclusions>
<exclusion>
<groupId>*</groupId>
<artifactId>*</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.google.cloud.bigdataoss</groupId>
<artifactId>gcs-connector</artifactId>
Expand Down
2 changes: 1 addition & 1 deletion pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -160,7 +160,7 @@
<aws.kinesis.client.version>1.12.0</aws.kinesis.client.version>
<!-- Should be consistent with Kinesis client dependency -->
<aws.java.sdk.version>1.11.655</aws.java.sdk.version>
<aws.java.sdk.v2.version>2.23.19</aws.java.sdk.v2.version>
<aws.java.sdk.v2.version>2.24.6</aws.java.sdk.v2.version>
<!-- the producer is used in tests -->
<aws.kinesis.producer.version>0.12.8</aws.kinesis.producer.version>
<gcs-connector.version>hadoop3-2.2.20</gcs-connector.version>
Expand Down

0 comments on commit e6144e4

Please sign in to comment.