[SPARK-30968][BUILD] Upgrade aws-java-sdk-sts to 1.11.655

[dongjoon-hyun]

What changes were proposed in this pull request?

This PR aims to upgrade aws-java-sdk-sts to 1.11.655.

Why are the changes needed?

SPARK-29677 upgrades AWS Kinesis Client to 1.12.0 for Apache Spark 2.4.5 and 3.0.0.

Since AWS Kinesis Client 1.12.0 is using AWS SDK 1.11.665, aws-java-sdk-sts should be consistent with Kinesis client dependency.

• https://github.com/awslabs/amazon-kinesis-client/releases/tag/v1.12.0

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Pass the Jenkins.

[kiszk]

Pending Jenkins result
Changes looks minor between 1.11.655 and 1.11.271.

[SparkQA]

Test build #119016 has finished for PR 27720 at commit 9362536.

• This patch fails PySpark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[HyukjinKwon]

retest this please

[SparkQA]

Test build #119025 has finished for PR 27720 at commit 9362536.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[maropu]

retest this please

[SparkQA]

Test build #119040 has finished for PR 27720 at commit 9362536.

• This patch fails PySpark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[dongjoon-hyun]

Retest this please.

[viirya]

Should we add a comment reminding to change aws.java.sdk.version too in front of aws.kinesis.client.version?

[dongjoon-hyun]

Thanks. For now, <!-- Should be consistent with Kinesis client dependency --> at line 151 looks enough to me. Although one line ahead (in front of) sounds better than one line below, the warning content will be the same.

[SparkQA]

Test build #119054 has finished for PR 27720 at commit 9362536.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[dongjoon-hyun]

Thank you all! Merged to master/3.0/2.4

[yhuai]

@dongjoon-hyun @viirya @HyukjinKwon @maropu I am not sure I get the rational of merging this into 2.4. We are bumping the version for a spark maintenance branch to include almost two years of aws sdk changes, which is very risky. I am worried about unexpected behavior changes.

[yhuai]

Also, newer version of aws sdk uses apache httpclient, 4.5.9, what is the reason we do not upgrade httpclient?

[dongjoon-hyun]

Hi, @yhuai . I'm wondering if you are suggesting to revert SPARK-29677 from branch-2.4? As I wrote in the PR description, this PR is trying to complete the following requirement after #26333 .

We are bumping the version for a spark maintenance branch to include almost two years of aws sdk changes, which is very risky.

    <aws.kinesis.client.version>1.12.0</aws.kinesis.client.version>
    <!-- Should be consistent with Kinesis client dependency -->
    <aws.java.sdk.version>1.11.655</aws.java.sdk.version>

[yhuai]

@dongjoon-hyun sorry for getting back late. What I mean is that 1.11.655 depends on apache httpclient 4.5.9. This pr does not upgrade httpclient. Was there a specific reason?