[SPARK-48225][BUILD] Upgrade sbt to 1.10.0

[dongjoon-hyun]

What changes were proposed in this pull request?

This PR aims to try sbt to 1.10.0.

Why are the changes needed?

So far, we couldn't upgrade SBT like the following.

• [WIP][Only Test] Test sbt 1.9.9 #45345
• [BUILD] Test sbt 1.9.9 #45312
• [BUILD] Test sbt 1.9.8 #44407
• [Don't Merge] upgrade sbt 1.9.8 #44516
• [SPARK-46096][BUILD] Upgrade sbt to 1.9.7 #44008
• [SPARK-44959][BUILD] Upgrade sbt to 1.9.5 #42673

Previously, we couldn't upgrade SBT due to its issues like the dependency resolution. SBT 1.10.0 is updated with Coursier 2.1.9 .

• https://github.com/sbt/sbt/releases/tag/v1.10.0
• https://github.com/sbt/sbt/releases/tag/v1.9.9
• https://github.com/sbt/sbt/releases/tag/v1.9.8
• https://github.com/sbt/sbt/releases/tag/v1.9.7
• https://github.com/sbt/sbt/releases/tag/v1.9.6
• https://github.com/sbt/sbt/releases/tag/v1.9.5
• https://github.com/sbt/sbt/releases/tag/v1.9.4

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Pass the CIs.

Was this patch authored or co-authored using generative AI tooling?

No.

[dongjoon-hyun]

This is a new test to check if the bug is fixed or not.

cc @panbingkun and @LuciferYang .

[dongjoon-hyun]

Unfortunately, it turns out that the bug still exists.

[error] lmcoursier.internal.shaded.coursier.error.FetchError$DownloadingArtifacts: Error fetching artifacts:
[error] file:/root/.m2/repository/org/scala-lang/scala-library/2.13.10/scala-library-2.13.10.jar: not found: /root/.m2/repository/org/scala-lang/scala-library/2.13.10/scala-library-2.13.10.jar
[error] 

[dongjoon-hyun]

Let me open this PR for a while before closing.

[LuciferYang]

@dongjoon-hyun I was discussing this issue with @panbingkun offline yesterday. From the responses of sbt and coursier, it seems difficult to solve this problem in the short term without some handling of Resolver.mavenLocal. I suggest that @panbingkun conduct some performance tests to determine whether this issue can be bypassed by disabling Resolver.mavenLocal:

• When there is a corrupted jar in Maven's local repo, it will fail sbt/sbt#7506 (comment)
• When there is a missing jar in Maven's local repo, it will fail. coursier/coursier#2942

[dongjoon-hyun]

Thank you so much for sharing!

[panbingkun]

@dongjoon-hyun I was discussing this issue with @panbingkun offline yesterday. From the responses of sbt and coursier, it seems difficult to solve this problem in the short term without some handling of Resolver.mavenLocal. I suggest that @panbingkun conduct some performance tests to determine whether this issue can be bypassed by disabling Resolver.mavenLocal:

• When there is a corrupted jar in Maven's local repo, it will fail sbt/sbt#7506 (comment)
• When there is a corrupted jar in Maven's local repo, it will fail. coursier/coursier#2942

Yeah, let me do some performance tests.