[SPARK-47538][BUILD] Remove commons-logging dependency

[dongjoon-hyun]

What changes were proposed in this pull request?

This PR aims to remove commons-logging dependency in favor of jcl-over-slf4j.

Why are the changes needed?

• https://slf4j.org/legacy.html#jclOverSLF4J

To ease migration to SLF4J from JCL, SLF4J distributions include the jar file jcl-over-slf4j.jar. This jar file is intended as a drop-in replacement for JCL version 1.1.1. It implements the public API of JCL but using SLF4J underneath, hence the name "JCL over SLF4J."

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Pass the CIs.

Was this patch authored or co-authored using generative AI tooling?

No.

[pan3793]

https://slf4j.org/legacy.html#jclOverSLF4J

jcl-over-slf4j is intended as a drop-in replacement for commons-logging, maybe we should exclude commons-logging in all places.

[dongjoon-hyun]

Thanks. Let me try that way, @pan3793 .

[dongjoon-hyun]

For the record, since Apache Hive 2.0.0 removed direct commons-logging dependency via HIVE-12237 Use slf4j as logging facade, we don't need this.

[dongjoon-hyun]

Thank you, @pan3793 .

[dongjoon-hyun]

All tests passed. Could you review this dependency PR, @yaooqinn ?

[yaooqinn]

Hi @dongjoon-hyun. We can also remove the lines from the NOTICE*/LICENSE*?

[yaooqinn]

LGTM, one one minor comment

[dongjoon-hyun]

Oh, thanks. Sure!

[dongjoon-hyun]

Thank you, @pan3793 , @yaooqinn , @HyukjinKwon !
Merged to master for Apache Spark 4.0.0.