chore(ci): Simplify test combinations on Spark in Github actions

[yihua]

Describe the issue this Pull Request addresses

There are redundant test combinations on Spark in Github actions. This PR improves the test combinations on Spark to keep only the necessary combinations.

Summary and Changelog

This PR improves the test combinations on Spark in Github actions:

• On default versions, Java 11 (build and test), running these combinations: Spark 3.3 Scala 2.12, Spark 3.4 Scala 2.12, Spark 3.5 Scala 2.12 (enabled in this PR, so that we can remove Azure CI, which runs on Spark 3.5 Scala 2.12). Spark 3.5 Scala 2.13 is removed and runs on Java 17 only.
• On Java 17 (build and test), running these combinations: Spark 3.5 Scala 2.13, Spark 4.0 Scala 2.13. Spark 3.3 and Spark 3.4 will no longer be run on this.
• Actions that build Hudi on Java 11 and run tests on Java 17 are removed.
• docker-java17-test: only keeping Spark 3.5 and Spark 4.0. Spark 3.4 is removed.
• build-spark-java17: Spark 3.5 and Spark 4.0 with Scala 2.13 are removed as they are already covered by other actions on Java 17.

Impact

Less resource consumed on Github actions.

Risk Level

none

Documentation Update

none

Contributor's checklist

• Read through contributor's guide
• Enough context is provided in the sections above
• Adequate tests were added if applicable

[hudi-bot]

CI report:

• bebf2d9 Azure: PENDING

Bot commands

@hudi-bot supports the following commands:

• @hudi-bot run azure re-run the last Azure build

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 69.20%. Comparing base (a179555) to head (bebf2d9).

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #18336      +/-   ##
============================================
- Coverage     69.26%   69.20%   -0.07%     
+ Complexity    27117    27113       -4     
============================================
  Files          2391     2391              
  Lines        129572   129559      -13     
  Branches      15366    15284      -82     
============================================
- Hits          89746    89655      -91     
- Misses        32969    32972       +3     
- Partials       6857     6932      +75     

Flag	Coverage Δ
common-and-other-modules	44.37% <ø> (-0.01%)	⬇️
hadoop-mr-java-client	45.16% <ø> (+<0.01%)	⬆️
spark-client-hadoop-common	48.33% <ø> (-0.01%)	⬇️
spark-java-tests	47.42% <ø> (-0.05%)	⬇️
spark-scala-tests	45.47% <ø> (-0.06%)	⬇️
utilities	38.69% <ø> (-0.03%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 67 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.