Simplify Maven dependency management after dropping building support for Spark 3.1

[pan3793]

🔍 Description

Issue References 🔗

SPARK-33212 (fixed in 3.2.0) moves from hadoop-client to shaded hadoop client, to simplify the dependency management, previously , we add some workaround to handle Spark 3.1 dependency issues. As we removed building support for Spark 3.1 now, we can remove those workaround to simplify pom.xml

Describe Your Solution 🔧

As above.

Types of changes 🔖

• Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Test Plan 🧪

Pass GA.

---

Checklist 📝

• This patch was not authored or co-authored using Generative Tooling

Be nice. Be informative.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 61.10%. Comparing base (9e7f2d0) to head (d6dcec3).
Report is 1 commits behind head on master.

❗ Current head d6dcec3 differs from pull request most recent head 1341065. Consider uploading reports for the commit 1341065 to get more accurate results

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #6131      +/-   ##
============================================
- Coverage     61.17%   61.10%   -0.07%     
  Complexity       23       23              
============================================
  Files           624      624              
  Lines         37260    37260              
  Branches       5046     5046              
============================================
- Hits          22795    22769      -26     
- Misses        12013    12031      +18     
- Partials       2452     2460       +8     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[pan3793]

Thanks, merged to master