[#580] chore: speed up CI workflows

[kaijchen]

What changes were proposed in this pull request?

1. Split unit test and integration test
2. Enable maven parallel build (-T1C)
3. Do not use matrix in deploy/kubernetes workflow
4. Delay some secondary workflows

Why are the changes needed?

Resolve #580
Make CI faster.
In case of failure, rerun the failed part only.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

https://github.com/kaijchen/incubator-uniffle/actions/runs/4180474332

[codecov-commenter]

Codecov Report

Merging #602 (ca12d53) into master (56145c2) will decrease coverage by 0.83%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master     #602      +/-   ##
============================================
- Coverage     61.67%   60.85%   -0.83%     
- Complexity     1668     1800     +132     
============================================
  Files           201      214      +13     
  Lines         11111    12387    +1276     
  Branches        922     1044     +122     
============================================
+ Hits           6853     7538     +685     
- Misses         3883     4444     +561     
- Partials        375      405      +30     

Impacted Files	Coverage Δ
...java/org/apache/hadoop/mapred/SortWriteBuffer.java	90.90% <0.00%> (ø)
.../java/org/apache/hadoop/mapreduce/RssMRConfig.java	23.07% <0.00%> (ø)
...rg/apache/hadoop/mapred/RssMapOutputCollector.java	0.00% <0.00%> (ø)
...preduce/task/reduce/RssRemoteMergeManagerImpl.java	82.43% <0.00%> (ø)
...apache/hadoop/mapreduce/v2/app/RssMRAppMaster.java	0.00% <0.00%> (ø)
.../hadoop/mapreduce/task/reduce/RssEventFetcher.java	88.57% <0.00%> (ø)
.../hadoop/mapreduce/task/reduce/RssBypassWriter.java	61.90% <0.00%> (ø)
...pache/hadoop/mapreduce/task/reduce/RssFetcher.java	92.30% <0.00%> (ø)
...g/apache/hadoop/mapred/SortWriteBufferManager.java	79.89% <0.00%> (ø)
...n/java/org/apache/hadoop/mapreduce/RssMRUtils.java	50.45% <0.00%> (ø)
... and 3 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[advancedxy]

Thanks @kaijchen for this awesome work. Now the CI is down to 20min.
However there might be some other improvement:

1. run integration tests run with unit test disabled?
2. some common integration tests are run multiple times. is it possible to skip integration test of common and spark-common when running spark3 integration test?

[kaijchen]

1. run integration tests run with unit test disabled?

UT is already excluded. Some integration tests are slow, maybe we can improve the speed of them.
And we can also split integration tests to multiple packages, to further break down the workflow and improve parallelism.

2. some common integration tests are run multiple times. is it possible to skip integration test of common and spark-common when running spark3 integration test?

I'm not sure, maybe some tests need and some tests don't. Let's check and split them into different packages later.

[advancedxy]

UT is already excluded. Some integration tests are slow, maybe we can improve the speed of them.
And we can also split integration tests to multiple packages, to further break down the workflow and improve parallelism.

Ah, I didn't see that coming since you don't specify -DskipIT..
But it seems integration tests are prefixed with test package.

I'm not sure, maybe some tests need and some tests don't. Let's check and split them into different packages later.

Yeah.. Integration tests should be in separated packages. But let's do that later.

[kaijchen]

Ah, I didn't see that coming since you don't specify -DskipIT..

Because integration tests are executed by the maven-surefire-plugin also.
maven-surefire-plugin is designed to run UT, and controlled by -DskipUT.
I tried to run integration tests by maven-failsafe-plugin in #598, but it was too slow.

[kaijchen]

Thanks @advancedxy for the review.