Flaky Test: ShuffleTaskManagerTest#removeShuffleDataWithHdfsTest

[zuston]

Code of Conduct

• I agree to follow this project's Code of Conduct

Search before asking

• I have searched in the issues and found no similar issues.

Describe the flaky test

Error:  Tests run: 14, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 85.858 s <<< FAILURE! - in org.apache.uniffle.server.ShuffleTaskManagerTest
Error:  removeShuffleDataWithHdfsTest  Time elapsed: 15.048 s  <<< ERROR!
java.lang.RuntimeException: Shuffle data commit timeout for 10000 ms
	at org.apache.uniffle.server.ShuffleTaskManager.commitShuffle(ShuffleTaskManager.java:259)
	at org.apache.uniffle.server.ShuffleTaskManagerTest.removeShuffleDataWithHdfsTest(ShuffleTaskManagerTest.java:427)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

Actions URL

https://github.com/apache/incubator-uniffle/actions/runs/4191298244/jobs/7265939805

Parent issue

#1733

Are you willing to submit PR?

• Yes I am willing to submit a PR!

[zuston]

Detailed info:

2023-02-16 06:57:00,663 INFO  [main] buffer.ShuffleBufferManager (ShuffleBufferManager.java:pickFlushedShuffle(500)) - Finish flush pick with 67 bytes
SLF4J: Failed toString() invocation on an object of type [org.apache.uniffle.server.ShuffleDataFlushEvent]
Reported exception:
java.lang.NullPointerException
	at org.apache.uniffle.server.ShuffleDataFlushEvent.toString(ShuffleDataFlushEvent.java:163)
	at org.slf4j.helpers.MessageFormatter.safeObjectAppend(MessageFormatter.java:277)
	at org.slf4j.helpers.MessageFormatter.deeplyAppendParameter(MessageFormatter.java:249)
	at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:211)
	at org.slf4j.helpers.MessageFormatter.arrayFormat(MessageFormatter.java:161)
	at org.slf4j.helpers.MessageFormatter.format(MessageFormatter.java:124)
	at org.slf4j.impl.Log4jLoggerAdapter.error(Log4jLoggerAdapter.java:516)
	at org.apache.uniffle.server.ShuffleFlushManager.flushToFile(ShuffleFlushManager.java:186)
	at org.apache.uniffle.server.ShuffleFlushManager.processEvent(ShuffleFlushManager.java:157)
	at org.apache.uniffle.server.ShuffleFlushManager.lambda$processNextEvent$0(ShuffleFlushManager.java:148)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:750)