[HUDI-6944] Fix Flink bootstrap concurrency issue causes job to keep rolling back

[zhangyue19921010]

Change Logs

There are some details
Commit 20231008185332158 start at 18:53:32.184 and Rolling back immediately after 20 milliseconds(18:53:32.212)

2023-10-08 18:53:32.155 INFO  [pool-20-thread-1:0-thread-1] o.a.hudi.table.action.rollback.BaseRollbackActionExecutor - Rollback of Commits [20231008185331924] is complete

.....

2023-10-08 18:53:32.184 INFO  [pool-20-thread-1:0-thread-1] org.apache.hudi.sink.StreamWriteOperatorCoordinator - Create instant [20231008185332158] for table [hudi_sss] with type [COPY_ON_WRITE]

..... 

2023-10-08 18:53:32.212 INFO  [pool-20-thread-1:0-thread-1] org.apache.hudi.client.BaseHoodieWriteClient - Begin rollback of instant 20231008185332158

-rw-rw-rw-   3 xxx xxx          0 2023-10-08 18:55 hdfs://xxx/user/xxx/bdm_dev.db/hudi_xxx/.hoodie/20231008185533653.rollback.inflight
-rw-rw-rw-   3 xxx xxx       1225 2023-10-08 18:55 hdfs://xxx/user/xxx/bdm_dev.db/hudi_xxx/.hoodie/20231008185533653.rollback.requested
-rw-rw-rw-   3 xxx xxx       1408 2023-10-08 18:55 hdfs://xxx/user/xxx/bdm_dev.db/hudi_xxx/.hoodie/20231008185533893.rollback
-rw-rw-rw-   3 xxx xxx          0 2023-10-08 18:55 hdfs://xxx/user/xxx/bdm_dev.db/hudi_xxx/.hoodie/20231008185533893.rollback.inflight
-rw-rw-rw-   3 xxx xxx       1225 2023-10-08 18:55 hdfs://xxx/user/xxx/bdm_dev.db/hudi_xxx/.hoodie/20231008185533893.rollback.requested
-rw-rw-rw-   3 xxx xxx       1408 2023-10-08 18:55 hdfs://xxx/user/xxx/bdm_dev.db/hudi_xxx/.hoodie/20231008185534199.rollback
-rw-rw-rw-   3 xxx xxx          0 2023-10-08 18:55 hdfs://xxx/user/xxx/bdm_dev.db/hudi_xxx/.hoodie/20231008185534199.rollback.inflight
-rw-rw-rw-   3 xxx xxx       1225 2023-10-08 18:55 hdfs://xxx/user/xxx/bdm_dev.db/hudi_xxx/.hoodie/20231008185534199.rollback.requested
-rw-rw-rw-   3 xxx xxx       1408 2023-10-08 18:55 hdfs://xxx/user/xxx/bdm_dev.db/hudi_xxx/.hoodie/20231008185534458.rollback
-rw-rw-rw-   3 xxx xxx          0 2023-10-08 18:55 hdfs://xxx/user/xxx/bdm_dev.db/hudi_xxx/.hoodie/20231008185534458.rollback.inflight
-rw-rw-rw-   3 xxx xxx       1225 2023-10-08 18:55 hdfs://xxx/user/xxx/bdm_dev.db/hudi_xxx/.hoodie/20231008185534458.rollback.requested
-rw-rw-rw-   3 xxx xxx       1408 2023-10-08 18:55 hdfs://xxx/user/xxx/bdm_dev.db/hudi_xxx/.hoodie/20231008185534699.rollback
-rw-rw-rw-   3 xxx xxx          0 2023-10-08 18:55 hdfs://xxx/user/xxx/bdm_dev.db/hudi_xxx/.hoodie/20231008185534699.rollback.inflight
-rw-rw-rw-   3 xxx xxx       1225 2023-10-08 18:55 hdfs://xxx/user/xxx/bdm_dev.db/hudi_xxx/.hoodie/20231008185534699.rollback.requested
-rw-rw-rw-   3 xxx xxx       1408 2023-10-08 18:55 hdfs://xxx/user/xxx/bdm_dev.db/hudi_xxx/.hoodie/20231008185535030.rollback
-rw-rw-rw-   3 xxx xxx          0 2023-10-08 18:55 hdfs://xxx/user/xxx/bdm_dev.db/hudi_xxx/.hoodie/20231008185535030.rollback.inflight
-rw-rw-rw-   3 xxx xxx       1225 2023-10-08 18:55 hdfs://xxx/user/xxx/bdm_dev.db/hudi_xxx/.hoodie/20231008185535030.rollback.requested
-rw-rw-rw-   3 xxx xxx       1408 2023-10-08 18:55 hdfs://xxx/user/xxx/bdm_dev.db/hudi_xxx/.hoodie/20231008185535295.rollback
-rw-rw-rw-   3 xxx xxx          0 2023-10-08 18:55 hdfs://xxx/user/xxx/bdm_dev.db/hudi_xxx/.hoodie/20231008185535295.rollback.inflight
-rw-rw-rw-   3 xxx xxx       1225 2023-10-08 18:55 hdfs://xxx/user/xxx/bdm_dev.db/hudi_xxx/.hoodie/20231008185535295.rollback.requested
-rw-rw-rw-   3 xxx xxx       1408 2023-10-08 18:55 hdfs://xxx/user/xxx/bdm_dev.db/hudi_xxx/.hoodie/20231008185536332.rollback
-rw-rw-rw-   3 xxx xxx          0 2023-10-08 18:55 hdfs://xxx/user/xxx/bdm_dev.db/hudi_xxx/.hoodie/20231008185536332.rollback.inflight
-rw-rw-rw-   3 xxx xxx       1225 2023-10-08 18:55 hdfs://xxx/user/xxx/bdm_dev.db/hudi_xxx/.hoodie/20231008185536332.rollback.requested
-rw-rw-rw-   3 xxx xxx       1408 2023-10-08 18:55 hdfs://xxx/user/xxx/bdm_dev.db/hudi_xxx/.hoodie/20231008185536674.rollback
-rw-rw-rw-   3 xxx xxx          0 2023-10-08 18:55 hdfs://xxx/user/xxx/bdm_dev.db/hudi_xxx/.hoodie/20231008185536674.rollback.inflight
-rw-rw-rw-   3 xxx xxx       1225 2023-10-08 18:55 hdfs://xxx/user/xxx/bdm_dev.db/hudi_xxx/.hoodie/20231008185536674.rollback.requested
-rw-rw-rw-   3 xxx xxx       1408 2023-10-08 18:55 hdfs://xxx/user/xxx/bdm_dev.db/hudi_xxx/.hoodie/20231008185537030.rollback
-rw-rw-rw-   3 xxx xxx          0 2023-10-08 18:55 hdfs://xxx/user/xxx/bdm_dev.db/hudi_xxx/.hoodie/20231008185537030.rollback.inflight
-rw-rw-rw-   3 xxx xxx       1225 2023-10-08 18:55 hdfs://xxx/user/xxx/bdm_dev.db/hudi_xxx/.hoodie/20231008185537030.rollback.requested

Impact

no

Risk level (write none, low medium or high below)

low

Documentation Update

Describe any necessary documentation update if there is any new feature, config, or user-facing change

• The config description must be updated if new configs are added or the default value of the configs are changed
• Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
  ticket number here and follow the instruction to make
  changes to the website.

Contributor's checklist

• Read through contributor's guide
• Change Logs and Impact were stated clearly
• Adequate tests were added if applicable
• CI passed

[hudi-bot]

CI report:

• 5877285 Azure: SUCCESS

Bot commands

@hudi-bot supports the following commands:

• @hudi-bot run azure re-run the last Azure build