[SUPPORT] Partition query result is unexpected because some partitions missed .hoodie_partition_metadata file

[beyond1920]

Dear community,
I recently encountered a problem: The .hoodie_partition_metadata does not exist in leaf partition path. But the writer job completed successfully. It would result in wrong query results.
It does not only happened for mor table, but also for cow table.
For example:

It would caused unexpected result of following SQL query because FileSystemBackedTableMetadata would skip p_date=20230219 when get partition paths list.
select * from sourceT where p_date between '20230201' and '20230230 '.

The writer job completed success, but the spark logs are already cleaned, so I could not find more information.

I guess something wrong when tasks call HoodiePartitionMetadata#trySave. No task ever created the hoodie_partition_metadata file successfully, but all exceptions are all ignored here.

Is there anyone encountered the similar issue?
Are there any suggestions?

[beyond1920]

Use 0.14 HUDI version.
and Use HDFS as storage.

[wecharyu]

Did you enable spark speculation? We have encountered this issue, will push a PR for discussion. cc: @boneanxs

[beyond1920]

Fixed in PR#10886