[SUPPORT] Incremental cleaning broken in 0.11.0

[parisni]

hudi 0.11.0 / spark 3.2.1
I experience a major problem with cleaning, which time increase linearly with partitions number.

See below logs, of batch loop of inserts in a hudi table with growing number of partitions:

Incremental Cleaning mode is enabled
Total Partitions to clean : 6960, with policy KEEP_LATEST_COMMITS
Total Partitions to clean : 7080, with policy KEEP_LATEST_COMMITS
Total Partitions to clean : 7200, with policy KEEP_LATEST_COMMITS
Total Partitions to clean : 7320, with policy KEEP_LATEST_COMMITS
Total Partitions to clean : 7440, with policy KEEP_LATEST_COMMITS
Total Partitions to clean : 7560, with policy KEEP_LATEST_COMMITS
Total Partitions to clean : 7680, with policy KEEP_LATEST_COMMITS
Total Partitions to clean : 7800, with policy KEEP_LATEST_COMMITS

I debugged this and the cleaner allways fall back to brute force partition cleaning, on file system because the lastClean commit is always empty:

hudi/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanPlanner.java

Line 155 in a048e94

if (lastClean.isPresent()) {

hudi/hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/clean/CleanPlanner.java

Lines 207 to 214 in a048e94

	private List<String> getPartitionPathsForFullCleaning() {
	// Go to brute force mode of scanning all partitions
	try {
	// Because the partition of BaseTableMetadata has been deleted,
	// all partition information can only be obtained from FileSystemBackedTableMetadata.
	FileSystemBackedTableMetadata fsBackedTableMetadata = new FileSystemBackedTableMetadata(context,
	context.getHadoopConf(), config.getBasePath(), config.shouldAssumeDatePartitioning());
	return fsBackedTableMetadata.getAllPartitionPaths();

This makes the cleaner unable to work on table with large number of partitions

[parisni]

Apparently this behavior is expected when using bulk_insert operation, which cannot lead to cleaning and thus no clean commit get in the timeline.

When bulk_insert, then one shall disable automatic cleaning