[HUDI-6076] Fix the unit of clustering.plan.strategy.target.file.max.bytes for clustering

[Zouxxyy]

Change Logs

clustering.plan.strategy.target.file.max.bytes's unit is byte, clustering.plan.strategy.small.file.limit should be unified with it. And they also compare sizes somewhere, there's not even a uniform unit here, like this

  this.conf.setLong(FlinkOptions.CLUSTERING_PLAN_STRATEGY_SMALL_FILE_LIMIT.key(),
      this.conf.getLong(FlinkOptions.CLUSTERING_PLAN_STRATEGY_TARGET_FILE_MAX_BYTES) > this.conf.getLong(FlinkOptions.CLUSTERING_PLAN_STRATEGY_SMALL_FILE_LIMIT)
        ? this.conf.getLong(FlinkOptions.CLUSTERING_PLAN_STRATEGY_SMALL_FILE_LIMIT)
          : this.conf.getLong(FlinkOptions.CLUSTERING_PLAN_STRATEGY_TARGET_FILE_MAX_BYTES));

Impact

Fix the unit of clustering.plan.strategy.target.file.max.bytes for clustering

Risk level (write none, low medium or high below)

low

Documentation Update

none

Contributor's checklist

• Read through contributor's guide
• Change Logs and Impact were stated clearly
• Adequate tests were added if applicable
• CI passed

[Zouxxyy]

@danny0405 Can you help with a review?

[SteNicholas]

@Zouxxyy, did you take the compatibility of this change into consideration? With this change, the config value of clustering.plan.strategy.small.file.limit must be changed when upgrade to the lastest version. IMO, you should introduce new option for the config unification.

[danny0405]

@Zouxxyy, did you take the compatibility of this change into consideration? With this change, the config value of clustering.plan.strategy.small.file.limit must be changed when upgrade to the lastest version. IMO, you should introduce new option for the config unification.

A valid concern, this may break the user code, we should take the modification as early and add some illustrations on the release note.

[Zouxxyy]

A valid concern, this may break the user code, we should take the modification as early and add some illustrations on the release note.

@SteNicholas @danny0405 So should we add new configuration, or just fix the bug and not change the unit?

[danny0405]

just fix the bug and not change the unit

Let's fix the bug first.

[Zouxxyy]

@danny0405 done

[hudi-bot]

CI report:

• e91532f Azure: FAILURE

Bot commands

@hudi-bot supports the following commands:

• @hudi-bot run azure re-run the last Azure build