[AMORO-3192] Add the "clean-orphan-file.interval-time-minutes" parameter

[lintingbin]

Why are the changes needed?

Close #3192.

Brief change log

• Add a parameter to replace the INTERVAL constant, which was previously fixed at 24 hours in the OrphanFilesCleaningExecutor class.

How was this patch tested?

• Add some test cases that check the changes thoroughly including negative and positive cases if possible

• Add screenshots for manual tests if appropriate

• Run test locally before making a pull request

Documentation

• Does this pull request introduce a new feature? yes
• If yes, how is the feature documented? docs

[tcodehuber]

Thanks for your work, but I think the default check interval should be less than the default min existing time.

[lintingbin]

Thanks for your work, but I think the default check interval should be less than the default min existing time.

Why is this necessary? The "min existing time" ensures that useful files are not mistakenly deleted, so setting it to two to three days is usually sufficient. However, cleaning orphan files is a performance-intensive task, so it's generally appropriate to configure the cleanup to occur once a week or once a month.

[zhoujinsong]

@lintingbin Thanks for the contribution.

The scheduling interval for the current expiration task is usually set as an AMS configuration like data-expiration.interval, which applies to all tables. I think this meets the requirements, as we generally do not adjust different execution interval for different tables. What do you think?

BTW, we are improving the configuration value for time duration with unit support like 3d, we can do it for the new configuration.

[zhoujinsong]

Why is this necessary? The "min existing time" ensures that useful files are not mistakenly deleted, so setting it to two to three days is usually sufficient. However, cleaning orphan files is a performance-intensive task, so it's generally appropriate to configure the cleanup to occur once a week or once a month.

I agree with @lintingbin, there is no connection between the values of these two configurations.

[lintingbin]

@lintingbin Thanks for the contribution.

The scheduling interval for the current expiration task is usually set as an AMS configuration like data-expiration.interval, which applies to all tables. I think this meets the requirements, as we generally do not adjust different execution interval for different tables. What do you think?

BTW, we are improving the configuration value for time duration with unit support like 3d, we can do it for the new configuration.

The modifications have been completed according to the suggestions.

[zhoujinsong]

LGTM.