[SUPPORT] cleaner service is not very well documented

[bithw1]

Hi, I am reading at https://hudi.apache.org/docs/hoodie_cleaner,

The usage of utility tool HoodieCleaner is not very well documented, it gives an example

[hoodie]$ spark-submit --class org.apache.hudi.utilities.HoodieCleaner \
  --props s3:///temp/hudi-ingestion-config/kafka-source.properties \
  --target-base-path s3:///temp/hudi \
  --spark-master yarn-cluster

but I don't know the content of the prop file：kafka-source.properties

I would like to know how to specify the configuration property, that I want to keep the latest 3 data versions when I run the cleaner

Thanks a lot

[yihua]

Hi @bithw1 thanks for reporting this. I created a Jira ticket, HUDI-4339, to track the work of improving the docs for HoodieCleaner.

If you want to keep the latest 3 file versions, you need to add these specific configs to the properties:

hoodie.cleaner.policy=KEEP_LATEST_FILE_VERSIONS
hoodie.cleaner.fileversions.retained=3

https://hudi.apache.org/docs/configurations#hoodiecleanerpolicy
https://hudi.apache.org/docs/configurations#hoodiecleanerfileversionsretained

[nsivabalan]

Closing the github issue as the question was answered. @bithw1 : if you have more questions, feel free to create new one.

[nsivabalan]

thanks @yihua