[GOBBLIN-25] Gobblin data-management run script and example configuration

[fermich]

This pull request adds a run script for the DatasetCleanerJob class which is used in the gobblin-data-management module to run retention process. There is also example configuration to complement the retention module docs.

[ibuenros]

Hi @fermich , thanks for setting up this script and example!

In terms of the script, I think it would be more appropriate to set up a CliApplication for this (see for example DecryptCli), which will also not require the Azkaban stuff.

[ibuenros]

You could put this in /hive/includes.conf instead.

[fermich]

@ibuenros CliApplication looks nice. I have prepared implementation of cli runner but it is failing on getting cluster configuration. Should I submit the app like hadoop jar command does or
CliApplication requires a special environment when running it?
Link to source: https://github.com/fermich/gobblin/blob/dataset-cleaner-cli/gobblin-data-management/src/main/java/gobblin/runtime/retention/DatasetCleanerCli.java

[ibuenros]

@fermich you need to make sure that environment variable HADOOP_HOME is defined. You can check the classpath Gobblin will use by running bin/gobblin classpath

[fermich]

@ibuenros After setting HADOOP_HOME it starts working like a charm! Thank you! Summing up, there are two additional changes:

• DatasetCleaner starts via CliApplication
• includes.conf is moved one directory up

[abti]

Issue: https://issues.apache.org/jira/browse/GOBBLIN-25

Please update your PR title with following prefix: [GOBBLIN-25]

[fermich]

@abti done

[abti]

Thanks.

+1 LGTM