[GOBBLIN-1372] Generalization of GobblinClusterUtils#setSystemProperties #3213
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Dear Gobblin maintainers,
Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!
JIRA
https://issues.apache.org/jira/browse/GOBBLIN-1372
Description
This PR mainly is for refactoring the method
GobblinClusterUtils#setSystemProperties
. The problem was: There are some unnecessary details being exposed:JAVA_TMP_DIR_KEY
and resolution ofYARN_CACHE
are all very specific to Gobblin-on-YARN applications. These things shouldn't be set through if-branch but through configurations only.After refactoring:
There's no specific key/value matching within
setSystemProperties
method, and we don't lose the capability for limiting resolution ofYARN_CACHE
only for a specific set of configuration with being backward-compatible.Also, the refactor also let the program to set additional configuration with value specified as
YARN_CACHE
using the following pairs:If trying to achieve the same purpose as
-Dgrape.root=YARN_CACHE
Also, it adds a bunch of entries in the .gitignore file.
Tests
Unit tested with resolution cases.
Commits