SAMZA-2441: Update ApplicationRunnerMain#ApplicationRunnerCommandLine not to load local file#1265
Conversation
… not to load local file Design: https://cwiki.apache.org/confluence/display/SAMZA/SEP-23%3A+Simplify+Job+Runner Changes: 1. Override ApplicationRunnerCommandLine#loadConfig not to invoke config loader. 2. Add JobRunnerCommandLine for JobRunner as it still needs full job config for special use cases. API Changes: None Upgrade Instructions: None Usage Instructions: None Tests: 1. Unit Tests
| } | ||
|
|
||
| @Override | ||
| public Config loadConfig(OptionSet options) { |
There was a problem hiding this comment.
Do you feel that the "base" use case for CommandLine is to need the full config? To me, it looks a little awkward to need to override loadConfig here with an implementation which is almost the same as the base loadConfig.
I suppose one alternative would be to make this impl here the base impl, and then have each use case call ConfigUtil.loadConfig if is needs it. What do you see as the tradeoffs?
There was a problem hiding this comment.
Good question, in all 13 direct usage of CommandLine, and 5 sub classes of CommandLine, they are all expecting full job configs to work properly, e.g. RocksDbReadingTool extends CommandLine to read full job config before being able to read data from Rocks DB, which is being invoked by CLI.
If we change CommandLine to the other way, that means we need to invoke ConfigUtil.loadConfig explicitly in 13 places, and override loadConfigs in 5 sub classes to invoke ConfigUtil.loadConfig too, compared to overriding once in ApplicationRunnerCommandLine only.
I would prefer the current way to keep the base class to serve majority use case.
…mandLine not to load local file (apache#1265)" This reverts commit 5a58877. Reverting this change for the 1.4.0 branch since it is backwards incompatible and we don't want to release this for 1.4.
This is backward incompatible and is only supposed to be included in Samza 1.5
Design:
https://cwiki.apache.org/confluence/display/SAMZA/SEP-23%3A+Simplify+Job+Runner
Changes:
API Changes:
None
Upgrade Instructions:
None
Usage Instructions:
None
Tests: