NIFI-2471 fix Hadoop configuration resources when talking to ... #779
Conversation
…ple Hadoop clusters
| @@ -177,9 +178,17 @@ protected KerberosProperties getKerberosProperties() { | |||
| return results; | |||
| } | |||
|
|
|||
| @Override | |||
| public void onPropertyModified(PropertyDescriptor descriptor, String oldValue, String newValue) { | |||
There was a problem hiding this comment.
@mosermw I'm not sure that this is really necessary. The abstractOnStopped() method resets hdfsResources to new HdfsResources(null, null, null) - so every time that it's stopped, it gets reset. I think maybe the only change needed is the change below to config.set(disableCacheName, "true") ??
There was a problem hiding this comment.
Thanks for looking at this @markap14. I originally thought the same way you describe. During testing, though, the AbstractHadoopProcessor's OnScheduled method works but the GetHDFS OnScheduled method throws a "Failed to invoke OnScheduled method due to java.lang.RuntimeException: Caused by java.io.IOException: PropertyDescriptor[Directory] has invalid value /some/directory. The directory does not exist." When stopping the invalid processor, the abstractOnStopped() method is not called. Perhaps that's an issue unto itself, but adding an onPropertyModified method here fixed the problem that HdfsResources was not null the next time the processor was started.
There was a problem hiding this comment.
Looks like you're right - it is not calling @OnStopped methods when @OnScheduled throws an Exception. I created a ticket for that - https://issues.apache.org/jira/browse/NIFI-2484. I think we should avoid working around the bug in this PR though and just address it in the framework.
There was a problem hiding this comment.
Alrighty, no problem. I've added a commit to this PR to remove this onPropertyModified method. Hopefully the two commits can be squashed to one. Thanks!
… Hadoop clusters This closes #779.
|
OK thanks @mosermw. Looks good. Merged to master & 0.x branches. |
…multiple Hadoop clusters