NIFI-2199 & NIFI-3112 - Handle propagation of stderr and allow ssh restart

[apiri]

NIFI-2199
NIFI-3112
Pipe stdout to /dev/null and allow stderr to propagate through.

This correction does not make NiFi fully LSB spec compliant but does solve the two issues presented in the listed tickets. This will allow stderr to propagate through and restart through SSH.

ssh can be tested by issuing a
ssh user@remote-host <path to nifi>/bin/nifi.sh restart
from a local machine to a remote host

3112 can be verified by changing a property to bootstrap causing it to be invalid and running the above. As listed in the prior discussion of 3112, making graceful.shutdown.seconds a negative number such as below:

graceful.shutdown.seconds=-20

Will cause an error to be triggered and the appropriate contents transmitted.

[michalklempa]

Thank you Aldrin, as a part of NIFI-2199 I was thinking about the solution of changing the RunNifi java app to close the relevant streams. Would be cleaner and as I already introduced problems with stream redirection using bash, i would go for changing the Java part.

[gresockj]

I tried this out, and it does appear to work when I run ssh user@remote-host "sudo service nifi restart". However, when I do the same with a salt command, it still hangs:
salt nifi cmd.run "sudo service nifi restart"

I believe salt uses the python "subprocess" call to invoke a command this way. Since this was working in 1.1.x, I feel like there should be a way to get it working again in 1.2.x.

Quick note: I changed 1>&- to 2>&- in order to also direct syserr to /dev/null, and this actually appears to work with salt. However, I don't know what adverse effects it may have. FWIW, I still saw the log statements in nifi-bootstrap.log during startup.

[apiri]

Hey @gresockj,

Don't have a lot of familiarity with python or Salt. Would you happen to have a sample salt script you would be able to share? I have a couple ideas on how to work with RunNiFi to make this happen.

[gresockj]

Salt is easy to set up with this walkthrough. You can run salt-master and salt-minion on the same nifi box if you want in order to reproduce this behavior. Once you have it installed, you can run the command:

salt 'minionID' cmd.run "service nifi start"

[apiri]

thanks, will see what I can find

[kevdoran]

@apiri I'd be happy to help dig into the salt use case if you would like

[apiri]

@kevdoran Sounds good! I propose we incorporate this PR for the time being to solve the prior two cases listed in the mentioned JIRAs and can open up a new JIRA to track issues with Salt and presumably other tools/scripts that may be Python driven. @gresockj let me know your thoughts. I will create a separate issue to track the Python side of the house.

[apiri]

Created https://issues.apache.org/jira/browse/NIFI-3899

[kevdoran]

@apiri sounds good

[YolandaMDavis]

@apiri will review

[YolandaMDavis]

@apiri validated ssh command on remote host with with start/stop/restart. All worked as expected, with NiFi starting appropriately and the IO stream for ssh was closed. Also validated that bootstrap errors are appropriately routed to the output stream and displayed before stream is closed.

+1

will merge shortly. Thanks Aldrin!