REX-Ray process disappears unexpectantly or appears as defunct

[clintkitson]

There may be a situation depending on the Linux distribution where REX-Ray unexpectantly stops running or the process ends up in a defunct state.

In reviewing the current documentation we have, the guidance clearly leans towards using rexray start to start the REX-Ray process for servicing Volume Driver requests. This is actually does not seem like the proper way to run REX-Ray in the long term. This is valid mostly for initially getting the service running. Once the proper configuration is in place, REX-Ray should be stopped and then started properly using the init system.

The docs for installing should also be clear in the manual case. 1) Download 2) move to proper bin dir 3) rexray install 4) init based start

For example on Ubuntu and CoreOS, a systemctl start rexray followed by systemctl status rexray to ensure it started properly.

Simply issuing an interactive CLI command of rexray start is not a consistent approach to daemonizing the REX-Ray process and ensuring it runs through session restarts.

[clintkitson]

@akutz What do you think here? 1) Are misleading by specifying start as the command line option? 2) If so should we add some info after this operation to state that it should be ran via an init process? 3) I can work on some doc updates. I don't think I am seeing a section for doing a rexray install after binary download or for starting via init system.

[akutz]

Hi @clintonskitson,

It's an issue with CoreOS behaving in concert with newer Linux distributions. SystemD is configured to clean up all process when a CGroup exits -- and an SSH session is now part of a CGroup. See this and this for how to change this behavior. Ultimately I think your recommended documentation change as well as a patch to REX-Ray so that rexray start calls SystemD if it's present is the best solution.

[akutz]

Hi @clintonskitson,

I won't close this issue until your documentation PR is merged as well. Then I think we can consider this closed.

[clintkitson]

#327