Clean up Ray processes after cluster util exits

[richardliaw]

Adds atexit hook to Cluster instantiation.

Closes #3277.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9204/
Test PASSed.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9208/
Test PASSed.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9210/
Test PASSed.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/9211/
Test PASSed.

[robertnishihara]

What is this intended to test? It seems redundant with the above test. Also, it doesn't hit the atexit code path right? Because the Python interpreter is not exiting.

[richardliaw]

It's to test that the shutdown endpoint works as expected.

[robertnishihara]

Ok, I see. It's not testing the atexit stuff though, right?

[richardliaw]

No; it's just an additional test piggy-backing on this approval..

[robertnishihara]

If you want to test this functionality, I think you need to do something like

ray.utils.run_string_as_driver(
"""
import ...
g = Cluster(initialize_head=False)
node = g.add_node()
node2 = g.add_node()
"""
)

# Then in python, check that the raylets and plasma stores and such are gone.