Client.shutdown claims to close cluster, but doesn't

[mrocklin]

Client.shutdown claims in its docstring to tear down the cluster. This isn't currently the case. It only closes its own connection. We can either

1. Change the docstring
2. Change the behavior

If we change only the docstring, then should we also add the shutdown functionality to the client?

If we change the behavior then should we also add a close method?

http://stackoverflow.com/questions/44021931/stopping-dask-ssh-created-scheduler-from-the-client-interface

[martindurant]

I believe there should be a method to tear down the cluster from the client. "shutdown" seems like the best name for this, so I would go for option 2. "close" would be the same thing as what happens on del?

[pitrou]

I believe things would be less confusing if cluster-affecting methods would be named explicitly, such as restart_cluster instead of restart and shutdown_cluster instead of shutdown. If I were to read user code without perusing the docs, I would never guess that Client.restart restarts the cluster, not the client.

Agreed to also have a separate close method that does what shutdown does right now, and have __del__ call it. It's good to allow explicit resource cleanup when an object can keep many resources alive.

[eriknw]

I'm in favor of restart_cluster, shutdown_cluster, and close methods. I'll work on a PR.

[GenevieveBuckley]

@eriknw are you still interested in working on this?

[eriknw]

Welp. I remember poking around at this, but I can't find my previous work. Oops. I doubt I'll be able to work on this this quarter.

[GenevieveBuckley]

Ok then. This might be a candidate for a good first or second issue, if someone is happy to leave a comment outlining roughly what should be done.

[xsqian]

Hi there, any progress on this?

[jrbourbeau]

There have been some related updates and today client.shutdown() does close the cluster

In [1]: from distributed import Client, LocalCluster

In [2]: cluster = LocalCluster()

In [3]: client = Client(cluster)

In [4]: cluster.status
Out[4]: <Status.running: 'running'>

In [5]: client.shutdown()

In [6]: cluster.status
Out[6]: <Status.closed: 'closed'>

Closing as resolved, though we can reopen as needed