Is possible to adding dinamically workers during the execution of scheduler ?

[giuseppe1993]

Hello, I was trying dask on a cluster.
In this cluster the number of nodes can increase or decrease during the resolution of problem.
Is possible doing this with dask ?

[mrocklin]

Yes

…

On Mon, Sep 24, 2018 at 10:25 AM Giuseppe Capaldo ***@***.***> wrote: Hello, I was trying dask on a cluster. In this cluster the number of nodes can increase or decrease during the resolution of problem. Is possible doing this with dask ? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#2269>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AASszCr8_QaowmqVNI4yYDG0C1yXDYojks5ueOtmgaJpZM4W2w6g> .

[VMois]

Yes, it's possible. The preferable way of setting up Dask cluster is using Kubernetes cluster.

This is a few links to help you:

• https://github.com/dask/dask-kubernetes (for dynamic Kubernetes Dask cluster)
• https://dask.pydata.org/en/latest/setup/kubernetes-helm.html (for static Kubernetes Dask cluster)
• https://github.com/VMois/dask-k8s-chart (it's Helm chart for setting up dynamic Kubernetes cluster, work is based on two above repositories, now, it's under active development)

BTW, I think, for this type of question is better to use StackOverflow.

[giuseppe1993]

Thank you very much for helping.
I apologize for posting the problem here, with this I close the issue goodbye

[mrocklin]

FWIW, kubernetes is only one way, not necessarily the preferred way. My experience is that groups don't have the ability to choose their cluster manager. It is usually chosen for them.

…

On Tue, Sep 25, 2018 at 5:27 AM Giuseppe Capaldo ***@***.***> wrote: Closed #2269 <#2269>. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2269 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AASszD3ASXZciEmt5E-16ZPz5q_mQIIYks5uefbqgaJpZM4W2w6g> .