Skip to content

Conversation

@jacobtomlinson
Copy link
Member

Closes #475
Closes #544

The KubeCluster class can take a selection of keyword arguments to make it quick and easy to get started, however the underlying DaskCluster resource can be much more complex and configured in many ways.
Rather than exposing every possibility via keyword arguments instead, you can pass a valid DaskCluster resource spec which will be used when creating the cluster.
You can also generate a spec with make_cluster_spec which KubeCluster uses internally and then modify it with your custom options.

   from dask_kubernetes.experimental import KubeCluster, make_cluster_spec

   config = {
      "name": "foo",
      "n_workers": 2,
      "resources":{"requests": {"memory": "2Gi"}, "limits": {"memory": "64Gi"}}
   }

   cluster = KubeCluster(**config)
   # is equivalent to
   cluster = KubeCluster(custom_cluster_spec=make_cluster_spec(**config))

You can also modify the spec before passing it to KubeCluster, for example if you want to set nodeSelector on your worker pods you could do it like this:

   from dask_kubernetes.experimental import KubeCluster, make_cluster_spec

   spec = make_cluster_spec(name="selector-example", n_workers=2)
   spec["spec"]["worker"]["spec"]["nodeSelector"] = {"disktype": "ssd"}

   cluster = KubeCluster(custom_cluster_spec=spec)

The cluster.add_worker_group() method also supports passing a custom_spec keyword argument which can be generated with make_worker_spec.

   from dask_kubernetes.experimental import KubeCluster, make_worker_spec

   cluster = KubeCluster(name="example")

   worker_spec = make_worker_spec(cluster_name=cluster.name, n_workers=2, resources={"limits"{"nvidia.com/gpu": 1}})
   worker_spec["spec"]["nodeSelector"] = {"cloud.google.com/gke-nodepool": "gpu-node-pool"}

   cluster.add_worker_group(custom_spec=worker_spec)

I also took this opportunity to remove the type suffixes mentioned in #544 because they were confusing me while writing this.

@jacobtomlinson jacobtomlinson merged commit 8540f76 into dask:main Aug 26, 2022
@jacobtomlinson jacobtomlinson deleted the custom_spec branch August 26, 2022 11:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove type suffixes Support Pod and Service specs in experimental.KubeCluster

1 participant