Enable passing custom specs #549
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #475
Closes #544
The
KubeClusterclass can take a selection of keyword arguments to make it quick and easy to get started, however the underlyingDaskClusterresource can be much more complex and configured in many ways.Rather than exposing every possibility via keyword arguments instead, you can pass a valid
DaskClusterresource spec which will be used when creating the cluster.You can also generate a spec with
make_cluster_specwhichKubeClusteruses internally and then modify it with your custom options.You can also modify the spec before passing it to
KubeCluster, for example if you want to setnodeSelectoron your worker pods you could do it like this:The
cluster.add_worker_group()method also supports passing acustom_speckeyword argument which can be generated withmake_worker_spec.I also took this opportunity to remove the type suffixes mentioned in #544 because they were confusing me while writing this.