Skip to content

Commit

Permalink
Clarify expected uses cases, and how the Kubernetes cluster is found.
Browse files Browse the repository at this point in the history
  • Loading branch information
chrish42 authored and mrocklin committed Jul 9, 2018
1 parent 537c125 commit 238c31a
Show file tree
Hide file tree
Showing 2 changed files with 16 additions and 1 deletion.
12 changes: 11 additions & 1 deletion dask_kubernetes/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,9 @@ class KubeCluster(Cluster):
""" Launch a Dask cluster on Kubernetes
This starts a local Dask scheduler and then dynamically launches
Dask workers on a Kubernetes cluster.
Dask workers on a Kubernetes cluster. The Kubernetes cluster is taken
to be either the current one on which this code is running, or as a
fallback, the default one configured in a kubeconfig file.
**Environments**
Expand All @@ -37,6 +39,14 @@ class KubeCluster(Cluster):
libraries that you may wish to use (like NumPy, Pandas, or Scikit-Learn).
See examples below for suggestions on how to manage and check for this.
**Network**
Since the Dask scheduler is launched locally, for it to work, we need to
be able to open network connections between this local node and all the
workers nodes on the Kubernetes cluster. If the current process is not
already on a Kubernetes node, some network configuration will likely be
required to make this work.
**Resources**
Your Kubernetes resource limits and requests should match the
Expand Down
5 changes: 5 additions & 0 deletions doc/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,11 @@ Dask Kubernetes deploys Dask workers on Kubernetes clusters using native
Kubernetes APIs. It is designed to dynamically launch short-lived deployments
of workers during the lifetime of a Python process.

Currently, it is designed to be run from a pod on a Kubernetes cluster that
has permissions to launch other pods. However, it can also work with a remote
Kubernetes cluster (configured via a kubeconfig file), as long as it is possible
to open network connections with all the workers nodes on the remote cluster.

Install
-------

Expand Down

0 comments on commit 238c31a

Please sign in to comment.