Add HTTPS support for the dashboard #2812
Conversation
Adds optional HTTPS support for the scheduler dashboard. This is only available via the configuration file, by setting the following fields: - `distributed.scheduler.tls.cert`: the certificate file - `distributed.scheduler.tls.key`: the key file, optional if the key file is concatenated with the cert above - `distributed.scheduler.tls.ca-file`: the CA file, optional These certs *may* be the same as those used for the scheduler/worker/client communication, but aren't required to be. The user is responsible for making this decision and providing the proper configuration. Likewise, the user is responsible for providing trusted certificates, or understanding the security implications of telling their browser "I understand the risks, trust this certificate" (this is more likely, given the transient nature of dask clusters). The generated dashboard links now format on an optional `scheme` parameter, which is either `http` or `https`, depending on if the TLS configuration fields above are configured.
|
This works, but due to the transient nature of Dask clusters (with potentially non-consistent addresses), it's not easy to provide certificates that your browser will trust. In the long term (still too alpha for me to recommend), a different option would be to use dask-gateway, which provides a consistent public address for all dask dashboards, and supports HTTPS configuration. This allows a trusted certificate (e.g. from letsencrypt) to be used, since the public address is known and consistent for all clusters. |
|
Failure is unrelated (is this a known flaky test?). cc @mrocklin. |
| @@ -88,7 +93,7 @@ distributed: | |||
| ################### | |||
|
|
|||
| dashboard: | |||
| link: "http://{host}:{port}/status" | |||
| link: "{scheme}://{host}:{port}/status" | |||
There was a problem hiding this comment.
scheme -> protocol?
(scheme may also be a standard term here, I just haven't heard of it before)
There was a problem hiding this comment.
Scheme is what urllib.parse.urlparse uses.
There was a problem hiding this comment.
There was a problem hiding this comment.
Alrighty then. Thanks for the reference.
|
I've seen similar failures in the past. We can probably ignore it. |
|
The failure is maybe addressed in #2813 ? I'd give it about a 40% chance of resolving the issue :/ |
|
Thanks @jcrist ! Merging this in. |
|
I just installed the latest dask and friends. To configure https for the dashboard, do I copy distributed.yaml to my .config/dask directory, and edit it to specify certs, etc? |
|
Yes. For more information on our configuration system see https://docs.dask.org/en/latest/configuration.html. |
* upstream/master: (33 commits) SpecCluster: move init logic into start (dask#2850) Dont reuse closed worker in get_worker (dask#2841) Add alternative SSHCluster implementation (dask#2827) Extend prometheus metrics endpoint (dask#2792) (dask#2833) Include type name in SpecCluster repr (dask#2834) Don't make False add-keys report to scheduler (dask#2421) Add Nanny to worker docs (dask#2826) Respect security configuration in LocalCluster (dask#2822) bump version to 2.1.0 Fix typo that prevented error message (dask#2825) Remove dask-mpi (dask#2824) Updates to use update_graph in task journey docs (dask#2821) Fix Client repr with memory_info=None (dask#2816) Fix case where key, rather than TaskState, could end up in ts.waiting_on (dask#2819) Use Keyword-only arguments (dask#2814) Relax check for worker references in cluster context manager (dask#2813) Add HTTPS support for the dashboard (dask#2812) CLN: Use dask.utils.format_bytes (dask#2810) bump version to 2.0.1 Add python_requires entry to setup.py (dask#2807) ...
I think you meant: Is my assumption correct? @jcrist |
Adds optional HTTPS support for the scheduler dashboard. This is only
available via the configuration file, by setting the following fields:
distributed.scheduler.tls.cert: the certificate filedistributed.scheduler.tls.key: the key file, optional if the keyfile is concatenated with the cert above
distributed.scheduler.tls.ca-file: the CA file, optionalThese certs may be the same as those used for the
scheduler/worker/client communication, but aren't required to be. The
user is responsible for making this decision and providing the proper
configuration.
Likewise, the user is responsible for providing trusted certificates, or
understanding the security implications of telling their browser "I
understand the risks, trust this certificate" (this is more likely,
given the transient nature of dask clusters).
The generated dashboard links now format on an optional
schemeparameter, which is either
httporhttps, depending on if the TLSconfiguration fields above are configured.
Fixes #2775