From 3d023252959e9bce7095e5ecac665fc71dd91544 Mon Sep 17 00:00:00 2001 From: Katharine Hyatt Date: Thu, 7 Mar 2019 02:06:30 -0500 Subject: [PATCH] Add docs for `ClusterManager` and `WorkerConfig` (#31234) --- stdlib/Distributed/docs/src/index.md | 2 ++ stdlib/Distributed/src/cluster.jl | 39 ++++++++++++++++++++++++++++ 2 files changed, 41 insertions(+) diff --git a/stdlib/Distributed/docs/src/index.md b/stdlib/Distributed/docs/src/index.md index 3ecbd51735667..eea3dbbc0d8f8 100644 --- a/stdlib/Distributed/docs/src/index.md +++ b/stdlib/Distributed/docs/src/index.md @@ -60,6 +60,8 @@ same host, and `SSHManager`, for launching on remote hosts via `ssh`. TCP/IP soc and transport messages between processes. It is possible for Cluster Managers to provide a different transport. ```@docs +Distributed.ClusterManager +Distributed.WorkerConfig Distributed.launch Distributed.manage Distributed.kill(::ClusterManager, ::Int, ::WorkerConfig) diff --git a/stdlib/Distributed/src/cluster.jl b/stdlib/Distributed/src/cluster.jl index f753fe607250c..7705f03ef3e52 100644 --- a/stdlib/Distributed/src/cluster.jl +++ b/stdlib/Distributed/src/cluster.jl @@ -1,7 +1,46 @@ # This file is a part of Julia. License is MIT: https://julialang.org/license +""" + ClusterManager + +Supertype for cluster managers, which control workers processes as a cluster. +Cluster managers implement how workers can be added, removed and communicated with. +`SSHManager` and `LocalManager` are subtypes of this. +""" abstract type ClusterManager end +""" + WorkerConfig + +Type used by [`ClusterManager`](@ref)s to control workers added to their clusters. Some fields +are used by all cluster managers to access a host: + * `io` -- the connection used to access the worker (a subtype of `IO` or `Nothing`) + * `host` -- the host address (either an `AbstractString` or `Nothing`) + * `port` -- the port on the host used to connect to the worker (either an `Int` or `Nothing`) + +Some are used by the cluster manager to add workers to an already-initialized host: + * `count` -- the number of workers to be launched on the host + * `exename` -- the path to the Julia executable on the host, defaults to `"\$(Sys.BINDIR)/julia"` or + `"\$(Sys.BINDIR)/julia-debug"` + * `exeflags` -- flags to use when lauching Julia remotely + +The `userdata` field is used to store information for each worker by external managers. + +Some fields are used by `SSHManager` and similar managers: + * `tunnel` -- `true` (use tunneling), `false` (do not use tunneling), or [`nothing`](@ref) (use default for the manager) + * `bind_addr` -- the address on the remote host to bind to + * `sshflags` -- flags to use in establishing the SSH connection + * `max_parallel` -- the maximum number of workers to connect to in parallel on the host + +Some fields are used by both `LocalManager`s and `SSHManager`s: + * `connect_at` -- determines whether this is a worker-to-worker or driver-to-worker setup call + * `process` -- the process which will be connected (usually the manager will assign this during [`addprocs`](@ref)) + * `ospid` -- the process ID according to the host OS, used to interrupt worker processes + * `environ` -- private dictionary used to store temporary information by Local/SSH managers + * `ident` -- worker as identified by the [`ClusterManager`](@ref) + * `connect_idents` -- list of worker ids the worker must connect to if using a custom topology + * `enable_threaded_blas` -- `true`, `false`, or `nothing`, whether to use threaded BLAS or not on the workers +""" mutable struct WorkerConfig # Common fields relevant to all cluster managers io::Union{IO, Nothing}