Skip to content

Commit

Permalink
Add docs for ClusterManager and WorkerConfig (#31234)
Browse files Browse the repository at this point in the history
  • Loading branch information
kshyatt authored and fredrikekre committed Mar 7, 2019
1 parent 62f5c2f commit 3d02325
Show file tree
Hide file tree
Showing 2 changed files with 41 additions and 0 deletions.
2 changes: 2 additions & 0 deletions stdlib/Distributed/docs/src/index.md
Expand Up @@ -60,6 +60,8 @@ same host, and `SSHManager`, for launching on remote hosts via `ssh`. TCP/IP soc
and transport messages between processes. It is possible for Cluster Managers to provide a different transport.

```@docs
Distributed.ClusterManager
Distributed.WorkerConfig
Distributed.launch
Distributed.manage
Distributed.kill(::ClusterManager, ::Int, ::WorkerConfig)
Expand Down
39 changes: 39 additions & 0 deletions stdlib/Distributed/src/cluster.jl
@@ -1,7 +1,46 @@
# This file is a part of Julia. License is MIT: https://julialang.org/license

"""
ClusterManager
Supertype for cluster managers, which control workers processes as a cluster.
Cluster managers implement how workers can be added, removed and communicated with.
`SSHManager` and `LocalManager` are subtypes of this.
"""
abstract type ClusterManager end

"""
WorkerConfig
Type used by [`ClusterManager`](@ref)s to control workers added to their clusters. Some fields
are used by all cluster managers to access a host:
* `io` -- the connection used to access the worker (a subtype of `IO` or `Nothing`)
* `host` -- the host address (either an `AbstractString` or `Nothing`)
* `port` -- the port on the host used to connect to the worker (either an `Int` or `Nothing`)
Some are used by the cluster manager to add workers to an already-initialized host:
* `count` -- the number of workers to be launched on the host
* `exename` -- the path to the Julia executable on the host, defaults to `"\$(Sys.BINDIR)/julia"` or
`"\$(Sys.BINDIR)/julia-debug"`
* `exeflags` -- flags to use when lauching Julia remotely
The `userdata` field is used to store information for each worker by external managers.
Some fields are used by `SSHManager` and similar managers:
* `tunnel` -- `true` (use tunneling), `false` (do not use tunneling), or [`nothing`](@ref) (use default for the manager)
* `bind_addr` -- the address on the remote host to bind to
* `sshflags` -- flags to use in establishing the SSH connection
* `max_parallel` -- the maximum number of workers to connect to in parallel on the host
Some fields are used by both `LocalManager`s and `SSHManager`s:
* `connect_at` -- determines whether this is a worker-to-worker or driver-to-worker setup call
* `process` -- the process which will be connected (usually the manager will assign this during [`addprocs`](@ref))
* `ospid` -- the process ID according to the host OS, used to interrupt worker processes
* `environ` -- private dictionary used to store temporary information by Local/SSH managers
* `ident` -- worker as identified by the [`ClusterManager`](@ref)
* `connect_idents` -- list of worker ids the worker must connect to if using a custom topology
* `enable_threaded_blas` -- `true`, `false`, or `nothing`, whether to use threaded BLAS or not on the workers
"""
mutable struct WorkerConfig
# Common fields relevant to all cluster managers
io::Union{IO, Nothing}
Expand Down

0 comments on commit 3d02325

Please sign in to comment.