diff --git a/website/content/docs/configuration/client.mdx b/website/content/docs/configuration/client.mdx index 6db9ded601a7..3fb250150753 100644 --- a/website/content/docs/configuration/client.mdx +++ b/website/content/docs/configuration/client.mdx @@ -196,6 +196,146 @@ the full list. As of Nomad 1.2, Nomad will never attempt to embed the `alloc_dir` in the chroot as doing so would cause infinite recursion. +### `options` Parameters + +~> Note: In Nomad 0.9 client configuration options for drivers were deprecated. +See the [plugin stanza][plugin-stanza] documentation for more information. + +The following is not an exhaustive list of options for only the Nomad +client. To find the options supported by each individual Nomad driver, please +see the [drivers documentation](/docs/drivers). + +- `"driver.allowlist"` `(string: "")` - Specifies a comma-separated list of + allowlisted drivers . If specified, drivers not in the allowlist will be + disabled. If the allowlist is empty, all drivers are fingerprinted and enabled + where applicable. + + ```hcl + client { + options = { + "driver.allowlist" = "docker,qemu" + } + } + ``` + +- `"driver.denylist"` `(string: "")` - Specifies a comma-separated list of + denylisted drivers . If specified, drivers in the denylist will be + disabled. + + ```hcl + client { + options = { + "driver.denylist" = "docker,qemu" + } + } + ``` + +- `"env.denylist"` `(string: see below)` - Specifies a comma-separated list of + environment variable keys not to pass to these tasks. Nomad passes the host + environment variables to `exec`, `raw_exec` and `java` tasks. If specified, + the defaults are overridden. If a value is provided, **all** defaults are + overridden (they are not merged). + + ```hcl + client { + options = { + "env.denylist" = "MY_CUSTOM_ENVVAR" + } + } + ``` + + The default list is: + + ```text + CONSUL_TOKEN + CONSUL_HTTP_TOKEN + VAULT_TOKEN + NOMAD_LICENSE + AWS_ACCESS_KEY_ID + AWS_SECRET_ACCESS_KEY + AWS_SESSION_TOKEN + GOOGLE_APPLICATION_CREDENTIALS + ``` + +- `"user.denylist"` `(string: see below)` - Specifies a comma-separated + denylist of usernames for which a task is not allowed to run. This only + applies if the driver is included in `"user.checked_drivers"`. If a value is + provided, **all** defaults are overridden (they are not merged). + + ```hcl + client { + options = { + "user.denylist" = "root,ubuntu" + } + } + ``` + + The default list is: + + ```text + root + Administrator + ``` + +- `"user.checked_drivers"` `(string: see below)` - Specifies a comma-separated + list of drivers for which to enforce the `"user.denylist"`. For drivers using + containers, this enforcement is usually unnecessary. If a value is provided, + **all** defaults are overridden (they are not merged). + + ```hcl + client { + options = { + "user.checked_drivers" = "exec,raw_exec" + } + } + ``` + + The default list is: + + ```text + exec + qemu + java + ``` + +- `"fingerprint.allowlist"` `(string: "")` - Specifies a comma-separated list of + allowlisted fingerprinters. If specified, any fingerprinters not in the + allowlist will be disabled. If the allowlist is empty, all fingerprinters are + used. + + ```hcl + client { + options = { + "fingerprint.allowlist" = "network" + } + } + ``` + +- `"fingerprint.denylist"` `(string: "")` - Specifies a comma-separated list of + denylisted fingerprinters. If specified, any fingerprinters in the denylist + will be disabled. + + ```hcl + client { + options = { + "fingerprint.denylist" = "network" + } + } + ``` + +- `"fingerprint.network.disallow_link_local"` `(string: "false")` - Specifies + whether the network fingerprinter should ignore link-local addresses in the + case that no globally routable address is found. The fingerprinter will always + prefer globally routable addresses. + + ```hcl + client { + options = { + "fingerprint.network.disallow_link_local" = "true" + } + } + ``` + ### `reserved` Parameters - `cpu` `(int: 0)` - Specifies the amount of CPU to reserve, in MHz. diff --git a/website/content/docs/drivers/docker.mdx b/website/content/docs/drivers/docker.mdx index 46b40fc1b0bc..20f47ec888f9 100644 --- a/website/content/docs/drivers/docker.mdx +++ b/website/content/docs/drivers/docker.mdx @@ -1000,6 +1000,101 @@ host system. wait before cancelling an in-progress pull of the Docker image as specified in `infra_image`. Defaults to `"5m"`. +## Client Configuration + +~> Note: client configuration options will soon be deprecated. Please use +[plugin options][plugin-options] instead. See the [plugin stanza][plugin-stanza] +documentation for more information. + +The `docker` driver has the following [client configuration +options](/docs/configuration/client#options): + +- `docker.endpoint` - If using a non-standard socket, HTTP or another location, + or if TLS is being used, `docker.endpoint` must be set. If unset, Nomad will + attempt to instantiate a Docker client using the `DOCKER_HOST` environment + variable and then fall back to the default listen address for the given + operating system. Defaults to `unix:///var/run/docker.sock` on Unix platforms + and `npipe:////./pipe/docker_engine` for Windows. + +- `docker.auth.config` - Allows an operator to specify a + JSON file which is in the dockercfg format containing authentication + information for a private registry, from either (in order) `auths`, + `credHelpers` or `credsStore`. + +- `docker.auth.helper` - Allows an operator to specify a + [credsStore](https://docs.docker.com/engine/reference/commandline/login/#credential-helper-protocol) + -like script on \$PATH to lookup authentication information from external + sources. The script's name must begin with `docker-credential-` and this + option should include only the basename of the script, not the path. + +- `docker.tls.cert` - Path to the server's certificate file (`.pem`). Specify + this along with `docker.tls.key` and `docker.tls.ca` to use a TLS client to + connect to the docker daemon. `docker.endpoint` must also be specified or this + setting will be ignored. + +- `docker.tls.key` - Path to the client's private key (`.pem`). Specify this + along with `docker.tls.cert` and `docker.tls.ca` to use a TLS client to + connect to the docker daemon. `docker.endpoint` must also be specified or this + setting will be ignored. + +- `docker.tls.ca` - Path to the server's CA file (`.pem`). Specify this along + with `docker.tls.cert` and `docker.tls.key` to use a TLS client to connect to + the docker daemon. `docker.endpoint` must also be specified or this setting + will be ignored. + +- `docker.cleanup.image` Defaults to `true`. Changing this to `false` will + prevent Nomad from removing images from stopped tasks. + +- `docker.cleanup.image.delay` A time duration, as [defined + here](https://golang.org/pkg/time/#ParseDuration), that defaults to `3m`. The + delay controls how long Nomad will wait between an image being unused and + deleting it. If a tasks is received that uses the same image within the delay, + the image will be reused. + +- `docker.volumes.enabled`: Defaults to `false`. Allows tasks to bind host paths + (`volumes`) inside their container and use volume drivers (`volume_driver`). + Binding relative paths is always allowed and will be resolved relative to the + allocation's directory. + +- `docker.volumes.selinuxlabel`: Allows the operator to set a SELinux label to + the allocation and task local bind-mounts to containers. If used with + `docker.volumes.enabled` set to false, the labels will still be applied to the + standard binds in the container. + +- `docker.privileged.enabled` Defaults to `false`. Changing this to `true` will + allow containers to use `privileged` mode, which gives the containers full + access to the host's devices. Note that you must set a similar setting on the + Docker daemon for this to work. + +- `docker.caps.allowlist`: A list of allowed Linux capabilities. Defaults to + `"CHOWN,DAC_OVERRIDE,FSETID,FOWNER,MKNOD,NET_RAW,SETGID,SETUID,SETFCAP, SETPCAP,NET_BIND_SERVICE,SYS_CHROOT,KILL,AUDIT_WRITE"`, which is the list of + capabilities allowed by docker by default, as [defined + here](https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities). + Allows the operator to control which capabilities can be obtained by tasks + using `cap_add` and `cap_drop` options. Supports the value `"ALL"` as a + shortcut for allowlisting all capabilities. + +- `docker.cleanup.container`: Defaults to `true`. This option can be used to + disable Nomad from removing a container when the task exits. Under a name + conflict, Nomad may still remove the dead container. + +- `docker.nvidia_runtime`: Defaults to `nvidia`. This option allows operators to select the runtime that should be used in order to expose Nvidia GPUs to the container. + +Note: When testing or using the `-dev` flag you can use `DOCKER_HOST`, +`DOCKER_TLS_VERIFY`, and `DOCKER_CERT_PATH` to customize Nomad's behavior. If +`docker.endpoint` is set Nomad will **only** read client configuration from the +config file. + +An example is given below: + +```hcl +client { + options { + "docker.cleanup.image" = "false" + } +} +``` + ## Client Attributes The `docker` driver will set the following client attributes: diff --git a/website/content/docs/drivers/raw_exec.mdx b/website/content/docs/drivers/raw_exec.mdx index 5ff807dda7c8..6087c93ca14c 100644 --- a/website/content/docs/drivers/raw_exec.mdx +++ b/website/content/docs/drivers/raw_exec.mdx @@ -98,6 +98,17 @@ plugin "raw_exec" { } ``` +Nomad versions before v0.9 use the following client configuration. This configuration is +also supported in Nomad v0.9.0, but is deprecated in favor of the plugin stanza: + +``` +client { + options = { + "driver.raw_exec.enable" = "1" + } +} +``` + ## Plugin Options - `enabled` - Specifies whether the driver should be enabled or disabled. @@ -111,6 +122,20 @@ plugin "raw_exec" { Nomad process. Using a cgroup significantly reduces Nomad's CPU usage when collecting process metrics. +## Client Options + +~> Note: client configuration options will soon be deprecated. Please use +[plugin options][plugin-options] instead. See the [plugin stanza][plugin-stanza] documentation for more information. + +- `driver.raw_exec.enable` - Specifies whether the driver should be enabled or + disabled. Defaults to `false`. + +- `driver.raw_exec.no_cgroups` - Specifies whether the driver should not use + cgroups to manage the process group launched by the driver. By default, + cgroups are used to manage the process tree to ensure full cleanup of all + processes started by the task. The driver only uses cgroups when Nomad is + launched as root, on Linux and when cgroups are detected. + ## Client Attributes The `raw_exec` driver will set the following client attributes: