Unable to deploy a docker container in "privileged" mode when user namespace is enabled for docker daemon

[ohmydata]

Nomad version

Nomad v0.4.1

Operating system and Environment details

Linux mo-9c187bcb3 4.2.0-27-generic #32~14.04.1-Ubuntu SMP Fri Jan 22 15:32:26 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

Issue

According to Docker documentation: https://docs.docker.com/engine/reference/commandline/dockerd/

Disable user namespace for a container

If you enable user namespaces on the daemon, all containers are started with user namespaces enabled. In some situations you might want to disable this feature for a container, for example, to start a privileged container (see user namespace known restrictions). To enable those advanced features for a specific container use --userns=host in the run/exec/create command. This option will completely disable user namespace mapping for the container’s user.

nomad run <>.hcl with

privileged = true			
uts_mode = "host"

fails with an exception :
API error (500): {"message":"Privileged mode is incompatible with user namespaces"}.

I am able to run the container in privileged mode directly like "docker run -itd --privileged --userns=host". (It does not run without the --userns=host flag). I tried adding args = ["--userns=host"] in the hcl, it still gave the error API error (500): {"message":"Privileged mode is incompatible with user namespaces"}.

Reproduction steps

1. Enable user namespace in docker daemon: --userns-remap=default
2. Configure Nomad agent
   client {
   options = {
   "driver.raw_exec.enable" = "1"
   "docker.privileged.enabled" = "true"
   }
3. Write a nomad.hcl with :
   privileged = true
   uts_mode = "host"
4. nomad run nomad.hcl

Nomad Server logs (if appropriate)

failed to create container from image API error (500): {"message":"Privileged mode is incompatible with user namespaces"}

Nomad Client logs (if appropriate)

Job file (if appropriate)

config {
............................
............................
privileged = true
uts_mode = "host"
}

[schmichael]

I confirmed this bug in both 0.4.1 and 0.5-rc1 with the following:

Added --userns-remap=default in my docker.service file:

ExecStart=/usr/bin/dockerd -H fd:// --userns-remap=default

Use the following nomad configuration:

client {
  enabled = true

  options = {
    "driver.raw_exec.enable"    = "1"
    "docker.privileged.enabled" = "true"
  }
}

Added the following to the example job file:

config {
        image      = "redis:3.2"
        privileged = true
        uts_mode   = "host"

        port_map {
...

We'll get this fixed for 0.5.

[schmichael]

I believe the docker setting you're looking for is --userns if you're using user namespaces. Nomad 0.4.1/0.5-rc1 doesn't support setting --userns=host.

I added --userns=host support under a userns_mode config option (like uts_mode) in #1940 and it will ship in 0.5-rc2 early next week.

[schmichael]

(As always feel free to reopen if that option isn't what you needed or doesn't address your issue.)

[huan]

If you enable user namespaces on the daemon, all containers are started with user namespaces enabled. In some situations you might want to disable this feature for a container, for example, to start a privileged container (see user namespace known restrictions). To enable those advanced features for a specific container use --userns=host in the run/exec/create command.

--userns=host helps me solve my problem.

[github-actions]

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.