Allow running dockerd as a non-root user (Rootless mode) #38050

AkihiroSuda · 2018-10-16T08:03:35Z

- What I did

Allow running dockerd in an unprivileged user namespace (rootless mode).
Close #37375

No SETUID/SETCAP binary is required, except newuidmap and newgidmap.

For Kubernetes integration, please refer to https://github.com/rootless-containers/usernetes .

This PR contains two commits, but the first one is same as #38038 (overlayfs in userns for Ubuntu).
I'll rebase this PR when #38038 gets merged. (Updated: #38083 is merged now)

- How I did it

By using user_namespaces(7), mount_namespaces(7), network_namespaces(7), and slirp4netns.

Please refer to docs/rootless.md for the details.

- How to verify it

Make sure /etc/subuid and /etc/subgid contain the entry for you

$ id -u
1001
$ whoami
penguin
$ grep ^$(whoami): /etc/subuid
penguin:231072:65536
$ grep ^$(whoami): /etc/subgid
penguin:231072:65536

Start daemon: dockerd-rootless.sh --experimental
Start client: docker -H unix://$XDG_RUNTIME_DIR/docker.sock run ...

Remarks:

Some distros such as Debian (excluding Ubuntu) and Arch Linux require sudo sh -c "echo 1 > /proc/sys/kernel/unprivileged_userns_clone".
Some distros require sudo modprobe ip_tables iptable_mangle iptable_nat iptable_filter.

Restrictions:

Only vfs graphdriver is supported. However, on Ubuntu and a few distros, overlay2 and overlay are also supported. Starting with Linux 4.18, we will be also able to implement FUSE snapshotters.
Cgroups (including docker top) and AppArmor are disabled at the moment. In future, Cgroups will be optionally available when delegation permission is confi
gured on the host.
Checkpoint is not supported at the moment.
Running rootless dockerd in rootless/rootful dockerd is also possible, but not fully tested.

- Description for the changelog

Allow running dockerd in an unprivileged user namespace (rootless mode)

- A picture of a cute animal (not mandatory but encouraged)

https://en.wikipedia.org/wiki/Little_penguin#/media/File:Eudyptula_minor_Bruny_1.jpg

Screenshot:

penguin0@suda-ws01:~$ id
uid=1002(penguin0) gid=1006(penguin0) groups=1006(penguin0)
penguin0@suda-ws01:~$ ps u
USER        PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
penguin0 122952  0.0  0.0  21484  5156 pts/3    Ss   16:58   0:00 /bin/bash -l
penguin0 123093  0.0  0.0  21484  5200 pts/4    Ss   16:58   0:00 /bin/bash -l
penguin0 123094  0.0  0.0 134792  2860 pts/4    S    16:58   0:00 (sd-pam)
penguin0 123252  0.0  0.0   4628   784 pts/4    S+   16:58   0:00 /bin/sh /usr/local/bin/dockerd-rootless.sh --experimental
penguin0 123253  0.0  0.0 105772  3696 pts/4    Sl+  16:58   0:00 rootlesskit --net=slirp4netns --mtu=65520 --copy-up=/etc --copy-up=/run /usr/local/bin/dockerd-rootless.sh --experimental
penguin0 123257  0.0  0.0 105516  4024 pts/4    Sl+  16:58   0:00 /proc/self/exe --net=slirp4netns --mtu=65520 --copy-up=/etc --copy-up=/run /usr/local/bin/dockerd-rootless.sh --experimental
penguin0 123265  0.0  0.0   2980  1072 pts/4    S+   16:58   0:00 slirp4netns --mtu 65520 123257 tap0
penguin0 123281  0.0  0.0   4628   828 pts/4    S+   16:58   0:00 /bin/sh /usr/local/bin/dockerd-rootless.sh --experimental
penguin0 123283  0.6  0.8 583536 65728 pts/4    Sl+  16:58   0:00 dockerd --experimental
penguin0 125126  0.0  0.0  38372  3688 pts/3    R+   17:00   0:00 ps u
penguin0@suda-ws01:~$ docker -H unix:///run/user/1002/docker.sock run --rm hello-world

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (amd64)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/get-started/

AkihiroSuda · 2018-10-16T08:07:02Z

cc @tonistiigi @tiborvass

codecov · 2018-10-16T19:56:05Z

Codecov Report

❗ No coverage uploaded for pull request base (master@50e63ad). Click here to learn what that means.
The diff coverage is 19.23%.

@@            Coverage Diff            @@
##             master   #38050   +/-   ##
=========================================
  Coverage          ?   36.54%           
=========================================
  Files             ?      610           
  Lines             ?    45368           
  Branches          ?        0           
=========================================
  Hits              ?    16581           
  Misses            ?    26497           
  Partials          ?     2290

sargun · 2018-10-17T06:20:30Z

How can you delegate cgroups? A piece of work prior to this might be supporting cgroup namespace?

AkihiroSuda · 2018-10-17T06:46:08Z

How can you delegate cgroups? A piece of work prior to this might be supporting cgroup namespace?

Cgroups delegation is disabled on this PR and it is likely to be a separate PR in future.

Until we can get full cgroups v2 support in runc (blocked due to lack of freezer and device subsystems, see opencontainers/runc#654), we would need to use pam_cgfs, although it is unlikely to be available on Red Hat distros: containers/podman#1429

thaJeztah

Not too familiar with all the requirements to make this work, but had a quick glance over, and left some comments/suggestions 🤗

Dockerfile

cmd/dockerd/config_common_unix.go

cmd/dockerd/daemon_unix.go

contrib/dockerd-rootless.sh

pkg/archive/archive_linux.go

AkihiroSuda · 2018-10-19T06:58:52Z

addressed comments

AkihiroSuda · 2019-01-31T13:45:43Z

Updated PR. Now user needs to be install either slirp4netns or vpnkit separately.

We should still include vpnkit in make install bundle, but let's discuss it separately. (And also slirp4netns in some "contrib" binary tgz maybe)

cpuguy83

LGTM

cpuguy83 · 2019-02-01T03:55:24Z

@thaJeztah You good?

thaJeztah

Reviewing from my phone, so just from looking over the changes; left some comments/questions

I'm good to move this forward if those were errors on my side (and this is really cool to see arrive 👌😍🥳)

daemon/config/config_unix.go

cmd/dockerd/daemon.go

daemon/daemon_unix.go

thaJeztah · 2019-02-03T01:14:31Z

@AkihiroSuda are any packaging changes needed for this?

/cc @seemethere

…ode) Please refer to `docs/rootless.md`. TLDR: * Make sure `/etc/subuid` and `/etc/subgid` contain the entry for you * `dockerd-rootless.sh --experimental` * `docker -H unix://$XDG_RUNTIME_DIR/docker.sock run ...` Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>

AkihiroSuda · 2019-02-03T15:26:42Z

Updated PR

cc @estesp @icecrime 😃

@AkihiroSuda are any packaging changes needed for this?

I suggest adding dockerd-rootless.sh, rootlesskit and vpnkit to official RPM/DEB and binary tar archive, but not sure how to build VPNKit for non-amd64.
Also, I suggest providing slirp4netns binary as "contrib" pkg.

thaJeztah · 2019-02-03T18:18:34Z

I suggest adding dockerd-rootless.sh, rootlesskit and vpnkit to official RPM/DEB and binary tar archive, but not sure how to build VPNKit for non-amd64.
Also, I suggest providing slirp4netns binary as "contrib" pkg.

Makes sense; if you have time; could you try opening a pull request in the https://github.com/docker/docker-ce-packaging repository? Perhaps the packaging team can work on it, but if you can prepare a PR, that may help speeding it up 🤗 (feel free to ping me if you need help/input on that one; I'll be on PTO for the next few days, but will try to catch up on notifications)

thaJeztah

LGTM! Thanks; this is really cool stuff 🥳

cyphar · 2019-02-04T03:16:53Z

@AkihiroSuda Great work. 🎉

I suggest adding dockerd-rootless.sh, rootlesskit and vpnkit to official RPM/DEB and binary tar archive, but not sure how to build VPNKit for non-amd64.

Can we use slirp4netns instead of VPNKit (asking for openSUSE when we package this).

thaJeztah · 2019-02-04T05:04:35Z

@cyphar I think the license for slirp4netns was the blocker for bundling it, but you can use it

cyphar · 2019-02-04T05:13:56Z

Right, because we use rootlesskit anyway. Thanks.

tianon · 2019-04-06T00:50:13Z

cmd/dockerd/config_unix.go

@@ -46,5 +49,7 @@ func installConfigFlags(conf *config.Config, flags *pflag.FlagSet) {
 	flags.BoolVar(&conf.NoNewPrivileges, "no-new-privileges", false, "Set no-new-privileges by default for new containers")
 	flags.StringVar(&conf.IpcMode, "default-ipc-mode", config.DefaultIpcMode, `Default mode for containers ipc ("shareable" | "private")`)
 	flags.Var(&conf.NetworkConfig.DefaultAddressPools, "default-address-pool", "Default address pools for node specific local networks")
-
+	// Mostly users don't need to set this flag explicitly.
+	flags.BoolVar(&conf.Rootless, "rootless", rootless.RunningWithNonRootUsername(), "Enable rootless mode (experimental)")


For anyone looking to thread the needle, this line appears to be the cause of #39009. 👍 ❤️

AkihiroSuda requested a review from tianon as a code owner October 16, 2018 08:03

GordonTheTurtle added the status/0-triage label Oct 16, 2018

AkihiroSuda mentioned this pull request Oct 16, 2018

[UMBRELLA] patch submission status rootless-containers/usernetes#42

Closed

3 tasks

vdemeester added status/1-design-review and removed status/0-triage labels Oct 16, 2018

vdemeester requested review from thaJeztah, cpuguy83 and dmcgowan October 16, 2018 08:06

AkihiroSuda force-pushed the rootless branch 2 times, most recently from 483ab2e to e183cfb Compare October 16, 2018 08:12

vdemeester requested review from tiborvass, tonistiigi and crosbymichael October 16, 2018 08:19

AkihiroSuda force-pushed the rootless branch from e183cfb to 21cc7a8 Compare October 16, 2018 08:26

AkihiroSuda changed the title ~~Rootless mode~~ Allow running dockerd as a non-root user (Rootless mode) Oct 16, 2018

This was referenced Oct 16, 2018

Proposal: allow running dockerd as an unprivileged user (aka rootless mode) #37375

Closed

rootless: make /sys/fs/cgroup/* read-only opencontainers/runc#1869

Closed

AkihiroSuda force-pushed the rootless branch 2 times, most recently from abb3322 to 79c8968 Compare October 16, 2018 17:35

thaJeztah requested changes Oct 17, 2018

View reviewed changes

AkihiroSuda force-pushed the rootless branch from 79c8968 to 76a6d66 Compare October 19, 2018 06:58

AkihiroSuda added rebuild/windowsRS5-process labels Oct 19, 2018

GordonTheTurtle removed the rebuild/experimental label Oct 19, 2018

AkihiroSuda added the rebuild/windowsRS1 label Jan 31, 2019

GordonTheTurtle removed the rebuild/windowsRS1 label Jan 31, 2019

cpuguy83 approved these changes Feb 1, 2019

View reviewed changes

thaJeztah requested changes Feb 3, 2019

View reviewed changes

daemon/config/config_unix.go Outdated Show resolved Hide resolved

cmd/dockerd/daemon.go Show resolved Hide resolved

daemon/daemon_unix.go Show resolved Hide resolved

AkihiroSuda force-pushed the rootless branch from 54e9c3f to ec87479 Compare February 3, 2019 15:25

thaJeztah approved these changes Feb 3, 2019

View reviewed changes

thaJeztah added status/4-merge and removed status/2-code-review labels Feb 3, 2019

thaJeztah merged commit 93d994e into moby:master Feb 3, 2019

AkihiroSuda mentioned this pull request Feb 3, 2019

unfork moby & containerd rootless-containers/usernetes#84

Merged

seemethere mentioned this pull request Feb 4, 2019

Build fails on armv7 rootless-containers/rootlesskit#41

Closed

tonistiigi mentioned this pull request Feb 6, 2019

hack: restore bundling vpnkit on amd64 #38684

Merged

asm0dey mentioned this pull request Feb 6, 2019

Episode #23 php-coder/ps-podcast#20

Closed

adamnovak mentioned this pull request Feb 9, 2019

Can't pull images from your Dockerfile into the container's Docker when using Docker in a container, because builds can't do priveleged actions and dockerd #38698

Closed

EML-github mentioned this pull request Feb 13, 2019

Merge #38050 causing a failure? #38725

Closed

thaJeztah added the area/rootless Rootless mode label Apr 4, 2019

tianon mentioned this pull request Apr 5, 2019

19.03.0-beta1: could not get XDG_RUNTIME_DIR (dockerd fails to start) #39009

Closed

tianon reviewed Apr 6, 2019

View reviewed changes

AkihiroSuda mentioned this pull request Jun 4, 2019

KEP: New cgroup driver: "none" (for Rootless) kubernetes/enhancements#1084

Closed

AkihiroSuda mentioned this pull request Mar 17, 2020

RFC: add GPL binaries (fuse-overlayfs and slirp4netns) to docker-rootless-extras.tgz #40706

Closed

thaJeztah mentioned this pull request Apr 25, 2022

minor rootless fix-ups #43521

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow running dockerd as a non-root user (Rootless mode) #38050

Allow running dockerd as a non-root user (Rootless mode) #38050

AkihiroSuda commented Oct 16, 2018 •

edited

Loading

AkihiroSuda commented Oct 16, 2018

codecov bot commented Oct 16, 2018 •

edited

Loading

sargun commented Oct 17, 2018

AkihiroSuda commented Oct 17, 2018 •

edited

Loading

thaJeztah left a comment

AkihiroSuda commented Oct 19, 2018

AkihiroSuda commented Jan 31, 2019

cpuguy83 left a comment

cpuguy83 commented Feb 1, 2019

thaJeztah left a comment

thaJeztah commented Feb 3, 2019

AkihiroSuda commented Feb 3, 2019

thaJeztah commented Feb 3, 2019

thaJeztah left a comment

cyphar commented Feb 4, 2019 •

edited

Loading

thaJeztah commented Feb 4, 2019

cyphar commented Feb 4, 2019

tianon Apr 6, 2019

Allow running dockerd as a non-root user (Rootless mode) #38050

Allow running dockerd as a non-root user (Rootless mode) #38050

Conversation

AkihiroSuda commented Oct 16, 2018 • edited Loading

AkihiroSuda commented Oct 16, 2018

codecov bot commented Oct 16, 2018 • edited Loading

Codecov Report

sargun commented Oct 17, 2018

AkihiroSuda commented Oct 17, 2018 • edited Loading

thaJeztah left a comment

Choose a reason for hiding this comment

AkihiroSuda commented Oct 19, 2018

AkihiroSuda commented Jan 31, 2019

cpuguy83 left a comment

Choose a reason for hiding this comment

cpuguy83 commented Feb 1, 2019

thaJeztah left a comment

Choose a reason for hiding this comment

thaJeztah commented Feb 3, 2019

AkihiroSuda commented Feb 3, 2019

thaJeztah commented Feb 3, 2019

thaJeztah left a comment

Choose a reason for hiding this comment

cyphar commented Feb 4, 2019 • edited Loading

thaJeztah commented Feb 4, 2019

cyphar commented Feb 4, 2019

tianon Apr 6, 2019

Choose a reason for hiding this comment

AkihiroSuda commented Oct 16, 2018 •

edited

Loading

codecov bot commented Oct 16, 2018 •

edited

Loading

AkihiroSuda commented Oct 17, 2018 •

edited

Loading

cyphar commented Feb 4, 2019 •

edited

Loading