Lightweight containers using Linux user namespaces
Linux 3.12 introduced a restriction which prevents new sysfs mounts unless the caller has CAP_SYS_ADMIN over the current network namespace. Consequently, contain -n cannot mount /sys in the usual way. In addition, it is not possible to bind the existing host /sys mount into a container's root filesystem without doing so recursively. This would leak any mounts on /sys/fs/cgroup, /sys/fs/debug, etc. into the container.
|.gitignore||Add inject utility to run a command in an existing container|
|COPYING||Import initial versions of contain and pseudo|
|Makefile||Add inject utility to run a command in an existing container|
|README||Remove unnecessary 'container' variable from environment of helpers|
|TIPS||Fix contain to work with Linux 3.12 and later|
|console.c||Stop console supervisor spinning at EOF on stdin|
|contain.c||Do not mount /sys in containers sharing the host network namespace|
|contain.h||Add -c option to contain which disables console emulation|
|inject.c||Use SHELL from contain.h as the default command|
|map.c||Import initial versions of contain and pseudo|
|mount.c||Do not mount /sys in containers sharing the host network namespace|
|pseudo.c||Use SHELL from contain.h as the default command|
|util.c||Import initial versions of contain and pseudo|
Containers ========== This package is a simple implementation of containers for Linux, making secure containers as easy to create and use as a traditional chroot. It comprises three utilities, contain, inject and pseudo, which use the kernel support for user namespaces merged in Linux 3.8. Demonstration ------------- With the utilities already installed, the demo begins in an unprivileged user's shell: $ echo $$ $UID 21260 1000 To create a simple test container, copy /bin and /lib* from the host into a temporary directory with the default UID/GID mappings applied: $ cd $(mktemp -d) $ tar -c -f - -C / bin lib lib32 lib64 | pseudo tar -x -f - It is very straightforward to launch a container with this newly-created root filesystem: $ contain . /bin/bash # The new shell has PID 1 within the container, and cannot see other processes on the host: # echo $$ $UID 1 0 # ps ax PID TTY STAT TIME COMMAND 1 console Ss 0:00 /bin/bash 2 console R+ 0:00 ps ax The container root user is able to manipulate ownerships and permissions within its filesystem: # ls -l /dev/console crw--w---- 1 0 5 136, 9 Jul 1 14:00 /dev/console # chown 12:34 /dev/console # chmod a+rw /dev/console # ls -l /dev/console crw-rw-rw- 1 12 34 136, 9 Jul 1 14:00 /dev/console and can also make other privileged changes such as setting the hostname: # echo -n "hostname $(hostname) -> " && hostname brian && hostname hostname alice -> brian or configuring the network stack: # ip link show 1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN mode DEFAULT link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 # ping -w 1 184.108.40.206 &>/dev/null && echo up || echo down down # ip addr add 220.127.116.11/32 dev lo && ip link set lo up # ping -w 1 18.104.22.168 &>/dev/null && echo up || echo down up # ip link add type veth && ip link show 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: veth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000 link/ether 3a:0c:96:36:2d:ff brd ff:ff:ff:ff:ff:ff 3: veth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000 link/ether a2:86:1a:92:58:cb brd ff:ff:ff:ff:ff:ff In all cases, these changes affect the container but not the host as a whole. Processes in the container live in different resource namespaces isolated from the host, and the container root user is unable to do anything that would require elevated capabilities or root privilege on the host itself. contain ------- The contain utility is invoked as contain [OPTIONS] DIR [CMD [ARG]...] with options -c disable console emulation in the container -g MAP set the container-to-host GID map -i CMD run a helper child inside the new namespaces -n share the host network unprivileged in the container -o CMD run a helper child outside the new namespaces -u MAP set the container-to-host UID map and creates a new container with DIR recursively bound as its root filesystem, running CMD as PID 1 within that container. If unspecified, CMD defaults to /bin/sh to start a shell, so to fully boot a distribution, specify CMD as /bin/init or /sbin/init. The container init process is isolated in new user, mount, IPC, UTS, and PID namespaces. A synthetic /dev with device nodes bound from the host /dev is automatically mounted within the new mount namespace, together with standard /dev/pts, /proc and /sys filesystems. Because it runs in its own user namespace, users and groups seen inside a container are not the same as the underlying credentials visible for the same processes and files on the host. Sensible default container-to-host UID and GID mappings are provided and described below, but the -u and -g options can be used to override the defaults. The container console is a host pseudo-terminal bound at /dev/console in the new /dev filesystem: stdin and stdout are copied to/from this, and it serves as stdin, stdout and stderr for the container init process. This console emulation can be disabled using the -c option: if -c is used, init is run directly with the stdin, stdout and stderr of the contain command. Containers are usually isolated in their own network namespace, with a distinct set of network interfaces from the host. By specifying the -n option, it is possible to safely share the host network stack instead. If you do this, user networking within the container will work normally, but the container has no privileges with respect to its network namespace so it isn't possible to (re)configure interfaces or routes, and setuid utilities like ping which use a raw socket will fail. Two different kinds of helper program can be used to help set up a container. A program specified with -i is run inside the new namespaces with the new root filesystem as its working directory, just before pivoting into it. Typically this type of helper is used to bind-mount additional parts of the host filesystem inside the container. A helper specified with -o is run outside the namespaces but as a direct child of the supervisor process which is running within them. This type of helper can be used to move host network interfaces (such as a macvtap interface or one half of a veth pair) into the container's network namespace. The environment of the container init process includes "container=contain" so that distributions can identify when they are running under contain. inject ------ The inject utility is invoked as inject PID [CMD [ARG]...] where PID is the process ID of a running container supervisor, and runs a command or shell inside the existing container. The environment, stdin, stdout and stderr of inject are all inherited by the command to be run. The container supervisor PID (i.e. that of contain itself) should be given to inject, not the PID of the descendant init process. The inject utility will only work if process specified has a child with "container=contain" in its environment, which it assumes to be the container init. Linux allows an unprivileged user to join the user namespace of any container started by his UID, so inject need not be installed setuid even if contain and pseudo are setuid root. It will refuse to run if it detects setuid/setgid operation. pseudo ------ The pseudo utility is invoked as pseudo [OPTIONS] [CMD [ARG]...] with options -g MAP set the user namespace GID map -u MAP set the user namespace UID map and runs a command or shell as root in a new user namespace, by analogy with sudo which runs a command as root in the host user namespace. Unlike contain, pseudo does not unshare other namespaces or attempt to isolate the new process from the rest of the host. It has identical default UID/GID mappings, -u and -g options, and support for /etc/subuid and /etc/subgid when installed setuid root, but no other contain options are supported. One use for pseudo is as a more capable replacement for fakeroot, useful for testing, when building software packages or for constructing system images. Unlike the traditional fakeroot approach based on LD_PRELOAD, static binaries and chroot jails are both handled correctly. It is also invaluable for running host software to access the same filesystem as a container, replicating the user and group file ownerships that the container would see. For example, in the demo above, the system image is untarred under pseudo so that files are written into the filesystem with UIDs and GIDs mapped for the container rather than unmapped as on the host. User and group mappings ----------------------- By default, when run as root, contain and pseudo will map container UID/GID 0 onto the highest available host UID/GID (4294967294 unless nested), and all other UIDs/GIDs are mapped onto themselves apart from the top container UID and GID which must be left unmapped. The default mappings avoid host UID and GID 0 as the host root user is still granted a variety of privileges even after dropping all capabilities in the host user namespace. For example, /proc and /sys files typically have (host) root:root ownership, and allowing the container access unfiltered access to things like /proc/sys is dangerous. Run as an unprivileged user, container UID/GID 0 is mapped onto the unprivileged user's UID/GID, then container UIDs/GIDs 1, 2, etc. are successively mapped onto any ranges delegated to that user in /etc/subuid and /etc/subgid. The -u and -g options can be used to specify custom mappings, in the format START:LOWER:COUNT[,START:LOWER:COUNT]... where START is the first UID/GID in a container range, LOWER is the first UID/GID in the corresponding range in the host, and COUNT is the length of these ranges. For example, -u 0:1000:1,1:4000:2000 will map container UID 0 onto host UID 1000 and container UIDs 1...2000 onto host UIDs 4000...5999. It is not possible to map more than one container ID onto a given host ID, nor to list the same container ID twice in a map specification. When invoked by an unprivileged user, all host ranges are checked against /etc/subuid and /etc/subgid. Unmapped users and groups are mapped by the kernel onto the overflow UID and GID set in /proc/sys/kernel/overflowuid and /proc/sys/kernel/overflowgid. By default the kernel sets both these values to 65534. Unprivileged operation, /etc/subuid and /etc/subgid --------------------------------------------------- When a non-root user runs contain or pseudo unprivileged, these tools can only map container UID/GIDs onto the host UID/GID of that user. The resulting container is not very useful as it has just a single user and group available. (Typically only root is mapped in the container.) However, contain and pseudo can also safely be installed setuid root, and in this case, unprivileged users can also map onto ranges of UIDs/GIDs that have been delegated for their use in /etc/subuid and /etc/subgid. The format of these files is similar to /etc/passwd, /etc/group and /etc/shadow. Each line specifies an additional range of UIDs/GIDs allocated to a particular user, and there can be zero, one, or multiple lines for any given user. There are three colon-delimited fields: the user's login name, the first UID/GID in the range, and the number of UIDs/GIDs in the range. For example, an /etc/subuid containing the lines chris:100000:10000 chris:120000:10000 allocates UID ranges 100000-109999 and 120000-129999 to my user 'chris' in addition to my normal login UID. The kernel user namespace author Eric Biederman <email@example.com> has proposed patches against the standard GNU/Linux Shadow package which add support for creating and updating these files in this format; they are likely to become a standard way to delegate sub-users and sub-groups. Building and installing ----------------------- Unpack the source tar.gz file and change to the unpacked directory. Run 'make', then 'make install' as root to install both binaries setuid root in /bin. Alternatively, you can set DESTDIR and/or BINDIR to install in a different location, or strip and copy the compiled binaries into the correct place manually. These utilities were developed on GNU/Linux and are not portable to other platforms as they rely on Linux-specific facilities such as namespaces. Please report any problems or bugs to Chris Webb <firstname.lastname@example.org>. Copying ------- This software was written by Chris Webb <email@example.com> and is distributed as Free Software under the terms of the MIT license in COPYING.