Docker inside unprivileged LXC container

[akshaykarle]

Hi,

I'm trying to run a Docker container inside an unprivileged LXC container. I'm able to start the docker daemon using the lxc driver but I have been having trouble with mknod /dev/fuse when trying to run the hello-world container:

root@u1:/# sudo docker run hello-world
INFO[0006] POST /v1.18/containers/create
INFO[0006] +job create()
INFO[0006] +job log(create, a4b9f1286eca35e5f6afc62aad466dfa80061086ccf309171941eb70e88a8335, hello-world:latest)
INFO[0006] -job log(create, a4b9f1286eca35e5f6afc62aad466dfa80061086ccf309171941eb70e88a8335, hello-world:latest) = OK (0)
INFO[0006] -job create() = OK (0)
INFO[0006] POST /v1.18/containers/a4b9f1286eca35e5f6afc62aad466dfa80061086ccf309171941eb70e88a8335/attach?stderr=1&stdout=1&stream=1
INFO[0006] +job container_inspect(a4b9f1286eca35e5f6afc62aad466dfa80061086ccf309171941eb70e88a8335)
INFO[0006] -job container_inspect(a4b9f1286eca35e5f6afc62aad466dfa80061086ccf309171941eb70e88a8335) = OK (0)
INFO[0006] +job attach(a4b9f1286eca35e5f6afc62aad466dfa80061086ccf309171941eb70e88a8335)
INFO[0006] POST /v1.18/containers/a4b9f1286eca35e5f6afc62aad466dfa80061086ccf309171941eb70e88a8335/start
INFO[0006] +job start(a4b9f1286eca35e5f6afc62aad466dfa80061086ccf309171941eb70e88a8335)
INFO[0006] +job allocate_interface(a4b9f1286eca35e5f6afc62aad466dfa80061086ccf309171941eb70e88a8335)
INFO[0006] -job allocate_interface(a4b9f1286eca35e5f6afc62aad466dfa80061086ccf309171941eb70e88a8335) = OK (0)
INFO[0006] +job log(start, a4b9f1286eca35e5f6afc62aad466dfa80061086ccf309171941eb70e88a8335, hello-world:latest)
INFO[0006] -job log(start, a4b9f1286eca35e5f6afc62aad466dfa80061086ccf309171941eb70e88a8335, hello-world:latest) = OK (0)
INFO[0006] -job attach(a4b9f1286eca35e5f6afc62aad466dfa80061086ccf309171941eb70e88a8335) = OK (0)
INFO[0006] +job release_interface(a4b9f1286eca35e5f6afc62aad466dfa80061086ccf309171941eb70e88a8335)
INFO[0006] -job release_interface(a4b9f1286eca35e5f6afc62aad466dfa80061086ccf309171941eb70e88a8335) = OK (0)
INFO[0006] +job release_interface(a4b9f1286eca35e5f6afc62aad466dfa80061086ccf309171941eb70e88a8335)
INFO[0006] -job release_interface(a4b9f1286eca35e5f6afc62aad466dfa80061086ccf309171941eb70e88a8335) = OK (0)
INFO[0006] +job log(die, a4b9f1286eca35e5f6afc62aad466dfa80061086ccf309171941eb70e88a8335, hello-world:latest)
INFO[0006] -job log(die, a4b9f1286eca35e5f6afc62aad466dfa80061086ccf309171941eb70e88a8335, hello-world:latest) = OK (0)
Cannot start container a4b9f1286eca35e5f6afc62aad466dfa80061086ccf309171941eb70e88a8335: mknod /dev/fuse operation not permitted
INFO[0006] -job start(a4b9f1286eca35e5f6afc62aad466dfa80061086ccf309171941eb70e88a8335) = ERR (1)
ERRO[0006] Handler for POST /containers/{name:.*}/start returned error: Cannot start container a4b9f1286eca35e5f6afc62aad466dfa80061086ccf309171941eb70e88a8335: mknod /dev/fuse operation not permitted

docker version:

Client version: 1.6.2
Client API version: 1.18
Go version (client): go1.4.2
Git commit (client): 7c8fca2
OS/Arch (client): linux/amd64
INFO[0110] GET /v1.18/version
INFO[0110] +job version()
INFO[0110] -job version() = OK (0)
Server version: 1.6.2
Server API version: 1.18
Go version (server): go1.4.2
Git commit (server): 7c8fca2
OS/Arch (server): linux/amd64

docker info:

INFO[0128] GET /v1.18/info
INFO[0128] +job info()
INFO[0128] +job subscribers_count()
INFO[0128] -job subscribers_count() = OK (0)
INFO[0128] +job registry_config()
INFO[0128] -job registry_config() = OK (0)
INFO[0128] -job info() = OK (0)
Containers: 2
Images: 2
Storage Driver: vfs
Execution Driver: lxc-1.0.7
Kernel Version: 3.13.0-53-generic
Operating System: Ubuntu 14.04.2 LTS (containerized)
CPUs: 4
Total Memory: 3.86 GiB
Name: u1
ID: HFT7:EGUJ:EKX4:2XHJ:RO7X:7SG2:XGMN:KJ6W:GTCN:RVXP:ID5C:4GVA
WARNING: No memory limit support
WARNING: No swap limit support

uname -a:

Linux u1 3.13.0-53-generic #89-Ubuntu SMP Wed May 20 10:34:39 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

LXC unprivileged container config:

# Template used to create this container: /usr/share/lxc/templates/lxc-download
# Parameters passed to the template: -d ubuntu -r trusty -a amd64
# For additional config options, please look at lxc.container.conf(5)

# Distribution configuration
lxc.include = /usr/share/lxc/config/ubuntu.common.conf
lxc.include = /usr/share/lxc/config/ubuntu.userns.conf
lxc.arch = x86_64

# Container specific configuration
lxc.mount.auto = cgroup
lxc.aa_profile = unconfined
lxc.id_map = u 0 100000 65536
lxc.id_map = g 0 100000 65536
lxc.rootfs = /home/vagrant/.local/share/lxc/u1/rootfs
lxc.utsname = u1

# Network configuration
lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = lxcbr0
lxc.network.hwaddr = 00:16:3e:53:e6:a2

LXC version inside and outside the unprivileged container: 1.0.7

I've managed to get a lxc container running inside a lxc container but fail to do so with docker container. I've also tried to supply the /dev/fuse device using the --device arg which still fails as it still tries to do the mknod. Even tried to switch to the native driver of docker instead of lxc but it fails to start because of cgroups. Any suggestions to what I might be missing or if it is possible to run this?

[unclejack]

It doesn't seem likely something like this will be supported in the near future.

I'm going to close this issue since there are multiple problems with doing this. Please feel free to comment on this issue. There's no need to open another issue on this topic.

[snajpa]

We would love to include support for running Docker CE/Moby on our vpsAdminOS platform for our community hosting.

vpsAdminOS uses LXC to start unprivileged containers, we have full control over LXC, kernel and we use ZFSonLinux as our storage backend. It would be awesome to reach a level of integration, where we could run Docker inside our containers with ZFS backend ultimately.

For now, I can't seem to be able to run Docker containers even with VFS driver.

@unclejack Can we please work together to get Docker working in unprivileged LXC containers?

Thanks!

https://vpsadminos.org
https://github.com/vpsfreecz/vpsadminos