New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Get docker to run inside LXD #608
Comments
Hi, Getting Docker to run inside a regular LXD container is on our TODO list. We don't expect any change to be needed on the LXD side of things, but instead, Docker will have to be taught to play nice with LXD's unprivileged (secure) containers. It's on our TODO list and I did some tests a week or so ago. It's however difficult to estimate exactly how long it'll take to resolve most issues. So far those I'm aware of are:
|
We've had some success running Docker in a privileged container with apparmor disabled but doing so bypasses every single security measure we've got in place so isn't something we really want people to do :) |
Haha, well that all sounds great I'm glad its on the roadmap. The potential for application and system container compositions is really exciting! |
will it be easier to support run app container spec based images under LXC/LXD[1]. Then we can just write lxc rootfs -> app container spec image converters, that will allow running rocket and similar (hopefully but less likely docker :-/ ) app containers. |
LXD is currently solely focused on running full system containers, we believe there are great container managers for application containers and rather than becoming yet another one of those, we instead want to ensure that they run inside our full containers. |
@stgraber I've done some investigation on this recently - and it appears we still need to lift all apparmor restrictions and set the container to unconstrained. Is this an accurate assessment of where we stand today on this? |
@chuckbutler that's where we were last I tried, yes. Frankly, there's not much we can do on the LXD side for this. We'll certainly be offering more security knobs to fine tune the apparmor profile, but we'll need some work on Docker to work inside an unprivileged container. |
Any news on this? |
Yes, slowly. I've proposed a few commits to opencontainer/docker. When you
build these with my patches, run a cgroup-namespaces kernel, and bind mount
a file reading '0' over /sys/module/apparmor/paramaters/enabled, docker
with the overlay driver works. (There is a bug in the unprivileged overlay
code which currently requires you to remove /dev/shm in the parent image,
but that will hopefully be fixed soon).
It's coming together.
|
@hallyn How difficult would it be to add detection of existing apparmor confinement to Docker / OpenContainer and have it ignore profile change failures in that case? That should fix things a bit for the pre-apparmor namespace world and should also just work once apparmor namespacing lands. |
I haven't sufficiently looked into it, but it shouldn't be
too hard. Thinking runc will start with a apparmor.AmConfinedAndLocked()
fn which docker will use in several places. I don't know whether
there will be complications/objections about intermediate domains.
|
I tried to run docker in a privileged container with apparmor disabled but it fails on cgroup mount detection. Where can i find the @hallyn patches to docker to give it a try? |
+1 |
Docker support is still work in progress but it will require you use a very recent Ubuntu kernel from the current development release combined with the Docker packaged in Ubuntu. That's because the changes needed to Docker and the kernel haven't made it to an upstream release of either. |
Closing this now as we have the new "docker" profile in LXD which comes with the config bits needed for docker to be happy and Ubuntu 16.04 now has a kernel and LXC that provide a cgroup setup compatible with Docker. That's not to say that Ubuntu's Docker or even the current upstream release works inside LXD with those. @hallyn has a bunch of patches that are required for that. Those have been sent upstream and for the most part merged, so hopefully an upcoming Docker release will work out of the box. For Ubuntu users, the version of Docker in the Ubuntu archive at 16.04 release time will include any needed patch so that it works inside LXD out of the box. |
Is there any way for me to know which docker release has all patches needed? |
@fuzzy76 using docker.io from the Ubuntu repository will always include all the needed patches as Ubuntu gates Docker on working inside LXD (automatic test). For upstream Docker, 1.13 was reported to work unmodified inside LXD containers. However, Docker upstream has since merged a patch to mitigate for a kernel CVE which interferes with running inside an unprivileged container and so effectively regressed LXD support. Ubuntu carries a new patch to revert that behavior when a user namespace is detected, but I don't think this made it upstream. Instead a proper kernel fix for the issue is being discussed by the Docker folks. |
@stgraber do you know if theres an open issue with docker or something so we can follow this? or if theres been any updates in the last 3 weeks? |
any update on this? following this article:
|
Description of problem:
When trying to run Docker in an ubuntu trusty lxc container created with lxd (cli, not openstack) on an ubuntu vivid host (created with vagrant from a cloud-images.ubuntu.com image) I get the following error:
I posted this as a Docker issue first (moby/moby#13054) but didn't get very far. Hoping to find out more here.
From VirtualBox host:
docker version
:docker info
:uname -a
:From
lxc
containerdocker version
:docker info
:uname -a
:Environment details (AWS, VirtualBox, physical, etc.):
lxd
lxc
guest is running Ubuntu TrustyHow reproducible:
Steps to Reproduce:
Actual Results:
Expected Results:
Additional info:
I have tried this without docker installed on the host and with it installed but not running in addition to the above "steps to reproduce."
My thoughts were that any kernel dependencies docker needs would get sucked in on the virtualbox host by installing docker there and would be made available to the lxd guest where I actually want to run docker from.
The text was updated successfully, but these errors were encountered: