New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker in privileged mode fails since 1.7.1 #15024

Closed
kanuku opened this Issue Jul 27, 2015 · 10 comments

Comments

Projects
None yet
6 participants
@kanuku

kanuku commented Jul 27, 2015

Description of problem: We encountered a problem while trying to run the docker command inside a privileged container when using docker version 1.7.1.

docker version:
Client version: 1.7.1
Client API version: 1.19
Go version (client): go1.4.2
Git commit (client): 786b29d
OS/Arch (client): linux/amd64
Server version: 1.7.1
Server API version: 1.19
Go version (server): go1.4.2
Git commit (server): 786b29d
OS/Arch (server): linux/amd64

docker info:
Containers: 1
Images: 142
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 144
Dirperm1 Supported: false
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 3.13.0-48-generic
Operating System: Ubuntu 14.04.2 LTS
CPUs: 4
Total Memory: 15.67 GiB
Name: ip-10-144-115-15
ID: WW3D:54FR:GJLM:3ESO:Z6NQ:Q572:IJ2X:43NB:COW4:FEGH:PJUO:DHLJ
WARNING: No swap limit support

uname -a:
Linux ip-10-144-115-15 3.13.0-48-generic #80-Ubuntu SMP Thu Mar 12 11:16:15 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

Environment details (AWS, VirtualBox, physical, etc.):
AWS

How reproducible:

Steps to Reproduce:

  1. sudo docker run --privileged -d -p 5000:50000 -p 80:8080
    -v $(which docker):/usr/bin/docker
    -v /var/run/docker.sock:/var/run/docker.sock
    -v /var/jenkins_home:/var/jenkins_home
    --name jenkins jenkins
  2. docker exec jenkins docker version

Actual Results:
docker: error while loading shared libraries: libapparmor.so.1: cannot open shared object file: No such file or directory

Expected Results:
Client version: 1.7.1
Client API version: 1.19
Go version (client): go1.4.2
Git commit (client): 786b29d
OS/Arch (client): linux/amd64
Server version: 1.7.1
Server API version: 1.19
Go version (server): go1.4.2
Git commit (server): 786b29d
OS/Arch (server): linux/amd64

Additional info:
This was working on the previous versions.

Workaround:
By mounting the missing shared library on the container, it will work as expected(without any error). Use the following command:
sudo docker run --privileged -d -p 5000:50000 -p 80:8080
-v $(which docker):/usr/bin/docker
-v /var/run/docker.sock:/var/run/docker.sock
-v /var/jenkins_home:/var/jenkins_home
-v /usr/lib/x86_64-linux-gnu/libapparmor.so.1:/usr/lib/x86_64-linux-gnu/libapparmor.so.1:ro
--name jenkins jenkins

@CesarNog

This comment has been minimized.

Show comment
Hide comment
@CesarNog

CesarNog Jul 27, 2015

Anyone has a solution for it?

I am having the same problem for a samba container (https://github.com/SvenDowideit/dockerfiles/tree/master/samba)

Tried to run:
docker run --rm -v $(which docker):/docker -v /var/run/docker.sock:/docker.sock svendowideit/samba app-data

/docker: error while loading shared libraries: libapparmor.so.1: cannot open shared object file: No such file or directory

CesarNog commented Jul 27, 2015

Anyone has a solution for it?

I am having the same problem for a samba container (https://github.com/SvenDowideit/dockerfiles/tree/master/samba)

Tried to run:
docker run --rm -v $(which docker):/docker -v /var/run/docker.sock:/docker.sock svendowideit/samba app-data

/docker: error while loading shared libraries: libapparmor.so.1: cannot open shared object file: No such file or directory

@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 Jul 28, 2015

Contributor

ping @ewindisch

Contributor

cpuguy83 commented Jul 28, 2015

ping @ewindisch

@ewindisch

This comment has been minimized.

Show comment
Hide comment
@ewindisch

ewindisch Jul 28, 2015

Contributor

Increasingly, users will find that Docker is now dynamically linked. This means that one cannot simply do '-v $(which docker):/usr/bin/docker' and expect it to work without having dependencies inside of the container. I recommend, instead, installing Docker properly inside of the container with its dependencies, or choose a statically linked binary.

Contributor

ewindisch commented Jul 28, 2015

Increasingly, users will find that Docker is now dynamically linked. This means that one cannot simply do '-v $(which docker):/usr/bin/docker' and expect it to work without having dependencies inside of the container. I recommend, instead, installing Docker properly inside of the container with its dependencies, or choose a statically linked binary.

@CesarNog

This comment has been minimized.

Show comment
Hide comment
@CesarNog

CesarNog Jul 28, 2015

@ewindisch, do you suggest to use your Samba container (found at: https://github.com/ewindisch/docker-samba), instead of using @SvenDowideit container? (located at: https://github.com/SvenDowideit/dockerfiles/tree/master/samba)

?

CesarNog commented Jul 28, 2015

@ewindisch, do you suggest to use your Samba container (found at: https://github.com/ewindisch/docker-samba), instead of using @SvenDowideit container? (located at: https://github.com/SvenDowideit/dockerfiles/tree/master/samba)

?

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Jul 28, 2015

Member

@CesarNog that won't make a difference, the issue at hand here, is that previously, the docker binary was build statically (i.e. the binary contained all the libraries it needed). The advantage of that was that you could just "drop" it in a container and it worked. The downside was, that some of the libraries were outdated, or didn't contain fixes for the specific platform docker was running on.

The new binaries are dynamically linked; they no longer contain all the libraries, so these libraries have to be available on the computer (or the container) the docker binary is running on. Those libraries have to be present / installed inside the container, otherwise docker won't be able to run inside that.

(Sorry for anything that isn't "factually" right, tried to keep the description simple)

Member

thaJeztah commented Jul 28, 2015

@CesarNog that won't make a difference, the issue at hand here, is that previously, the docker binary was build statically (i.e. the binary contained all the libraries it needed). The advantage of that was that you could just "drop" it in a container and it worked. The downside was, that some of the libraries were outdated, or didn't contain fixes for the specific platform docker was running on.

The new binaries are dynamically linked; they no longer contain all the libraries, so these libraries have to be available on the computer (or the container) the docker binary is running on. Those libraries have to be present / installed inside the container, otherwise docker won't be able to run inside that.

(Sorry for anything that isn't "factually" right, tried to keep the description simple)

@CesarNog

This comment has been minimized.

Show comment
Hide comment
@CesarNog

CesarNog Jul 28, 2015

So, for the global community that is probably intensively using both samba containers examples from @ewindisch and @SvenDowideit we should take an action not?

CesarNog commented Jul 28, 2015

So, for the global community that is probably intensively using both samba containers examples from @ewindisch and @SvenDowideit we should take an action not?

@dangerp

This comment has been minimized.

Show comment
Hide comment
@dangerp

dangerp Oct 15, 2015

Based on the conversation here: rayrutjes/simple-gitlab-runner#1 (comment), it looks like a straightforward workaround is to install lxc within the container (which contains apparmor), while still mounting in the docker binary from the host. In our environment, this resolved the libapparmor error. Are there any known issues with this approach?

dangerp commented Oct 15, 2015

Based on the conversation here: rayrutjes/simple-gitlab-runner#1 (comment), it looks like a straightforward workaround is to install lxc within the container (which contains apparmor), while still mounting in the docker binary from the host. In our environment, this resolved the libapparmor error. Are there any known issues with this approach?

@ewindisch

This comment has been minimized.

Show comment
Hide comment
@ewindisch

ewindisch Oct 15, 2015

Contributor

It's still possible to build a static binary afaik.

@CesarNog I don't see what this has to do with samba containers? Privileged mode still works and those images should also still work. My own image works fine without privileged mode, anyway.

What's changed is that using a docker binary provided via a volume does not "just" work anymore. Passing binaries via volumes was never a great solution anyway, it's better to install the binary inside of the image, ideally using the package manager.

Contributor

ewindisch commented Oct 15, 2015

It's still possible to build a static binary afaik.

@CesarNog I don't see what this has to do with samba containers? Privileged mode still works and those images should also still work. My own image works fine without privileged mode, anyway.

What's changed is that using a docker binary provided via a volume does not "just" work anymore. Passing binaries via volumes was never a great solution anyway, it's better to install the binary inside of the image, ideally using the package manager.

@ewindisch

This comment has been minimized.

Show comment
Hide comment
@ewindisch

ewindisch Oct 15, 2015

Contributor

@thaJeztah I suggest to close this or decide to take action on documentation.

Contributor

ewindisch commented Oct 15, 2015

@thaJeztah I suggest to close this or decide to take action on documentation.

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Oct 15, 2015

Member

Thanks for the ping @ewindisch. I don't think we have documentation on running docker in this way, so not sure we should be adding something to the docs (we can't possibly describe all use-cases)

However @SvenDowideit may have to update his readme (hm, @jpetazzo seems to use the same approach in his latest article http://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/)

Member

thaJeztah commented Oct 15, 2015

Thanks for the ping @ewindisch. I don't think we have documentation on running docker in this way, so not sure we should be adding something to the docs (we can't possibly describe all use-cases)

However @SvenDowideit may have to update his readme (hm, @jpetazzo seems to use the same approach in his latest article http://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment