Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use docker inside docker with jenkins user #263

Closed
lvthillo opened this issue May 9, 2016 · 81 comments
Closed

Use docker inside docker with jenkins user #263

lvthillo opened this issue May 9, 2016 · 81 comments

Comments

@lvthillo
Copy link

@lvthillo lvthillo commented May 9, 2016

version:

Client:
 Version:      1.11.1
 API version:  1.23
 Go version:   go1.5.4
 Git commit:   5604cbe
 Built:        Wed Apr 27 00:34:42 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.11.1
 API version:  1.23
 Go version:   go1.5.4
 Git commit:   5604cbe
 Built:        Wed Apr 27 00:34:42 2016
 OS/Arch:      linux/amd64

I have a docker container (jenkins). I've mounted the sockets to my container so that I can perform docker commands inside my jenkins container. This works fine when I am root user in my container:

docker run -d -v /var/run/docker.sock:/var/run/docker.sock -v
/usr/bin/docker:/usr/bin/docker:ro -v
/lib64/libdevmapper.so.1.02:/usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02
-v /lib64/libudev.so.0:/usr/lib/x86_64-linux-gnu/libudev.so.0
-p 8080:8080 --name jenkins -u root --privileged=true -t -i
my-jenkins:1.0

My dockerfile now looks like this: I add my jenkins user to the docker group so I can perform docker commands with my jenkins user:


My dockerfile:
FROM jenkins:1.651.1
COPY plugins.txt /usr/share/jenkins/plugins.txt
RUN /usr/local/bin/plugins.sh /usr/share/jenkins/plugins.txt
 USER root 
 RUN apt-get update 
 RUN groupadd docker && gpasswd -a jenkins docker 
 USER jenkins

When I start this container I'm not able to perform the docker commands with my jenkins user. But jenkins is in the dockergroup:

docker run -d -v /var/run/docker.sock:/var/run/docker.sock -v
/usr/bin/docker:/usr/bin/docker:ro -v
/lib64/libdevmapper.so.1.02:/usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02
-v /lib64/libudev.so.0:/usr/lib/x86_64-linux-gnu/libudev.so.0
-p 8080:8080 --name jenkins -u jenkins --privileged=true -t -i
my-jenkins:1.0

But than it does not work

jenkins@bc145b8cfc1d:/$ docker ps
Cannot connect to the Docker daemon. Is the docker daemon running on this host?
jenkins@bc145b8cfc1d:/$ whoami
jenkins

This is the content of my /etc/groupon my container


jenkins:x:1000:
docker:x:1001:jenkins

my jenkins user is in the docker group

jenkins@bc145b8cfc1d:/$ groups jenkins
jenkins : jenkins docker
@carlossg
Copy link
Contributor

@carlossg carlossg commented May 9, 2016

most likely you are mixing your container users/groups with the host users/groups, that have the same name but not the same uid/gid.
You should ask in docker mailing list
https://github.com/jenkinsci/docker/blob/master/CONTRIBUTING.md

@konradstrack
Copy link

@konradstrack konradstrack commented May 9, 2016

It's quite likely that @carlossg is right about mixing container and host users and groups.

Instead of mounting everything inside the Jenkins container, consider following https://docs.docker.com/engine/installation/linux/debian/ and building an image based on the Jenkins image but with Docker installed, i.e. by doing something like (untested):

RUN apt-get update \
  && apt-get install apt-transport-https ca-certificates \
  && echo "deb https://apt.dockerproject.org/repo debian-jessie main" > /etc/apt/sources.list.d/docker.list \
  && apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D \
  && apt-get update \
  && apt-get install -y docker-engine

Then you only need to mount /var/run/docker.sock to get Docker working inside the container, and that shouldn't cause any permission-related issues.

@lvthillo
Copy link
Author

@lvthillo lvthillo commented May 9, 2016

@konradstrack
Thanks for your answer. I tried this as dockerfile but still I'm not able to use docker commands as jenkins user:

FROM jenkins:1.651.1
#COPY plugins.txt /usr/share/jenkins/plugins.txt
#RUN /usr/local/bin/plugins.sh /usr/share/jenkins/plugins.txt
USER root
RUN apt-get update \
  && apt-get install -y apt-transport-https ca-certificates \
  && echo "deb https://apt.dockerproject.org/repo debian-jessie main" > /etc/apt/sources.list.d/docker.list \
  && apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D \
  && apt-get update -y \
  && apt-get install -y docker-engine
RUN gpasswd -a jenkins docker
USER jenkins
@dweomer
Copy link

@dweomer dweomer commented May 9, 2016

@lorenzvth7, please see #196 (comment). The gist of this is that as root I run

#!/bin/bash -x

# this only works if the docker group does not already exist

DOCKER_SOCKET=/var/run/docker.sock
DOCKER_GROUP=docker

if [ -S ${DOCKER_SOCKET} ]; then
    DOCKER_GID=$(stat -c '%g' ${DOCKER_SOCKET})
    groupadd -for -g ${DOCKER_GID} ${DOCKER_GROUP}
    usermod -aG ${DOCKER_GROUP} ${JENKINS_USER}
fi

... and then drop down to jenkins via gosu

@lvthillo
Copy link
Author

@lvthillo lvthillo commented May 10, 2016

@konradstrack your solution seems to work on Ubuntu but not on CentOS

@lvthillo
Copy link
Author

@lvthillo lvthillo commented May 10, 2016

@dweomer I don't know gosu.
Can you help me a bit further:
This are my dockerfile and jenkins.sh at the moment:

FROM jenkins:1.651.1
MAINTAINER xxx
USER root
RUN apt-get update \
  && apt-get install -y apt-transport-https ca-certificates \
  && echo "deb https://apt.dockerproject.org/repo debian-jessie main" > /etc/apt/sources.list.d/docker.list \
  && apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D \
  && apt-get update -y \
  && apt-get install -y docker-engine \
  && apt-get install sudo \
  && apt-get clean \
  && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
ADD jenkins.sh /usr/local/bin/jenkins.sh
RUN chmod a+x /usr/local/bin/jenkins.sh
ENTRYPOINT ["/usr/local/bin/jenkins.sh"]

jenkins.sh:

#!/bin/bash -x

# this only works if the docker group does not already exist

DOCKER_SOCKET=/var/run/docker.sock
DOCKER_GROUP=docker

if [ -S ${DOCKER_SOCKET} ]; then
    DOCKER_GID=$(stat -c '%g' ${DOCKER_SOCKET})
    groupadd -for -g ${DOCKER_GID} ${DOCKER_GROUP}
    usermod -aG ${DOCKER_GROUP} jenkins
fi
@carlossg
Copy link
Contributor

@carlossg carlossg commented May 10, 2016

You don't need gosu. What you need is for the user 1000 in the host to have permissions to access /var/run/docker.sock
Try chmod 777 /var/run/docker.sock and then reduce permissions as needed

@DonGiulio
Copy link

@DonGiulio DonGiulio commented Jun 17, 2016

I'm having the same problem, and tracked it down to the fact that the jenkins vanilla docker image already comes with a /var/run/docker.sock directory and docker installed, belonging to root:root, it seems that this socket is somehow in use and the sock belongs to root:root with readonly rights, so that's why jenkins can't use it.

I can chmod the socket if I disable tini, but still I can't write anything to it even as root.

$ docker run -i -t jenkins /bin/bash
jenkins@4ffd3500c50b:/$ ls -la /var/run/docker.sock
drwxr-xr-x 2 root root 4096 Jun 17 11:05 docker.sock
jenkins@4ffd3500c50b:/$ docker ps
FATA[0000] Get http:///var/run/docker.sock/v1.18/containers/json: dial unix /var/run/docker.sock: permission denied. Are you trying to connect to a TLS-enabled daemon without TLS?

Currently I'm unable to use this jenkins image to run docker containers, which seems to me something that definitely should work, has anyone managed to use it?

thanks

@carlossg
Copy link
Contributor

@carlossg carlossg commented Jun 17, 2016

you must be using your own jenkins image, the latest one from the hub doesn't have /var/run/docker.sock

docker run -ti --rm --entrypoint ls jenkins -alF /var/run/docker.sock
ls: cannot access /var/run/docker.sock: No such file or directory
@DonGiulio
Copy link

@DonGiulio DonGiulio commented Jun 17, 2016

Thanks for the reply,

That's odd:

if I run your command:

$ docker run -ti --rm --entrypoint ls jenkins -alF /var/run/docker.sock
total 8
drwxr-xr-x 2 root root 4096 Jun 17 12:07 ./
drwxr-xr-x 4 root root 4096 Jun 10 14:01 ../ 

and if I run:

$ docker images --format "table {{.ID}}\t{{.Repository}}\t{{.Tag}}" jenkins
IMAGE ID            REPOSITORY          TAG
f947cbd645d7        jenkins             latest

I dropped all the images on my local machine and ran your command and I get your same result... the file is not there.

what could have happened?

@tn-osimis
Copy link

@tn-osimis tn-osimis commented Jun 17, 2016

@DonGiulio Bad volume mount most-like.

Long version: docker.sock is not supposed to be a directory, it's a UDS (Unix domain socket) which we sometimes like to share between host and containers when we want to access the host Docker engine from containers. We do this with --volume=/run/docker.sock:/run/docker.sock (n.b. /var/run is usually just a symlink to ../run). Now this is the fun part: when the file on the host does not exist, Docker doesn't complain and instead maps an empty volume as a directory inside the container. I remember reading an issue which implied this was by design for various reasons, but naturally this generates very odd behaviors down the line when we're not careful.

So, random guess: maybe you've used -v /var/run/docker.sock:/var/run/docker.sock at some point when Docker wasn't installed on the host and sometime after that image was committed with the same tag (didn't test to see if it was plausible). Or, you've angered the sea gods somehow.

@ndeloof
Copy link
Contributor

@ndeloof ndeloof commented Sep 7, 2016

Closing. Docker in docker is a bad idea for a jenkins CI setup anyway

@ndeloof ndeloof closed this Sep 7, 2016
@laugimethods
Copy link

@laugimethods laugimethods commented Mar 2, 2017

@ndeloof ,

This is not [in fact] “Docker-in-Docker”; the container only runs the CLI and connects back to the host to start sister containers.
pipeline-as-code-github-demo

@laugimethods
Copy link

@laugimethods laugimethods commented Mar 2, 2017

Actually, I'm able to sucessfully run the https://jenkins.io/doc/pipeline/tour/hello-world/ Java Pipeline Example by using this homemade Dockerfile:

FROM jenkins:2.32.3

USER root
RUN apt-get -qq update \
   && apt-get -qq -y install \
   curl

RUN curl -sSL https://get.docker.com/ | sh

RUN usermod -a -G staff jenkins

USER jenkins
> docker build -t jenkins_docker .
> docker run --rm -p 8080:8080 -p 4040:4040 -v /var/run/docker.sock:/var/run/docker.sock jenkins_docker
@brthor
Copy link

@brthor brthor commented Mar 12, 2017

Closing. Docker in docker is a bad idea for a jenkins CI setup anyway

@ndeloof
Why do you think having jenkins in docker is a bad setup? I rather prefer it to managing the installation at the host level.

@laugimethods , this did not work at all for me.

I ended up with the following:

docker-compose.yaml

jenkins:
  build: jenkins
  volumes:
   - jenkins-master-data:/var/jenkins_home
   - /var/run/docker.sock:/var/run/docker.sock
   - /usr/bin/docker:/usr/bin/docker

jenkins dockerfile

FROM jenkins:2.7.1
USER 0
CMD DOCKER_GID=$(stat -c '%g' /var/run/docker.sock) && \
    groupadd -for -g ${DOCKER_GID} docker && \
    usermod -aG docker jenkins && \
    sudo -E -H -u jenkins bash -c /usr/local/bin/jenkins.sh
@ndeloof
Copy link
Contributor

@ndeloof ndeloof commented Mar 12, 2017

@brthor I'm not saying running Jenkins in docker is a bad idea, I'm saying using docker-in-docker (https://github.com/jpetazzo/dind) is bad, with terrible infrastructure impacts. Prefer access to the underlying docker daemon from your jenkins container.

@laugimethods
Copy link

@laugimethods laugimethods commented Mar 15, 2017

Here are ready to use Jenkins & Blueocean Docker Images that can call the underlying docker daemon: https://github.com/Logimethods/docker-jenkins

@brthor Could you explain your environment & what did not work for you with my solution? (so to try to to fix it if possible)

@ndeloof I agree with you that literally running docker inside docker is a bad idea... But it is not what we all expect. So, would you agree to reopen that issue after renaming it like "Using Docker FROM a Jenkins Container"?

@jamshid
Copy link

@jamshid jamshid commented Apr 3, 2017

@ndeloof, please reconsider and reopen this issue. I agree with others here that this feature would be useful and it should just be a simple Dockerfile or jenkins.sh change.

No one is talking about the old "docker in docker" scenario. We just want to use the docker client within jenkins and have this image automatically configure /var/run/docker.sock so that the jenkins user has access.

@ndeloof
Copy link
Contributor

@ndeloof ndeloof commented Apr 4, 2017

I can't consider running a container as root as a "solution"

@laugimethods
Copy link

@laugimethods laugimethods commented Apr 4, 2017

@ndeloof , what do you mean by "as root"?
If you look at https://github.com/Logimethods/docker-jenkins/blob/master/jenkins/Dockerfile , the user at runtime is jenkins, not root.

@ndeloof
Copy link
Contributor

@ndeloof ndeloof commented Apr 4, 2017

yes, as you explicitly had to switch to root user to add some tools, it's clear you have to roll back to jenkins user. If the docker image has USER root as last statement, 90% jenkins container will run as root without end user to customize it.
Making access to docker.sock easier would then just introduce a significant security regression (I'm not even sure this would be accepted for an official docker image on docker store).
So I'm definitively 👎 on this. If you want to contribute doc on setting up access to docker infrastructure, you're welcome

@laugimethods
Copy link

@laugimethods laugimethods commented Apr 4, 2017

@ndeloof So, then, how could we run Declarative Pipelines made of Docker Agents (https://jenkins.io/doc/pipeline/tour/hello-world/)?

Btw, that's probably my last comment.

"Le mieux est l’ennemi du bien." — (Voltaire, La Bégueule, 1772)
https://en.wikipedia.org/wiki/Perfect_is_the_enemy_of_good

@ndeloof
Copy link
Contributor

@ndeloof ndeloof commented Apr 4, 2017

@laugimethods declarative pipeline doesn't use "docker agent", it uses plain classic jenkins agent with a local docker daemon so it can run docker containers (DSL translates into docker-pipeline-plugin).

Also, this isn't just a beauty consideration to prevent root usage, for official docker image we can't just let end-user shoot in their own foot. I guess most production users will anyway prefer to run their own baked jenkins docker image with adequate tools/version under their own control

@aholbreich
Copy link

@aholbreich aholbreich commented May 11, 2017

@brthor see

Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock

in setup like yours. Any ideas?

@ndeloof
Copy link
Contributor

@ndeloof ndeloof commented May 11, 2017

You need the jenkins user to be in the "docker" group so it can access this socket. Start your container with --group-add xx (using adequate group ID)

@aholbreich
Copy link

@aholbreich aholbreich commented May 13, 2017

actually i gave the user permissions like this, but it didn't worked somehow.

 CMD DOCKER_GID=$(stat -c '%g' /var/run/docker.sock) && \
    groupadd -for -g ${DOCKER_GID} docker && \
    usermod -aG docker jenkins && \

what helped is:

sudo chmod 777 /var/run/docker.sock

inside container.

@goforgold
Copy link

@goforgold goforgold commented Sep 11, 2017

I am facing the exactly same issue.

How we are supposed to do that? I tried to run first sample pipeline and it failed. It was really frustrating and disappointed.

Any proper workaround?

@ntwrkguru
Copy link

@ntwrkguru ntwrkguru commented Aug 7, 2018

I'm suspicious that anyone, anywhere, has been able to exploit a normal installation of Jenkins bind-mounted to the docker socket or running as root. Those stories are typically theoretical and rely on myraid factors that just don't exist in the real world. Exposing your docker daemon to the host's tcp stack (and thus exposing the daemon to the outside world) seems like a much riskier proposition, IMO.

@ntwrkguru
Copy link

@ntwrkguru ntwrkguru commented Aug 7, 2018

@sudo-bmitch, that's an interesting approach, but the container itself is still running as root, correct? Wouldn't that provide the same "risks" as running the container and Jenkins process as root within the container? Or am I missing something in the way in which Jenkins uses the socket binding?

@sudo-bmitch
Copy link

@sudo-bmitch sudo-bmitch commented Aug 8, 2018

@ntwrkguru The gosu command is a su+exec to drop privileges to the jenkins user, so the Jenkins process is not running as root and there is no process still running as root in the container after the entrypoint completes. There's the small risk that something breaks the entrypoint which starts as root, and there's the risk that someone will docker exec to run a command as root in the container, but the more critical issue of a vulnerability of Jenkins resulting in a process running as root inside the container is eliminated.

@ntwrkguru
Copy link

@ntwrkguru ntwrkguru commented Aug 8, 2018

@sudo-bmitch Kinda what I was thinking. So, all of this really just helps to protect against some possible Jenkins security flaw. I don't consider docker exec a possible vector because if someone has the ability to run docker commands from the host's shell, you're already done. Thanks.

@bfwg
Copy link

@bfwg bfwg commented Oct 13, 2018

Thanks @laugimethods!

@geerlingguy
Copy link

@geerlingguy geerlingguy commented Nov 9, 2018

@sudo-bmitch's example (https://github.com/sudo-bmitch/jenkins-docker) is the best/cleanest way I've been able to do this—I have tested in a build I'm using in a variety of environments (local Mac, local Windows, Ubuntu in AWS, Kubernetes in AKS, and Kubernetes in local VirtualBox with Debian).

To be clear, to do DinD with Jenkins in Docker, you should:

  1. Mount /var/run/docker.sock into the container from the host.
  2. Have Docker installed inside the container (as part of your Dockerfile build).
  3. Have the Docker container start as root (USER root in the Dockerfile before your entrypoint at least).
  4. Have an entrypoint script which:
    1. Retrieves the docker gid from the host and modifies the container docker gid to match.
    2. Adds the docker group to the jenkins user.
    3. Starts Jenkins as the jenkins user. (Start it as root at your own peril.)

Most of the other solutions I've seen in this thread are just too fragile or need a lot of weird conditions (or are outright dangerous, like setting 777 permissions on the socket file, or running Jenkins as root). Note that you don't need to use gosu; I just use the following command to actually start the Jenkins process at the end of my entrypoint script:

su -s /bin/bash -c "/usr/local/bin/jenkins.sh" jenkins
@ndeloof
Copy link
Contributor

@ndeloof ndeloof commented Nov 10, 2018

Mount /var/run/docker.sock into the container from the host.

at your own peril. Actually way more dangerous than running jenkins as root. Or maybe you use some docker API proxy to limit the risks ?

Have Docker installed inside the container (as part of your Dockerfile build).

Would be nice not to fully install docker, but only get the CLI. Jenkins image is already big ;)

Have an entrypoint script which: Retrieves the docker gid from the host and modifies the container docker gid to match. Adds the docker group to the jenkins user.

Running container with --group-add docker does the same without having to manage this from whithin the container.

@sudo-bmitch
Copy link

@sudo-bmitch sudo-bmitch commented Nov 10, 2018

Would be nice not to fully install docker, but only get the CLI. Jenkins image is already big ;)

This would be fairly trivial to add with the 18.09 packaging, so I pushed an update to do exactly that just now. (https://github.com/sudo-bmitch/jenkins-docker)

Have an entrypoint script which: Retrieves the docker gid from the host and modifies the container docker gid to match. Adds the docker group to the jenkins user.

Running container with --group-add docker does the same without having to manage this from whithin the container.

It does not. It will add a user to a group according to the GID already defined inside the container. It does not correct the GID of that group to match the GID of the host. In fact the group doesn't even need to exist on the host.

Note that you don't need to use gosu; I just use the following command to actually start the Jenkins process at the end of my entrypoint script:

su -s /bin/bash -c "/usr/local/bin/jenkins.sh" jenkins

You do not need gosu, but you will want it for signal handling. Using gosu will eliminate the additional process running:

# su -s /bin/bash jenkins
jenkins@c3b212b41830:/$ ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  0 14:23 pts/0    00:00:00 /bin/sh
root         8     1  0 14:24 pts/0    00:00:00 su -s /bin/bash jenkins
jenkins      9     8  0 14:24 pts/0    00:00:00 bash
jenkins     10     9  0 14:24 pts/0    00:00:00 ps -ef
jenkins@c3b212b41830:/$ exit
exit
# gosu jenkins /bin/bash
jenkins@c3b212b41830:/$ ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  0 14:23 pts/0    00:00:00 /bin/sh
jenkins     20     1  0 14:24 pts/0    00:00:00 /bin/bash
jenkins     25    20  0 14:24 pts/0    00:00:00 ps -ef

Adding exec to that means that the command being run will now show up as pid 1, and receives signals like the request to stop the container gracefully, without the need for intermediate processes to forward those signals. The issue most encounter is that a /bin/sh running as pid 1 does not respond to a SIGTERM resulting in a 10 second delay before the SIGKILL is sent stopping the application without any graceful shutdown.

@sudo-bmitch
Copy link

@sudo-bmitch sudo-bmitch commented Nov 10, 2018

It does not. It will add a user to a group according to the GID already defined inside the container. It does not correct the GID of that group to match the GID of the host. In fact the group doesn't even need to exist on the host.

Rereading some of the above comments, I see the suggestion was to use --group-add ${docker_gid} rather than a direct --group-add docker. The former will work, but requires that you get your local GID, which can differ on each host you run the container on. I've just moved the steps to get that GID and fix permissions from outside of the container to an entrypoint that drops permissions from root to the jenkins user when done. It means someone only needs to run docker run or docker stack deploy without any external scripting to set the variable. It's also a lot easier for users where docker is an embedded VM or external system and the GID where they run docker commands is different from the GID where docker runs containers.

@ayasuda2OO3
Copy link

@ayasuda2OO3 ayasuda2OO3 commented Jan 2, 2019

Most of the other solutions I've seen in this thread are just too fragile or need a lot of weird conditions (or are outright dangerous, like setting 777 permissions on the socket file, or running Jenkins as root). Note that you don't need to use gosu; I just use the following command to actually start the Jenkins process at the end of my entrypoint script:

Agree with @geerlingguy However, the major issue I see all come down to docker designed to run as a root account, which is a fundamental mistaken in the modern distributed systems.

@gdzien
Copy link

@gdzien gdzien commented Feb 23, 2019

you can just use --user "UID:{docker-sock-gid-here}"
Or User: "1000:994" in AWS docker compose. It won't solve underlying permissions mismatch issue, but your user will at least be able to use socket and you will be able to update image with docker pull without further modifications/hacks.

@zoulux
Copy link

@zoulux zoulux commented Feb 27, 2019

You don't need gosu. What you need is for the user 1000 in the host to have permissions to access /var/run/docker.sock
Try chmod 777 /var/run/docker.sock and then reduce permissions as needed

it's ok

@ndeloof
Copy link
Contributor

@ndeloof ndeloof commented Feb 27, 2019

chmod 777 /var/run/docker.sock is the worst thing to do on host from a security point of view. It means you potentially elevate EVERY unix user to root privileges by letting them run arbitrary docker commands.

@gdzien
Copy link

@gdzien gdzien commented Feb 27, 2019

chmod 777 /var/run/docker.sock is the worst thing to do on host from a security point of view. It means you potentially elevate EVERY unix user to root privileges by letting them run arbitrary docker commands.

any form of docker in docker have the same issue basically. But yeah, 777 on docker socket is something people who don't understand basics of Linux would do.

But setting 777 on any file is generally a bad idea.

@phk0
Copy link

@phk0 phk0 commented Mar 15, 2019

Got the same pb. Solved adding docker run parameter --group-add $(stat -c '%g' /var/run/docker.sock), because the host docker gid was different from the container docker gid.

@PeterProbst
Copy link

@PeterProbst PeterProbst commented Mar 28, 2019

Thanks @phk0 👍 perfect solution...

@rogaha
Copy link

@rogaha rogaha commented Apr 20, 2019

nice! thanks for sharing @ndeloof @phk0

@ashokaklt
Copy link

@ashokaklt ashokaklt commented Jun 29, 2019

@phk0 Perfect solution. It saved many days for me.

@tuanalumi
Copy link

@tuanalumi tuanalumi commented Jul 2, 2019

Got the same pb. Solved adding docker run parameter --group-add $(stat -c '%g' /var/run/docker.sock), because the host docker gid was different from the container docker gid.

@phk0 I got this error after adding your run parameter:

webroot: EnvVars.masterEnvVars.get("JENKINS_HOME")
Exception in thread "main" java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at Main._main(Main.java:375)
	at Main.main(Main.java:151)
Caused by: java.lang.IllegalArgumentException: Unrecognized option: --group-add
	at winstone.cmdline.CmdLineParser.parse(CmdLineParser.java:52)
	at winstone.Launcher.getArgsFromCommandLine(Launcher.java:363)
	at winstone.Launcher.main(Launcher.java:334)

Any idea?

@tuanalumi
Copy link

@tuanalumi tuanalumi commented Jul 2, 2019

Oh shoot, I added the param in wrong place. It needs to be before the jenkinsci/blueocean part.

I am feeling stupid now...

@infa-rbliznet
Copy link

@infa-rbliznet infa-rbliznet commented Aug 15, 2019

Tried several posted solutions but only this helped:

jenkins:
      build:
        context: .
        dockerfile: jenkins.Dockerfile
      container_name: jenkins
      ports:
          - '8080:8080'
          - '50000:50000'
      image: cicd/jenkins
      volumes:
      - "jenkins_home:/var/jenkins_home"
      - "/var/run/docker.sock:/var/run/docker.sock"

Dockerfile:

FROM jenkins/jenkins:2.150.1
...
USER root
RUN curl -fsSL https://get.docker.com -o get-docker.sh && sh get-docker.sh
RUN usermod -a -G root jenkins
USER jenkins
@felipecrs
Copy link

@felipecrs felipecrs commented Sep 17, 2019

Got the same pb. Solved adding docker run parameter --group-add $(stat -c '%g' /var/run/docker.sock), because the host docker gid was different from the container docker gid.

Is there any way to make this solution to work on Docker for Windows?

@deskoh
Copy link

@deskoh deskoh commented Nov 4, 2019

Got the same pb. Solved adding docker run parameter --group-add $(stat -c '%g' /var/run/docker.sock), because the host docker gid was different from the container docker gid.

Is there any way to make this solution to work on Docker for Windows?

Run with --group-add 0.

Can confirm the group id of docker.sock using the following command on windows (turns out you can mount volume on MobyLinux VM in Windows):

docker run -it --rm -v /var/run:/var/run busybox stat -c %g /var/run/docker.sock
@felipecrs
Copy link

@felipecrs felipecrs commented Nov 8, 2019

Got the same pb. Solved adding docker run parameter --group-add $(stat -c '%g' /var/run/docker.sock), because the host docker gid was different from the container docker gid.

Is there any way to make this solution to work on Docker for Windows?

Run with --group-add 0.

Can confirm the group id of docker.sock using the following command on windows (turns out you can mount volume on MobyLinux VM in Windows):

docker run -it --rm -v /var/run:/var/run busybox stat -c %g /var/run/docker.sock

Thank you so much. It works :).

My final code is:

DOCKER_SOCK=/var/run/docker.sock
if [[ "$WINDOWS" == "true" ]]; then
  DOCKER_SOCK_GID=$($DOCKER run --rm -v /$DOCKER_SOCK:$DOCKER_SOCK alpine stat -c '%g' /$DOCKER_SOCK)
else
  DOCKER_SOCK_GID=$(stat -c '%g' $DOCKER_SOCK)
fi
@imsathyakumar
Copy link

@imsathyakumar imsathyakumar commented Nov 14, 2019

This solved our problem after going through all the options. Adding jenkins to sudoers

USER root

#install docker
RUN curl -sSL https://get.docker.com/ | sh

#install sudo
RUN apt-get update \
      && apt-get install -y sudo \
      && rm -rf /var/lib/apt/lists/*

#Adding jenkins to sudoers list and making an alias for sudo docker
RUN echo "jenkins ALL=NOPASSWD: ALL" >> /etc/sudoers \
      && printf '#!/bin/bash\nsudo /usr/bin/docker "$@"' > /usr/local/bin/docker \
      && chmod +x /usr/local/bin/docker

USER jenkins
@yadoudou
Copy link

@yadoudou yadoudou commented Feb 9, 2020

You don't need gosu. What you need is for the user 1000 in the host to have permissions to access /var/run/docker.sock
Try chmod 777 /var/run/docker.sock and then reduce permissions as needed

yes , this help me!, chmod a+rw /var/run/docker.sock

@bverkron
Copy link

@bverkron bverkron commented Nov 24, 2020

Is there a way to get this working from a Jenkins pipeline job?

Example:

pipeline{
    agent{
      sh "echo test"
      docker{
        label "linux-docker-proper"
        image "test-image"
        args "--group-add \$(stat -c '%g' /var/run/docker.sock) -v /var/run/docker.sock:/var/run/docker.sock"
      }
    }
    stages{
      stage("test"){
        steps{
          sh "docker ps -a"
        }
      }
    }
}

The job produces this command + error.

$ docker run -t -d -u 153734:153734 --group-add $(stat -c %g /var/run/docker.sock) -v /var/run/docker.sock:/var/run/docker.sock -w "/home/_jenkinssoftserv/remote/workspace/AutomationServices/dockerAgents PoC/Linux Docker Agent via Pipeline Script" -v "/home/_jenkinssoftserv/remote/workspace/AutomationServices/dockerAgents PoC/Linux Docker Agent via Pipeline Script:/home/_jenkinssoftserv/remote/workspace/AutomationServices/dockerAgents PoC/Linux Docker Agent via Pipeline Script:rw,z" -v "/home/_jenkinssoftserv/remote/workspace/AutomationServices/dockerAgents PoC/Linux Docker Agent via Pipeline Script@tmp:/home/_jenkinssoftserv/remote/workspace/AutomationServices/dockerAgents PoC/Linux Docker Agent via Pipeline Script@tmp:rw,z" -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** test-image cat
[Pipeline] // withDockerContainer
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
java.io.IOException: Failed to run image 'test-image'. Error: invalid argument "%g" for "-c, --cpu-shares" flag: strconv.ParseInt: parsing "%g": invalid syntax
See 'docker run --help'.

If I copy / paste the exact docker run command Jenkins shows in the output it works fine. Something to do with the way Jenkins is trying to parse things, but I haven't figured out the solution.

Ask expected if I quote the $() portion it treats it as a literal string (despite this working on the CLI too)
args "--group-add \"\$(stat -c '%g' /var/run/docker.sock)\" -v /var/run/docker.sock:/var/run/docker.sock"
Error: docker: Error response from daemon: Unable to find group $(stat -c '%g' /var/run/docker.sock).

@bverkron
Copy link

@bverkron bverkron commented Nov 24, 2020

Also, what are the repercussions of mapping the /etc/passwd (and in this case /etc/group) file as read-only in the docker run command? I did that initially and it worked perfectly and easily but I could not get a good sense of the risks so went back to manually or dynamically creating them in the container. One drawback though is this won't work for LDAP users as they're not listed in the /etc/passwd file

pipeline{
    agent{
      sh "echo test"
      docker{
        label "linux-docker-proper"
        image "test-image"
        args "-v /var/run/docker.sock:/var/run/docker.sock -v /etc/passwd:/etc/passwd:ro"
      }
    }
    stages{
      stage("test"){
        steps{
          sh "docker ps -a"
        }
      }
    }
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet