New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use docker inside docker with jenkins user #263

Closed
lvthillo opened this Issue May 9, 2016 · 63 comments

Comments

Projects
None yet
@lvthillo
Copy link

lvthillo commented May 9, 2016

version:

Client:
 Version:      1.11.1
 API version:  1.23
 Go version:   go1.5.4
 Git commit:   5604cbe
 Built:        Wed Apr 27 00:34:42 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.11.1
 API version:  1.23
 Go version:   go1.5.4
 Git commit:   5604cbe
 Built:        Wed Apr 27 00:34:42 2016
 OS/Arch:      linux/amd64

I have a docker container (jenkins). I've mounted the sockets to my container so that I can perform docker commands inside my jenkins container. This works fine when I am root user in my container:

docker run -d -v /var/run/docker.sock:/var/run/docker.sock -v
/usr/bin/docker:/usr/bin/docker:ro -v
/lib64/libdevmapper.so.1.02:/usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02
-v /lib64/libudev.so.0:/usr/lib/x86_64-linux-gnu/libudev.so.0
-p 8080:8080 --name jenkins -u root --privileged=true -t -i
my-jenkins:1.0

My dockerfile now looks like this: I add my jenkins user to the docker group so I can perform docker commands with my jenkins user:


My dockerfile:
FROM jenkins:1.651.1
COPY plugins.txt /usr/share/jenkins/plugins.txt
RUN /usr/local/bin/plugins.sh /usr/share/jenkins/plugins.txt
 USER root 
 RUN apt-get update 
 RUN groupadd docker && gpasswd -a jenkins docker 
 USER jenkins

When I start this container I'm not able to perform the docker commands with my jenkins user. But jenkins is in the dockergroup:

docker run -d -v /var/run/docker.sock:/var/run/docker.sock -v
/usr/bin/docker:/usr/bin/docker:ro -v
/lib64/libdevmapper.so.1.02:/usr/lib/x86_64-linux-gnu/libdevmapper.so.1.02
-v /lib64/libudev.so.0:/usr/lib/x86_64-linux-gnu/libudev.so.0
-p 8080:8080 --name jenkins -u jenkins --privileged=true -t -i
my-jenkins:1.0

But than it does not work

jenkins@bc145b8cfc1d:/$ docker ps
Cannot connect to the Docker daemon. Is the docker daemon running on this host?
jenkins@bc145b8cfc1d:/$ whoami
jenkins

This is the content of my /etc/groupon my container


jenkins:x:1000:
docker:x:1001:jenkins

my jenkins user is in the docker group

jenkins@bc145b8cfc1d:/$ groups jenkins
jenkins : jenkins docker
@carlossg

This comment has been minimized.

Copy link

carlossg commented May 9, 2016

most likely you are mixing your container users/groups with the host users/groups, that have the same name but not the same uid/gid.
You should ask in docker mailing list
https://github.com/jenkinsci/docker/blob/master/CONTRIBUTING.md

@konradstrack

This comment has been minimized.

Copy link

konradstrack commented May 9, 2016

It's quite likely that @carlossg is right about mixing container and host users and groups.

Instead of mounting everything inside the Jenkins container, consider following https://docs.docker.com/engine/installation/linux/debian/ and building an image based on the Jenkins image but with Docker installed, i.e. by doing something like (untested):

RUN apt-get update \
  && apt-get install apt-transport-https ca-certificates \
  && echo "deb https://apt.dockerproject.org/repo debian-jessie main" > /etc/apt/sources.list.d/docker.list \
  && apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D \
  && apt-get update \
  && apt-get install -y docker-engine

Then you only need to mount /var/run/docker.sock to get Docker working inside the container, and that shouldn't cause any permission-related issues.

@lvthillo

This comment has been minimized.

Copy link
Author

lvthillo commented May 9, 2016

@konradstrack
Thanks for your answer. I tried this as dockerfile but still I'm not able to use docker commands as jenkins user:

FROM jenkins:1.651.1
#COPY plugins.txt /usr/share/jenkins/plugins.txt
#RUN /usr/local/bin/plugins.sh /usr/share/jenkins/plugins.txt
USER root
RUN apt-get update \
  && apt-get install -y apt-transport-https ca-certificates \
  && echo "deb https://apt.dockerproject.org/repo debian-jessie main" > /etc/apt/sources.list.d/docker.list \
  && apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D \
  && apt-get update -y \
  && apt-get install -y docker-engine
RUN gpasswd -a jenkins docker
USER jenkins
@dweomer

This comment has been minimized.

Copy link

dweomer commented May 9, 2016

@lorenzvth7, please see #196 (comment). The gist of this is that as root I run

#!/bin/bash -x

# this only works if the docker group does not already exist

DOCKER_SOCKET=/var/run/docker.sock
DOCKER_GROUP=docker

if [ -S ${DOCKER_SOCKET} ]; then
    DOCKER_GID=$(stat -c '%g' ${DOCKER_SOCKET})
    groupadd -for -g ${DOCKER_GID} ${DOCKER_GROUP}
    usermod -aG ${DOCKER_GROUP} ${JENKINS_USER}
fi

... and then drop down to jenkins via gosu

@lvthillo

This comment has been minimized.

Copy link
Author

lvthillo commented May 10, 2016

@konradstrack your solution seems to work on Ubuntu but not on CentOS

@lvthillo

This comment has been minimized.

Copy link
Author

lvthillo commented May 10, 2016

@dweomer I don't know gosu.
Can you help me a bit further:
This are my dockerfile and jenkins.sh at the moment:

FROM jenkins:1.651.1
MAINTAINER xxx
USER root
RUN apt-get update \
  && apt-get install -y apt-transport-https ca-certificates \
  && echo "deb https://apt.dockerproject.org/repo debian-jessie main" > /etc/apt/sources.list.d/docker.list \
  && apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D \
  && apt-get update -y \
  && apt-get install -y docker-engine \
  && apt-get install sudo \
  && apt-get clean \
  && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
ADD jenkins.sh /usr/local/bin/jenkins.sh
RUN chmod a+x /usr/local/bin/jenkins.sh
ENTRYPOINT ["/usr/local/bin/jenkins.sh"]

jenkins.sh:

#!/bin/bash -x

# this only works if the docker group does not already exist

DOCKER_SOCKET=/var/run/docker.sock
DOCKER_GROUP=docker

if [ -S ${DOCKER_SOCKET} ]; then
    DOCKER_GID=$(stat -c '%g' ${DOCKER_SOCKET})
    groupadd -for -g ${DOCKER_GID} ${DOCKER_GROUP}
    usermod -aG ${DOCKER_GROUP} jenkins
fi
@carlossg

This comment has been minimized.

Copy link

carlossg commented May 10, 2016

You don't need gosu. What you need is for the user 1000 in the host to have permissions to access /var/run/docker.sock
Try chmod 777 /var/run/docker.sock and then reduce permissions as needed

@DonGiulio

This comment has been minimized.

Copy link

DonGiulio commented Jun 17, 2016

I'm having the same problem, and tracked it down to the fact that the jenkins vanilla docker image already comes with a /var/run/docker.sock directory and docker installed, belonging to root:root, it seems that this socket is somehow in use and the sock belongs to root:root with readonly rights, so that's why jenkins can't use it.

I can chmod the socket if I disable tini, but still I can't write anything to it even as root.

$ docker run -i -t jenkins /bin/bash
jenkins@4ffd3500c50b:/$ ls -la /var/run/docker.sock
drwxr-xr-x 2 root root 4096 Jun 17 11:05 docker.sock
jenkins@4ffd3500c50b:/$ docker ps
FATA[0000] Get http:///var/run/docker.sock/v1.18/containers/json: dial unix /var/run/docker.sock: permission denied. Are you trying to connect to a TLS-enabled daemon without TLS?

Currently I'm unable to use this jenkins image to run docker containers, which seems to me something that definitely should work, has anyone managed to use it?

thanks

@carlossg

This comment has been minimized.

Copy link

carlossg commented Jun 17, 2016

you must be using your own jenkins image, the latest one from the hub doesn't have /var/run/docker.sock

docker run -ti --rm --entrypoint ls jenkins -alF /var/run/docker.sock
ls: cannot access /var/run/docker.sock: No such file or directory
@DonGiulio

This comment has been minimized.

Copy link

DonGiulio commented Jun 17, 2016

Thanks for the reply,

That's odd:

if I run your command:

$ docker run -ti --rm --entrypoint ls jenkins -alF /var/run/docker.sock
total 8
drwxr-xr-x 2 root root 4096 Jun 17 12:07 ./
drwxr-xr-x 4 root root 4096 Jun 10 14:01 ../ 

and if I run:

$ docker images --format "table {{.ID}}\t{{.Repository}}\t{{.Tag}}" jenkins
IMAGE ID            REPOSITORY          TAG
f947cbd645d7        jenkins             latest

I dropped all the images on my local machine and ran your command and I get your same result... the file is not there.

what could have happened?

@tn-osimis

This comment has been minimized.

Copy link

tn-osimis commented Jun 17, 2016

@DonGiulio Bad volume mount most-like.

Long version: docker.sock is not supposed to be a directory, it's a UDS (Unix domain socket) which we sometimes like to share between host and containers when we want to access the host Docker engine from containers. We do this with --volume=/run/docker.sock:/run/docker.sock (n.b. /var/run is usually just a symlink to ../run). Now this is the fun part: when the file on the host does not exist, Docker doesn't complain and instead maps an empty volume as a directory inside the container. I remember reading an issue which implied this was by design for various reasons, but naturally this generates very odd behaviors down the line when we're not careful.

So, random guess: maybe you've used -v /var/run/docker.sock:/var/run/docker.sock at some point when Docker wasn't installed on the host and sometime after that image was committed with the same tag (didn't test to see if it was plausible). Or, you've angered the sea gods somehow.

@ndeloof

This comment has been minimized.

Copy link
Member

ndeloof commented Sep 7, 2016

Closing. Docker in docker is a bad idea for a jenkins CI setup anyway

@ndeloof ndeloof closed this Sep 7, 2016

@laugimethods

This comment has been minimized.

Copy link

laugimethods commented Mar 2, 2017

@ndeloof ,

This is not [in fact] “Docker-in-Docker”; the container only runs the CLI and connects back to the host to start sister containers.
pipeline-as-code-github-demo

@laugimethods

This comment has been minimized.

Copy link

laugimethods commented Mar 2, 2017

Actually, I'm able to sucessfully run the https://jenkins.io/doc/pipeline/tour/hello-world/ Java Pipeline Example by using this homemade Dockerfile:

FROM jenkins:2.32.3

USER root
RUN apt-get -qq update \
   && apt-get -qq -y install \
   curl

RUN curl -sSL https://get.docker.com/ | sh

RUN usermod -a -G staff jenkins

USER jenkins
> docker build -t jenkins_docker .
> docker run --rm -p 8080:8080 -p 4040:4040 -v /var/run/docker.sock:/var/run/docker.sock jenkins_docker
@brthor

This comment has been minimized.

Copy link

brthor commented Mar 12, 2017

Closing. Docker in docker is a bad idea for a jenkins CI setup anyway

@ndeloof
Why do you think having jenkins in docker is a bad setup? I rather prefer it to managing the installation at the host level.

@laugimethods , this did not work at all for me.

I ended up with the following:

docker-compose.yaml

jenkins:
  build: jenkins
  volumes:
   - jenkins-master-data:/var/jenkins_home
   - /var/run/docker.sock:/var/run/docker.sock
   - /usr/bin/docker:/usr/bin/docker

jenkins dockerfile

FROM jenkins:2.7.1
USER 0
CMD DOCKER_GID=$(stat -c '%g' /var/run/docker.sock) && \
    groupadd -for -g ${DOCKER_GID} docker && \
    usermod -aG docker jenkins && \
    sudo -E -H -u jenkins bash -c /usr/local/bin/jenkins.sh
@ndeloof

This comment has been minimized.

Copy link
Member

ndeloof commented Mar 12, 2017

@brthor I'm not saying running Jenkins in docker is a bad idea, I'm saying using docker-in-docker (https://github.com/jpetazzo/dind) is bad, with terrible infrastructure impacts. Prefer access to the underlying docker daemon from your jenkins container.

@laugimethods

This comment has been minimized.

Copy link

laugimethods commented Mar 15, 2017

Here are ready to use Jenkins & Blueocean Docker Images that can call the underlying docker daemon: https://github.com/Logimethods/docker-jenkins

@brthor Could you explain your environment & what did not work for you with my solution? (so to try to to fix it if possible)

@ndeloof I agree with you that literally running docker inside docker is a bad idea... But it is not what we all expect. So, would you agree to reopen that issue after renaming it like "Using Docker FROM a Jenkins Container"?

@jamshid

This comment has been minimized.

Copy link

jamshid commented Apr 3, 2017

@ndeloof, please reconsider and reopen this issue. I agree with others here that this feature would be useful and it should just be a simple Dockerfile or jenkins.sh change.

No one is talking about the old "docker in docker" scenario. We just want to use the docker client within jenkins and have this image automatically configure /var/run/docker.sock so that the jenkins user has access.

@ndeloof

This comment has been minimized.

Copy link
Member

ndeloof commented Apr 4, 2017

I can't consider running a container as root as a "solution"

@laugimethods

This comment has been minimized.

Copy link

laugimethods commented Apr 4, 2017

@ndeloof , what do you mean by "as root"?
If you look at https://github.com/Logimethods/docker-jenkins/blob/master/jenkins/Dockerfile , the user at runtime is jenkins, not root.

@ndeloof

This comment has been minimized.

Copy link
Member

ndeloof commented Apr 4, 2017

yes, as you explicitly had to switch to root user to add some tools, it's clear you have to roll back to jenkins user. If the docker image has USER root as last statement, 90% jenkins container will run as root without end user to customize it.
Making access to docker.sock easier would then just introduce a significant security regression (I'm not even sure this would be accepted for an official docker image on docker store).
So I'm definitively 👎 on this. If you want to contribute doc on setting up access to docker infrastructure, you're welcome

@laugimethods

This comment has been minimized.

Copy link

laugimethods commented Apr 4, 2017

@ndeloof So, then, how could we run Declarative Pipelines made of Docker Agents (https://jenkins.io/doc/pipeline/tour/hello-world/)?

Btw, that's probably my last comment.

"Le mieux est l’ennemi du bien." — (Voltaire, La Bégueule, 1772)
https://en.wikipedia.org/wiki/Perfect_is_the_enemy_of_good

@ndeloof

This comment has been minimized.

Copy link
Member

ndeloof commented Apr 4, 2017

@laugimethods declarative pipeline doesn't use "docker agent", it uses plain classic jenkins agent with a local docker daemon so it can run docker containers (DSL translates into docker-pipeline-plugin).

Also, this isn't just a beauty consideration to prevent root usage, for official docker image we can't just let end-user shoot in their own foot. I guess most production users will anyway prefer to run their own baked jenkins docker image with adequate tools/version under their own control

@aholbreich

This comment has been minimized.

Copy link

aholbreich commented May 11, 2017

@brthor see

Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock

in setup like yours. Any ideas?

@ndeloof

This comment has been minimized.

Copy link
Member

ndeloof commented May 11, 2017

You need the jenkins user to be in the "docker" group so it can access this socket. Start your container with --group-add xx (using adequate group ID)

@aholbreich

This comment has been minimized.

Copy link

aholbreich commented May 13, 2017

actually i gave the user permissions like this, but it didn't worked somehow.

 CMD DOCKER_GID=$(stat -c '%g' /var/run/docker.sock) && \
    groupadd -for -g ${DOCKER_GID} docker && \
    usermod -aG docker jenkins && \

what helped is:

sudo chmod 777 /var/run/docker.sock

inside container.

@goforgold

This comment has been minimized.

Copy link

goforgold commented Sep 11, 2017

I am facing the exactly same issue.

How we are supposed to do that? I tried to run first sample pipeline and it failed. It was really frustrating and disappointed.

Any proper workaround?

@jonefeewang

This comment has been minimized.

Copy link

jonefeewang commented Sep 29, 2017

loop @jamtur01 in this issue.

I run into this problem when building the continuous integration example in Chapter 5 "Testing with Docker" of "The docker book". Running docker commands in Jenkins Container using "/var/run/docker.sock" could cause "permission denied" issue:

Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.29/containers/json: dial unix /var/run/docker.sock: connect: permission denied

hope @jamtur01 checking this issue

@afspear

This comment has been minimized.

Copy link

afspear commented Nov 29, 2017

@aholbreich Running chmod 777 /var/run/docker.sock as root in the running container worked for me as well. Could that happen in the Dockerfile before the container is started?

@ndeloof

This comment has been minimized.

Copy link
Member

ndeloof commented Nov 29, 2017

@afspear you can't change permission of a bind mount (hopefully ! this would be a major security issue)

@zlance

This comment has been minimized.

Copy link

zlance commented Jan 8, 2018

@ndeloof could you point me in the direction for materials on connecting docker binary to an API? I'm running jenkins agents on kubernetes and would like to not expose docker socket if possible as well as not run container as root.

@ndeloof

This comment has been minimized.

Copy link
Member

ndeloof commented Jan 8, 2018

@zlance your docker CLI or API client inside container will have to be configured using DOCKER_HOST env variable to connect to the actual docker daemon endpoint (or a proxy) and pass adequate credentials. Just like you do on any non-containerized host to access your dicker infrastructure.

@AshCoolman

This comment has been minimized.

Copy link

AshCoolman commented Jan 30, 2018

If anyone has a Dockerfile that runs any of the Jenkins Hello world examples out of the box - I'd be super grateful!

(ive tried the above)

UPDATE: This one worked for me :)
https://getintodevops.com/blog/the-simple-way-to-run-docker-in-docker-for-ci

@KeithTt

This comment has been minimized.

Copy link

KeithTt commented Feb 9, 2018

Resolve it by chmod 777 /var/run/docker.sock.

@dweomer

This comment has been minimized.

Copy link

dweomer commented Feb 9, 2018

@KeithTt wrote:

Resolve it by chmod 777 /var/run/docker.sock .

Honestly, it'd be more secure to just run the Jenkins container as root.

@KeithTt

This comment has been minimized.

Copy link

KeithTt commented Feb 9, 2018

@dweomer My jenkins installed locally in tomcat, and running as user tomcat...

@raphaelsoul

This comment has been minimized.

Copy link

raphaelsoul commented Mar 8, 2018

got error not found group docker. When i use --group-add docker options @ndeloof

docker: Error response from daemon: linux spec user: Unable to find group docker.

I also tried to add a jenkins user with 1000 id and add it into group docker in host

@n3v3rf411

This comment has been minimized.

Copy link

n3v3rf411 commented Apr 13, 2018

The user in the container must belong to a group in the container having a group id same with the the docker group in the host system.

e.g. if running on Amazon EC2 AMI, the docker group id when installing docker is 994. In my docker image, added these:

RUN groupadd -g 1000 jenkins \
  && useradd -c "Jenkins user" -d $HOME -u 1000 -g 1000 -m jenkins \
  && groupadd -g 994 dockerami \
  && usermod -aG dockerami jenkins
@pcmanprogrammeur

This comment has been minimized.

Copy link

pcmanprogrammeur commented Apr 14, 2018

Thanks @n3v3rf411 ! It works like a charm !

@ndeloof

This comment has been minimized.

Copy link
Member

ndeloof commented Apr 15, 2018

better than hard-coding the host's ID for docker group in your Dockerfile, you can run your container with --group-add docker

@ptsneves

This comment has been minimized.

Copy link

ptsneves commented May 1, 2018

@ndeloof --group-add docker fails with the error @raphaelsoul showed.
If you pass the gid directly on the docker run, and now you do not need to hardcode it in your image as per your suggestion, it works. eg:
getent group docker|cut -d: -f3

Also, the justification and clarification of the previous post was the answer that perhaps should have been given when you closed the issue. It is better to convince and explain people of the consequences of their choices than just saying "we do not support your crazyness" ;). Regardless thank you

@ukid

This comment has been minimized.

Copy link

ukid commented Jun 16, 2018

@laugimethods Thank you

@sudo-bmitch

This comment has been minimized.

Copy link

sudo-bmitch commented Jun 30, 2018

The solution I've come up with is to start the Jenkins container as root, fix the gid inside the container in the entrypoint, and then run the Jenkins app as the Jenkins user with the corrected gid. That solution has an example here: https://github.com/sudo-bmitch/jenkins-docker

@ntwrkguru

This comment has been minimized.

Copy link

ntwrkguru commented Aug 7, 2018

I'm suspicious that anyone, anywhere, has been able to exploit a normal installation of Jenkins bind-mounted to the docker socket or running as root. Those stories are typically theoretical and rely on myraid factors that just don't exist in the real world. Exposing your docker daemon to the host's tcp stack (and thus exposing the daemon to the outside world) seems like a much riskier proposition, IMO.

@ntwrkguru

This comment has been minimized.

Copy link

ntwrkguru commented Aug 7, 2018

@sudo-bmitch, that's an interesting approach, but the container itself is still running as root, correct? Wouldn't that provide the same "risks" as running the container and Jenkins process as root within the container? Or am I missing something in the way in which Jenkins uses the socket binding?

@sudo-bmitch

This comment has been minimized.

Copy link

sudo-bmitch commented Aug 8, 2018

@ntwrkguru The gosu command is a su+exec to drop privileges to the jenkins user, so the Jenkins process is not running as root and there is no process still running as root in the container after the entrypoint completes. There's the small risk that something breaks the entrypoint which starts as root, and there's the risk that someone will docker exec to run a command as root in the container, but the more critical issue of a vulnerability of Jenkins resulting in a process running as root inside the container is eliminated.

@ntwrkguru

This comment has been minimized.

Copy link

ntwrkguru commented Aug 8, 2018

@sudo-bmitch Kinda what I was thinking. So, all of this really just helps to protect against some possible Jenkins security flaw. I don't consider docker exec a possible vector because if someone has the ability to run docker commands from the host's shell, you're already done. Thanks.

@bfwg

This comment has been minimized.

Copy link

bfwg commented Oct 13, 2018

Thanks @laugimethods!

@geerlingguy

This comment has been minimized.

Copy link

geerlingguy commented Nov 9, 2018

@sudo-bmitch's example (https://github.com/sudo-bmitch/jenkins-docker) is the best/cleanest way I've been able to do this—I have tested in a build I'm using in a variety of environments (local Mac, local Windows, Ubuntu in AWS, Kubernetes in AKS, and Kubernetes in local VirtualBox with Debian).

To be clear, to do DinD with Jenkins in Docker, you should:

  1. Mount /var/run/docker.sock into the container from the host.
  2. Have Docker installed inside the container (as part of your Dockerfile build).
  3. Have the Docker container start as root (USER root in the Dockerfile before your entrypoint at least).
  4. Have an entrypoint script which:
    1. Retrieves the docker gid from the host and modifies the container docker gid to match.
    2. Adds the docker group to the jenkins user.
    3. Starts Jenkins as the jenkins user. (Start it as root at your own peril.)

Most of the other solutions I've seen in this thread are just too fragile or need a lot of weird conditions (or are outright dangerous, like setting 777 permissions on the socket file, or running Jenkins as root). Note that you don't need to use gosu; I just use the following command to actually start the Jenkins process at the end of my entrypoint script:

su -s /bin/bash -c "/usr/local/bin/jenkins.sh" jenkins
@ndeloof

This comment has been minimized.

Copy link
Member

ndeloof commented Nov 10, 2018

Mount /var/run/docker.sock into the container from the host.

at your own peril. Actually way more dangerous than running jenkins as root. Or maybe you use some docker API proxy to limit the risks ?

Have Docker installed inside the container (as part of your Dockerfile build).

Would be nice not to fully install docker, but only get the CLI. Jenkins image is already big ;)

Have an entrypoint script which: Retrieves the docker gid from the host and modifies the container docker gid to match. Adds the docker group to the jenkins user.

Running container with --group-add docker does the same without having to manage this from whithin the container.

@sudo-bmitch

This comment has been minimized.

Copy link

sudo-bmitch commented Nov 10, 2018

Would be nice not to fully install docker, but only get the CLI. Jenkins image is already big ;)

This would be fairly trivial to add with the 18.09 packaging, so I pushed an update to do exactly that just now. (https://github.com/sudo-bmitch/jenkins-docker)

Have an entrypoint script which: Retrieves the docker gid from the host and modifies the container docker gid to match. Adds the docker group to the jenkins user.

Running container with --group-add docker does the same without having to manage this from whithin the container.

It does not. It will add a user to a group according to the GID already defined inside the container. It does not correct the GID of that group to match the GID of the host. In fact the group doesn't even need to exist on the host.

Note that you don't need to use gosu; I just use the following command to actually start the Jenkins process at the end of my entrypoint script:

su -s /bin/bash -c "/usr/local/bin/jenkins.sh" jenkins

You do not need gosu, but you will want it for signal handling. Using gosu will eliminate the additional process running:

# su -s /bin/bash jenkins
jenkins@c3b212b41830:/$ ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  0 14:23 pts/0    00:00:00 /bin/sh
root         8     1  0 14:24 pts/0    00:00:00 su -s /bin/bash jenkins
jenkins      9     8  0 14:24 pts/0    00:00:00 bash
jenkins     10     9  0 14:24 pts/0    00:00:00 ps -ef
jenkins@c3b212b41830:/$ exit
exit
# gosu jenkins /bin/bash
jenkins@c3b212b41830:/$ ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  0 14:23 pts/0    00:00:00 /bin/sh
jenkins     20     1  0 14:24 pts/0    00:00:00 /bin/bash
jenkins     25    20  0 14:24 pts/0    00:00:00 ps -ef

Adding exec to that means that the command being run will now show up as pid 1, and receives signals like the request to stop the container gracefully, without the need for intermediate processes to forward those signals. The issue most encounter is that a /bin/sh running as pid 1 does not respond to a SIGTERM resulting in a 10 second delay before the SIGKILL is sent stopping the application without any graceful shutdown.

@sudo-bmitch

This comment has been minimized.

Copy link

sudo-bmitch commented Nov 10, 2018

It does not. It will add a user to a group according to the GID already defined inside the container. It does not correct the GID of that group to match the GID of the host. In fact the group doesn't even need to exist on the host.

Rereading some of the above comments, I see the suggestion was to use --group-add ${docker_gid} rather than a direct --group-add docker. The former will work, but requires that you get your local GID, which can differ on each host you run the container on. I've just moved the steps to get that GID and fix permissions from outside of the container to an entrypoint that drops permissions from root to the jenkins user when done. It means someone only needs to run docker run or docker stack deploy without any external scripting to set the variable. It's also a lot easier for users where docker is an embedded VM or external system and the GID where they run docker commands is different from the GID where docker runs containers.

@ayasuda2003

This comment has been minimized.

Copy link

ayasuda2003 commented Jan 2, 2019

Most of the other solutions I've seen in this thread are just too fragile or need a lot of weird conditions (or are outright dangerous, like setting 777 permissions on the socket file, or running Jenkins as root). Note that you don't need to use gosu; I just use the following command to actually start the Jenkins process at the end of my entrypoint script:

Agree with @geerlingguy However, the major issue I see all come down to docker designed to run as a root account, which is a fundamental mistaken in the modern distributed systems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment