-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add rootless DinD runner #1644
Merged
Merged
Add rootless DinD runner #1644
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,139 @@ | ||
FROM ubuntu:20.04 | ||
|
||
# Target architecture | ||
ARG TARGETPLATFORM=linux/amd64 | ||
|
||
# GitHub runner arguments | ||
ARG RUNNER_VERSION=2.294.0 | ||
|
||
# Docker and Docker Compose arguments | ||
ENV CHANNEL=stable | ||
ARG COMPOSE_VERSION=v2.6.0 | ||
|
||
# Dumb-init version | ||
ARG DUMB_INIT_VERSION=1.2.5 | ||
|
||
# Other arguments | ||
ARG DEBUG=false | ||
|
||
# Set environment variables needed at build | ||
ENV DEBIAN_FRONTEND=noninteractive | ||
|
||
RUN apt update -y \ | ||
&& apt-get install -y software-properties-common \ | ||
&& add-apt-repository -y ppa:git-core/ppa \ | ||
&& apt-get update -y \ | ||
&& apt-get install -y --no-install-recommends \ | ||
build-essential \ | ||
curl \ | ||
ca-certificates \ | ||
dnsutils \ | ||
ftp \ | ||
fuse-overlayfs \ | ||
git \ | ||
iproute2 \ | ||
iputils-ping \ | ||
iptables \ | ||
jq \ | ||
libunwind8 \ | ||
locales \ | ||
netcat \ | ||
net-tools \ | ||
openssh-client \ | ||
parallel \ | ||
python3-pip \ | ||
rsync \ | ||
shellcheck \ | ||
supervisor \ | ||
software-properties-common \ | ||
sudo \ | ||
telnet \ | ||
time \ | ||
tzdata \ | ||
uidmap \ | ||
unzip \ | ||
upx \ | ||
wget \ | ||
zip \ | ||
zstd \ | ||
&& ln -sf /usr/bin/python3 /usr/bin/python \ | ||
&& ln -sf /usr/bin/pip3 /usr/bin/pip \ | ||
&& rm -rf /var/lib/apt/lists/* | ||
|
||
# Runner user | ||
RUN adduser --disabled-password --gecos "" --uid 1000 runner | ||
|
||
RUN test -n "$TARGETPLATFORM" || (echo "TARGETPLATFORM must be set" && false) | ||
|
||
# Setup subuid and subgid so that "--userns-remap=default" works | ||
RUN set -eux; \ | ||
addgroup --system dockremap; \ | ||
adduser --system --ingroup dockremap dockremap; \ | ||
echo 'dockremap:165536:65536' >> /etc/subuid; \ | ||
echo 'dockremap:165536:65536' >> /etc/subgid | ||
|
||
ENV RUNNER_ASSETS_DIR=/runnertmp | ||
|
||
# Runner download supports amd64 as x64 | ||
RUN ARCH=$(echo ${TARGETPLATFORM} | cut -d / -f2) \ | ||
&& export ARCH \ | ||
&& if [ "$ARCH" = "amd64" ]; then export ARCH=x64 ; fi \ | ||
&& mkdir -p "$RUNNER_ASSETS_DIR" \ | ||
&& cd "$RUNNER_ASSETS_DIR" \ | ||
&& curl -L -o runner.tar.gz https://github.com/actions/runner/releases/download/v${RUNNER_VERSION}/actions-runner-linux-${ARCH}-${RUNNER_VERSION}.tar.gz \ | ||
&& tar xzf ./runner.tar.gz \ | ||
&& rm runner.tar.gz \ | ||
&& ./bin/installdependencies.sh \ | ||
&& apt-get install -y libyaml-dev \ | ||
&& rm -rf /var/lib/apt/lists/* | ||
|
||
RUN echo AGENT_TOOLSDIRECTORY=/opt/hostedtoolcache > /runner.env \ | ||
&& mkdir /opt/hostedtoolcache \ | ||
&& chgrp runner /opt/hostedtoolcache \ | ||
&& chmod g+rwx /opt/hostedtoolcache | ||
|
||
# Configure hooks folder structure. | ||
COPY hooks /etc/arc/hooks/ | ||
|
||
# arch command on OS X reports "i386" for Intel CPUs regardless of bitness | ||
RUN export ARCH=$(echo ${TARGETPLATFORM} | cut -d / -f2) \ | ||
&& if [ "$ARCH" = "arm64" ]; then export ARCH=aarch64 ; fi \ | ||
&& if [ "$ARCH" = "amd64" ] || [ "$ARCH" = "i386" ]; then export ARCH=x86_64 ; fi \ | ||
&& curl -f -L -o /usr/local/bin/dumb-init https://github.com/Yelp/dumb-init/releases/download/v${DUMB_INIT_VERSION}/dumb-init_${DUMB_INIT_VERSION}_${ARCH} \ | ||
&& chmod +x /usr/local/bin/dumb-init | ||
|
||
COPY entrypoint.sh logger.bash rootless-startup.sh update-status /usr/bin/ | ||
|
||
RUN chmod +x /usr/bin/rootless-startup.sh /usr/bin/entrypoint.sh | ||
|
||
# Make the rootless runner directory executable | ||
RUN mkdir /run/user/1000 \ | ||
&& chown runner:runner /run/user/1000 \ | ||
&& chmod a+x /run/user/1000 | ||
|
||
# Add the Python "User Script Directory" to the PATH | ||
ENV PATH="${PATH}:${HOME}/.local/bin:/home/runner/bin" | ||
ENV ImageOS=ubuntu20 | ||
ENV DOCKER_HOST=unix:///run/user/1000/docker.sock | ||
ENV XDG_RUNTIME_DIR=/run/user/1000 | ||
|
||
RUN echo "PATH=${PATH}" > /etc/environment \ | ||
&& echo "ImageOS=${ImageOS}" >> /etc/environment \ | ||
&& echo "DOCKER_HOST=${DOCKER_HOST}" >> /etc/environment \ | ||
&& echo "XDG_RUNTIME_DIR=${XDG_RUNTIME_DIR}" >> /etc/environment | ||
|
||
ENV HOME=/home/runner | ||
|
||
# No group definition, as that makes it harder to run docker. | ||
USER runner | ||
|
||
# Docker installation | ||
ENV SKIP_IPTABLES=1 | ||
RUN curl -fsSL https://get.docker.com/rootless | sh | ||
|
||
# Docker-compose installation | ||
RUN curl -L "https://github.com/docker/compose/releases/download/${COMPOSE_VERSION}/docker-compose-Linux-x86_64" -o /home/runner/bin/docker-compose ; \ | ||
chmod +x /home/runner/bin/docker-compose | ||
|
||
ENTRYPOINT ["/usr/local/bin/dumb-init", "--"] | ||
CMD ["rootless-startup.sh"] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
#!/bin/bash | ||
source logger.bash | ||
|
||
log.notice "Writing out Docker config file" | ||
/bin/bash <<SCRIPT | ||
mkdir -p /home/runner/.config/docker/ | ||
|
||
if [ ! -f /home/runner/.config/docker/daemon.json ]; then | ||
echo "{}" > /home/runner/.config/docker/daemon.json | ||
fi | ||
|
||
if [ -n "${MTU}" ]; then | ||
jq ".\"mtu\" = ${MTU}" /home/runner/.config/docker/daemon.json > /tmp/.daemon.json && mv /tmp/.daemon.json /home/runner/.config/docker/daemon.json | ||
# See https://docs.docker.com/engine/security/rootless/ | ||
echo "environment=DOCKERD_ROOTLESS_ROOTLESSKIT_MTU=${MTU}" >> /etc/supervisor/conf.d/dockerd.conf | ||
fi | ||
|
||
if [ -n "${DOCKER_REGISTRY_MIRROR}" ]; then | ||
jq ".\"registry-mirrors\"[0] = \"${DOCKER_REGISTRY_MIRROR}\"" /home/runner/.config/docker/daemon.json > /tmp/.daemon.json && mv /tmp/.daemon.json /home/runner/.config/docker/daemon.json | ||
fi | ||
SCRIPT | ||
|
||
log.notice "Starting Docker (rootless)" | ||
/home/runner/bin/dockerd-rootless.sh --config-file /home/runner/.config/docker/daemon.json >> /dev/null 2>&1 & | ||
|
||
# Wait processes to be running | ||
entrypoint.sh |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I had this PR tested on our self hosted runners infra in GKE. We found fuse-overlayfs not working very well on Ubuntu (super slow, deadlocking, etc.). Everything is working as expected when using default overlay filesystem.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@isarkis, I think it's a bit out of scope for this PR - I superimposed my work on this topic from my demo repository onto this one for the PR. I don't run
fuse-overlayfs
on my own images either, for much the same reason.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Then, let's remove fuse-overlayfs from the runner/actions-runner-dind-rootless.dockerfile. Per https://docs.docker.com/engine/security/rootless/, overlay2, which is the default, is the recommended file system for Ubuntu.