Skip to content

Commit

Permalink
Adding proxy caching for faster and more resilient CI
Browse files Browse the repository at this point in the history
Signed-off-by: apostasie <spam_blackhole@farcloser.world>
  • Loading branch information
apostasie committed May 24, 2024
1 parent 88cb67b commit 58bf1f5
Show file tree
Hide file tree
Showing 3 changed files with 54 additions and 0 deletions.
4 changes: 4 additions & 0 deletions .github/config/daemon.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"registry-mirrors": ["http://localhost:5000"],
"debug": true
}
4 changes: 4 additions & 0 deletions .github/config/hosts.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
server = "https://registry-1.docker.io"

[host."http://localhost:5000"]
capabilities = ["pull", "resolve"]
46 changes: 46 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,52 @@ jobs:
- uses: actions/checkout@v4.1.6
with:
fetch-depth: 1
- name: "Cache for Docker Hub images"
id: registry-cache
uses: actions/cache@v4
with:
path: |
"$GITHUB_WORKSPACE"/registry-proxy-cache
key: ${{ matrix.os }}-registry
- name: "Configure and start proxy cache"
run: |
# Configure docker to use the proxy cache
mkdir -p ~/.config/docker
cp .github/config/daemon.json ~/.config/docker/
sudo cp .github/config/daemon.json /etc/docker/
# Configure containerd to use the proxy cache
sudo mkdir -p /etc/containerd/certs.d/docker.io
sudo cp .github/config/hosts.toml /etc/containerd/certs.d/docker.io/
# Restart services
sudo systemctl restart docker
sudo systemctl restart containerd
# Start the proxy cache (FIXME: workaround github secret protection - should be a secret - though it does not matter, it is a readonly token on a test account)
docker run -d --net bridge --restart always --name registry-proxy-cache \
-e REGISTRY_PROXY_REMOTEURL=https://registry-1.docker.io \
-e REGISTRY_PROXY_USERNAME=dubogus \
-e REGISTRY_PROXY_PASSWORD="$(echo ZGNrcl9wYXRfUk8zdzV6UFdhWDdVWk1yRFh2NDRRYjU4ZVhnCg== | base64 -d)" \
--volume "$GITHUB_WORKSPACE"/registry-proxy-cache:/var/lib/registry -p 5000:5000 registry:2
# Get curl
sudo apt-get update -qq
sudo apt-get install -qq curl
# Wait until it is ready
attempts=0
while ! curl localhost:5000 && [ "$attempts" -lt 10 ] 1>/dev/null 2>&1 ; do
sleep 0.5
attempts=$((attempts + 1))
done
[ "$attempts" -lt 10 ] || {
>&2 printf "Failed contacting proxy server in less than five seconds. Giving up"
exit 1
}
# We are ready
echo "===== TESTING ===="
ls -lA "$GITHUB_WORKSPACE"/registry-proxy-cache
echo "===== Was there some cache already? SHOULD BE ^ ===="
time docker pull alpine || true
docker logs registry-proxy-cache
ls -lA "$GITHUB_WORKSPACE"/registry-proxy-cache
exit 1
- name: "Prepare integration test environment"
run: DOCKER_BUILDKIT=1 docker build -t test-integration --target test-integration --build-arg UBUNTU_VERSION=${UBUNTU_VERSION} --build-arg CONTAINERD_VERSION=${CONTAINERD_VERSION} .
- name: "Remove snap loopback devices (conflicts with our loopback devices in TestRunDevice)"
Expand Down

0 comments on commit 58bf1f5

Please sign in to comment.