generate multiarch images for non-architecture tags (#9586)

* generate multiarch images for non-architecture tags * lock docker build to legacy docker buider, and bugfix * rename deploy.sh to deploy_by_arch.sh * Update documentation related to multiarch Docker * Consistent IFS value with respect to other scripts Co-authored-by: humanoid2050 <humanoid2050@monolith> Co-authored-by: Brad Warren <bmw@users.noreply.github.com>
certbot · Feb 23, 2023 · a42cffc · a42cffc
1 parent 1cb48ec
commit a42cffc
Show file tree

Hide file tree

Showing 6 changed files with 94 additions and 52 deletions.
diff --git a/.azure-pipelines/templates/stages/deploy-stage.yml b/.azure-pipelines/templates/stages/deploy-stage.yml
@@ -11,7 +11,23 @@ stages:
       - template: ../jobs/snap-deploy-job.yml
         parameters:
           snapReleaseChannel: ${{ parameters.snapReleaseChannel }}
-      - job: publish_docker
+      # The credentials used in the following jobs are for the shared
+      # certbotbot account on Docker Hub. The credentials are stored 
+      # in a service account which was created by following the 
+      # instructions at
+      # https://docs.microsoft.com/en-us/azure/devops/pipelines/library/service-endpoints?view=azure-devops&tabs=yaml#sep-docreg.
+      # The name given to this service account must match the value
+      # given to containerRegistry below. The authentication used when
+      # creating this service account was a personal access token
+      # rather than a password to bypass 2FA. When Brad set this up,
+      # Azure Pipelines failed to verify the credentials with an error
+      # like "access is forbidden with a JWT issued from a personal
+      # access token", but after saving them without verification, the
+      # access token worked when the pipeline actually ran. "Grant
+      # access to all pipelines" should also be checked on the service
+      # account. The access token can be deleted on Docker Hub if
+      # these credentials need to be revoked.
+      - job: publish_docker_by_arch
         pool:
           vmImage: ubuntu-22.04
         strategy:
@@ -33,22 +49,19 @@ stages:
           - task: Docker@2
             inputs:
               command: login
-              # The credentials used here are for the shared certbotbot account
-              # on Docker Hub. The credentials are stored in a service account
-              # which was created by following the instructions at
-              # https://docs.microsoft.com/en-us/azure/devops/pipelines/library/service-endpoints?view=azure-devops&tabs=yaml#sep-docreg.
-              # The name given to this service account must match the value
-              # given to containerRegistry below. The authentication used when
-              # creating this service account was a personal access token
-              # rather than a password to bypass 2FA. When Brad set this up,
-              # Azure Pipelines failed to verify the credentials with an error
-              # like "access is forbidden with a JWT issued from a personal
-              # access token", but after saving them without verification, the
-              # access token worked when the pipeline actually ran. "Grant
-              # access to all pipelines" should also be checked on the service
-              # account. The access token can be deleted on Docker Hub if
-              # these credentials need to be revoked.
               containerRegistry: docker-hub
             displayName: Login to Docker Hub
-          - bash: set -e && tools/docker/deploy.sh $(dockerTag) $DOCKER_ARCH
-            displayName: Deploy the Docker images
+          - bash: set -e && tools/docker/deploy_by_arch.sh $(dockerTag) $DOCKER_ARCH
+            displayName: Deploy the Docker images by architecture
+      - job: publish_docker_multiarch
+        dependsOn: publish_docker_by_arch
+        pool:
+          vmImage: ubuntu-22.04
+        steps:
+          - task: Docker@2
+            inputs:
+              command: login
+              containerRegistry: docker-hub
+            displayName: Login to Docker Hub
+          - bash: set -e && tools/docker/deploy_multiarch.sh $(dockerTag)
+            displayName: Deploy the Docker multiarch manifests
diff --git a/certbot/CHANGELOG.md b/certbot/CHANGELOG.md
@@ -11,7 +11,7 @@ Certbot adheres to [Semantic Versioning](https://semver.org/).
 
 ### Changed
 
-*
+* Docker build and deploy scripts now generate multiarch manifests for non-architecture-specific tags, instead of defaulting to amd64 images.
 
 ### Fixed
 

diff --git a/tools/docker/README.md b/tools/docker/README.md
@@ -20,17 +20,23 @@ DNS plugin Docker images to Docker Hub.
 High-level behavior
 -------------------
 
-Running `./build.sh <TAG> all && ./deploy.sh <TAG> all` causes the Docker
-images to be built and deployed to Docker Hub for all supported architectures
-where `<TAG>` is the base of the tag that should be given to the given images.
-The tag should either be `nightly` or a git version tag like `v0.34.0`. The
-given tag is only the base of the tag because the CPU architecture is also
-added to the tag.
+Running `./build.sh <TAG> all` causes the Docker images to be built for all 
+supported architectures, where `<TAG>` is the base of the tag that should be 
+given to the generated images. The tag should either be `nightly` or a git 
+version tag like `v2.2.0`. The given tag is only the base of the tag because 
+the CPU architecture is also added to the tag. For version tags above `v2.0.0`,
+Additional tags for `latest` are also generated. The generated images are stored 
+in the local docker image cache.
+
+Running `./deploy_by_arch.sh <TAG> all && ./deploy_multiarch.sh <TAG>` will 
+push the previously generated images to Docker Hub and then generate multi-arch
+manifests for easy access to the underlying images appropriate for a given 
+architecture.
 
 Configuration
 -------------
 
 To run these scripts you need:
 
 1. An x86_64 machine with Docker installed and the Docker daemon running. You probably don't want to use the docker snap as these scripts have failed when using that in the past.
-2. To be logged into Docker Hub with an account able to push to the Certbot and Certbot DNS Docker images on Docker Hub.
+2. To be logged into Docker Hub with an account able to push to the Certbot and Certbot DNS Docker images on Docker Hub. Altering the value of `DOCKER_HUB_ORG` in `lib/common` will allow you to push to your own account for testing.
diff --git a/tools/docker/build.sh b/tools/docker/build.sh
@@ -79,7 +79,7 @@ for TARGET_ARCH in "${ALL_REQUESTED_ARCH[@]}"; do
     pushd "${REPO_ROOT}"
     DownloadQemuStatic "${TARGET_ARCH}"
     QEMU_ARCH=$(GetQemuArch "${TARGET_ARCH}")
-    docker build \
+    DOCKER_BUILDKIT=0 docker build \
         --build-arg TARGET_ARCH="${TARGET_ARCH}" \
         --build-arg QEMU_ARCH="${QEMU_ARCH}" \
         -f "${WORK_DIR}/core/Dockerfile" \
@@ -97,7 +97,7 @@ for plugin in "${CERTBOT_PLUGINS[@]}"; do
     for TARGET_ARCH in "${ALL_REQUESTED_ARCH[@]}"; do
         QEMU_ARCH=$(GetQemuArch "${TARGET_ARCH}")
         BASE_IMAGE="${DOCKER_HUB_ORG}/certbot:${TARGET_ARCH}-${TAG_BASE}"
-        docker build \
+        DOCKER_BUILDKIT=0 docker build \
             --build-arg BASE_IMAGE="${BASE_IMAGE}" \
             --build-arg QEMU_ARCH="${QEMU_ARCH}" \
             -f "${WORK_DIR}/plugin/Dockerfile" \

diff --git a/tools/docker/deploy.sh → tools/docker/deploy_by_arch.sh b/tools/docker/deploy.sh → tools/docker/deploy_by_arch.sh
@@ -4,7 +4,7 @@ IFS=$'\n\t'
 
 # This script deploys new versions of Certbot and Certbot plugin docker images.
 
-# Usage: ./deploy.sh [TAG] [all|amd64|arm32v6|arm64v8]
+# Usage: ./deploy_by_arch.sh [TAG] [all|amd64|arm32v6|arm64v8]
 #   with the [TAG] value corresponding the base of the tag to give the Docker
 #   images and the 2nd value being the architecture to build snaps for.
 #   Values should be something like `v0.34.0` or `nightly`. The given value is
@@ -24,19 +24,16 @@ ParseRequestedArch "${2}"
 # Creates and pushes all Docker images aliases for the requested architectures
 # set in the environment variable ALL_REQUESTED_ARCH.  If the value of the
 # global variable TAG_BASE is a 2.0.0 or greater version tag such as v2.1.0,
-# the "latest" tag is also updated. Tags without the architecture part are also
-# created for the default architecture.
-# As an example, for amd64 (the default architecture) and the tag v0.35.0, the
-# following tags would be created:
-#  - certbot/certbot:v0.35.0
-#  - certbot/certbot:latest
-#  - certbot/certbot:amd64-latest
-# For the architecture arm32v6 and the tag v0.35.0, only the following tag
-# would be created:
-#  - certbot/certbot:arm32v6-latest
-# For other tags such as "nightly", aliases are only created for the default
-# architecture where the tag "nightly" would be used without an architecture
-# part.
+# tags for "latest" are also created. Tags such as "nightly" do not recieve 
+# "latest" tags.
+# As an example, for the tag v2.2.0 and the default set of all target 
+# architectures as of writing this, the following tags would be created:
+#  - certbot/certbot:amd64-v2.2.0       <- image
+#  - certbot/certbot:arm32v6-v2.2.0     <- image
+#  - certbot/certbot:arm64v8-v2.2.0     <- image
+#  - certbot/certbot:amd64-latest       <- image
+#  - certbot/certbot:arm32v6-latest     <- image
+#  - certbot/certbot:arm64v8-latest     <- image
 # Usage: TagAndPushForAllRequestedArch [IMAGE NAME]
 #   where [IMAGE NAME] is the name of the Docker image in the Docker repository
 #   such as "certbot" or "dns-cloudflare".
@@ -51,19 +48,9 @@ TagAndPushForAllRequestedArch() {
         # added them, we haven't had another timeout, so until we experience
         # another timeout & can get the deubg logs, we're leaving them in.
         docker --debug push "${DOCKER_REPO}:${TARGET_ARCH}-${TAG_BASE}"
-
-        # If TAG_BASE is a valid tag for version 2.0.0 or greater
         if [[ "${TAG_BASE}" =~ ^v([2-9]|[1-9][0-9]+)\.[0-9]+\.[0-9]+$ ]]; then
             docker tag "${DOCKER_REPO}:${TARGET_ARCH}-${TAG_BASE}" "${DOCKER_REPO}:${TARGET_ARCH}-latest"
             docker --debug push "${DOCKER_REPO}:${TARGET_ARCH}-latest"
-            if [ "${TARGET_ARCH}" == "${DEFAULT_ARCH}" ]; then
-                docker tag "${DOCKER_REPO}:${TARGET_ARCH}-${TAG_BASE}" "${DOCKER_REPO}:latest"
-                docker --debug push "${DOCKER_REPO}:latest"
-            fi
-        fi
-        if [ "${TARGET_ARCH}" == "${DEFAULT_ARCH}" ]; then
-            docker tag "${DOCKER_REPO}:${TARGET_ARCH}-${TAG_BASE}" "${DOCKER_REPO}:${TAG_BASE}"
-            docker --debug push "${DOCKER_REPO}:${TAG_BASE}"
         fi
     done
 }

diff --git a/tools/docker/deploy_multiarch.sh b/tools/docker/deploy_multiarch.sh
@@ -0,0 +1,36 @@
+#!/bin/bash
+set -euxo pipefail
+IFS=$'\n\t'
+
+WORK_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )"
+
+TAG_BASE="$1"  # Eg. v0.35.0 or nightly
+if [ -z "$TAG_BASE" ]; then
+    echo "We cannot tag Docker images with an empty string!" >&2
+    exit 1
+fi
+source "$WORK_DIR/lib/common"
+
+# Creates multiarch manifests for TAG_BASE, and 'latest' if TAG_BASE > 2.0.0
+#  - certbot/certbot:v2.2.0             <- multiarch manifest 
+#  - certbot/certbot:latest             <- multiarch manifest
+MakeMultiarchManifestForAllTargetArch() {
+    DOCKER_REPO="${DOCKER_HUB_ORG}/${1}"
+    SRC_IMAGES=()
+    for TARGET_ARCH in "${ALL_TARGET_ARCH[@]}"; do
+        SRC_IMAGES+=("${DOCKER_REPO}:${TARGET_ARCH}-${TAG_BASE}")
+    done
+    docker buildx imagetools create -t ${DOCKER_REPO}:${TAG_BASE} "${SRC_IMAGES[@]}"
+    if [[ "${TAG_BASE}" =~ ^v([2-9]|[1-9][0-9]+)\.[0-9]+\.[0-9]+$ ]]; then
+        docker buildx imagetools create -t ${DOCKER_REPO}:latest "${SRC_IMAGES[@]}"
+    fi
+}
+
+
+# Step 1: Certbot core Docker
+MakeMultiarchManifestForAllTargetArch "certbot"
+
+# Step 2: Certbot DNS plugins Docker images
+for plugin in "${CERTBOT_PLUGINS[@]}"; do
+    MakeMultiarchManifestForAllTargetArch "${plugin}"
+done