Migrate dockerfiles to use multi-stage builds.

ilya-biryukov · ilya-biryukov · commit d95020108c48 · 2018-03-26T15:12:30.000Z
Summary: We previously emulated multi-staged builds using two dockerfiles, native support from Docker allows us to merge them into one, simplifying our scripts. For more details about multi-stage builds, see: https://docs.docker.com/develop/develop-images/multistage-build/ Reviewers: mehdi_amini, klimek, sammccall Reviewed By: sammccall Subscribers: llvm-commits, ioeric, cfe-commits Differential Revision: https://reviews.llvm.org/D44787 llvm-svn: 328503
diff --git a/llvm/docs/Docker.rst b/llvm/docs/Docker.rst
@@ -53,24 +53,15 @@ serve as a basis for anyone who wants to create their own Docker image with
 LLVM components, compiled from sources. The sources are checked out from the
 upstream svn repository when building the image.
 
-Inside each subfolder we host Dockerfiles for two images:
-
-- ``build/`` image is used to compile LLVM, it installs a system compiler and all
-  build dependencies of LLVM. After the build process is finished, the build
-  image will have an archive with compiled components at ``/tmp/clang.tar.gz``.
-- ``release/`` image usually only contains LLVM components, compiled by the
-  ``build/`` image, and also libstdc++ and binutils to make image minimally
-  useful for C++ development. The assumption is that you usually want clang to
-  be one of the provided components.
-
-To build both of those images, use ``build_docker_image.sh`` script.
-It will checkout LLVM sources and build clang in the ``build`` container, copy results
-of the build to the local filesystem and then build the ``release`` container using
-those. The ``build_docker_image.sh`` accepts a list of LLVM repositories to
-checkout, and arguments for CMake invocation.
+The resulting image contains only the requested LLVM components and a few extra
+packages to make the image minimally useful for C++ development, e.g. libstdc++
+and binutils.
+
+The interface to run the build is ``build_docker_image.sh`` script. It accepts a
+list of LLVM repositories to checkout and arguments for CMake invocation.
 
 If you want to write your own docker image, start with an ``example/`` subfolder.
-It provides incomplete Dockerfiles with (very few) FIXMEs explaining the steps
+It provides an incomplete Dockerfile with (very few) FIXMEs explaining the steps
 you need to take in order to make your Dockerfiles functional.
 
 Usage
@@ -110,10 +101,10 @@ this command will do that:
 	-DBOOTSTRAP_CMAKE_BUILD_TYPE=Release \
 	-DCLANG_ENABLE_BOOTSTRAP=ON -DCLANG_BOOTSTRAP_TARGETS="install-clang;install-clang-headers"
 	
-This will produce two images, a release image ``clang-debian8:staging`` and a
-build image ``clang-debian8-build:staging`` from the latest upstream revision.
-After the image is built you can run bash inside a container based on your
-image like this:
+This will produce a new image ``clang-debian8:staging`` from the latest
+upstream revision.
+After the image is built you can run bash inside a container based on your image
+like this:
 
 .. code-block:: bash
 
@@ -181,19 +172,14 @@ debian8-based image using the latest ``google/stable`` sources for you:
 
 Minimizing docker image size
 ============================
-Due to Docker restrictions we use two images (i.e., build and release folders)
-for the release image to be as small as possible. It's much easier to achieve
-that using two images, because Docker would store a filesystem layer for each
-command in the  Dockerfile, i.e. if you install some packages in one command,
-then remove  those in a separate command, the size of the resulting image will
-still be proportinal to the size of an image with installed packages.
-Therefore, we strive to provide a very simple release image which only copies
-compiled clang and does not do anything else.
-
-Docker 1.13 added a ``--squash`` flag that allows to flatten the layers of the
-image, i.e. remove the parts that were actually deleted. That is an easier way
-to produce the smallest images possible by using just a single image. We do not
-use it because as of today the flag is in experimental stage and not everyone
-may have the latest docker version available. When the flag is out of
-experimental stage, we should investigate replacing two images approach with
-just a single image, built using ``--squash`` flag.
+Due to how Docker's filesystem works, all intermediate writes are persisted in
+the resulting image, even if they are removed in the following commands.
+To minimize the resulting image size we use `multi-stage Docker builds
+<https://docs.docker.com/develop/develop-images/multistage-build/>`_.
+Internally Docker builds two images. The first image does all the work: installs
+build dependencies, checks out LLVM source code, compiles LLVM, etc.
+The first image is only used during build and does not have a descriptive name,
+i.e. it is only accessible via the hash value after the build is finished.
+The second image is our resulting image. It contains only the built binaries
+and not any build dependencies. It is also accessible via a descriptive name
+(specified by -d and -t flags).
diff --git a/llvm/utils/docker/build_docker_image.sh b/llvm/utils/docker/build_docker_image.sh
@@ -163,19 +163,9 @@ if [ "$DOCKER_TAG" != "" ]; then
   DOCKER_TAG=":$DOCKER_TAG"
 fi
 
-echo "Building from $IMAGE_SOURCE"
-echo "Building $DOCKER_REPOSITORY-build$DOCKER_TAG"
-docker build -t "$DOCKER_REPOSITORY-build$DOCKER_TAG" \
+echo "Building ${DOCKER_REPOSITORY}${DOCKER_TAG} from $IMAGE_SOURCE"
+docker build -t "${DOCKER_REPOSITORY}${DOCKER_TAG}" \
   --build-arg "buildscript_args=$BUILDSCRIPT_ARGS" \
-  -f "$BUILD_DIR/$IMAGE_SOURCE/build/Dockerfile" \
+  -f "$BUILD_DIR/$IMAGE_SOURCE/Dockerfile" \
   "$BUILD_DIR"
-
-echo "Copying clang installation to release image sources"
-docker run -v "$BUILD_DIR/$IMAGE_SOURCE:/workspace" "$DOCKER_REPOSITORY-build$DOCKER_TAG" \
-  cp /tmp/clang.tar.gz /workspace/release
-
-echo "Building release image"
-docker build -t "${DOCKER_REPOSITORY}${DOCKER_TAG}" \
-  "$BUILD_DIR/$IMAGE_SOURCE/release"
-
 echo "Done"
diff --git a/llvm/utils/docker/debian8/Dockerfile b/llvm/utils/docker/debian8/Dockerfile
@@ -6,33 +6,27 @@
 # License. See LICENSE.TXT for details.
 #
 #===----------------------------------------------------------------------===//
-# Produces an image that compiles and archives clang, based on debian8.
-FROM launcher.gcr.io/google/debian8:latest
-
+# Stage 1. Check out LLVM source code and run the build.
+FROM launcher.gcr.io/google/debian8:latest as builder
 LABEL maintainer "LLVM Developers"
-
 # Install build dependencies of llvm.
 # First, Update the apt's source list and include the sources of the packages.
 RUN grep deb /etc/apt/sources.list | \
     sed 's/^deb/deb-src /g' >> /etc/apt/sources.list
-
 # Install compiler, python and subversion.
 RUN apt-get update && \
     apt-get install -y --no-install-recommends ca-certificates gnupg \
            build-essential python wget subversion unzip && \
     rm -rf /var/lib/apt/lists/*
-
 # Install a newer ninja release. It seems the older version in the debian repos
 # randomly crashes when compiling llvm.
 RUN wget "https://github.com/ninja-build/ninja/releases/download/v1.8.2/ninja-linux.zip" && \
     echo "d2fea9ff33b3ef353161ed906f260d565ca55b8ca0568fa07b1d2cab90a84a07 ninja-linux.zip" \
         | sha256sum -c  && \
     unzip ninja-linux.zip -d /usr/local/bin && \
     rm ninja-linux.zip
-
 # Import public key required for verifying signature of cmake download.
 RUN gpg --keyserver hkp://pgp.mit.edu --recv 0x2D2CEF1034921684
-
 # Download, verify and install cmake version that can compile clang into /usr/local.
 # (Version in debian8 repos is is too old)
 RUN mkdir /tmp/cmake-install && cd /tmp/cmake-install && \
@@ -47,9 +41,18 @@ RUN mkdir /tmp/cmake-install && cd /tmp/cmake-install && \
 
 ADD checksums /tmp/checksums
 ADD scripts /tmp/scripts
-
 # Arguments passed to build_install_clang.sh.
 ARG buildscript_args
-
-# Run the build. Results of the build will be available as /tmp/clang.tar.gz.
+# Run the build. Results of the build will be available at /tmp/clang-install/.
 RUN /tmp/scripts/build_install_llvm.sh ${buildscript_args}
+
+
+# Stage 2. Produce a minimal release image with build results.
+FROM launcher.gcr.io/google/debian8:latest
+LABEL maintainer "LLVM Developers"
+# Install packages for minimal useful image.
+RUN apt-get update && \
+    apt-get install -y --no-install-recommends libstdc++-4.9-dev binutils && \
+    rm -rf /var/lib/apt/lists/*
+# Copy build results of stage 1 to /usr/local.
+COPY --from=builder /tmp/clang-install/ /usr/local/
diff --git a/llvm/utils/docker/debian8/release/Dockerfile b/llvm/utils/docker/debian8/release/Dockerfile
diff --git a/llvm/utils/docker/example/Dockerfile b/llvm/utils/docker/example/Dockerfile
@@ -9,20 +9,29 @@
 # This is an example Dockerfile to build an image that compiles clang.
 # Replace FIXMEs to prepare your own image.
 
+# Stage 1. Check out LLVM source code and run the build.
 # FIXME: Replace 'ubuntu' with your base image
-FROM ubuntu
-
+FROM ubuntu as builder
 # FIXME: Change maintainer name
 LABEL maintainer "Maintainer <maintainer@email>"
-
-# FIXME: Install llvm/clang build dependencies. Including compiler to
+# FIXME: Install llvm/clang build dependencies here. Including compiler to
 # build stage1, cmake, subversion, ninja, etc.
 
 ADD checksums /tmp/checksums
 ADD scripts /tmp/scripts
-
 # Arguments passed to build_install_clang.sh.
 ARG buildscript_args
-
-# Run the build. Results of the build will be available as /tmp/clang.tar.gz.
+# Run the build. Results of the build will be available as /tmp/clang-install.
 RUN /tmp/scripts/build_install_llvm.sh ${buildscript_args}
+
+
+# Stage 2. Produce a minimal release image with build results.
+# FIXME: Replace 'ubuntu' with your base image.
+FROM ubuntu
+# FIXME: Change maintainer name.
+LABEL maintainer "Maintainer <maintainer@email>"
+# FIXME: Install all packages you want to have in your release container.
+# A minimal useful installation should include at least libstdc++ and binutils.
+
+# Copy build results of stage 1 to /usr/local.
+COPY --from=builder /tmp/clang-install/ /usr/local/
diff --git a/llvm/utils/docker/example/release/Dockerfile b/llvm/utils/docker/example/release/Dockerfile
diff --git a/llvm/utils/docker/nvidia-cuda/Dockerfile b/llvm/utils/docker/nvidia-cuda/Dockerfile
@@ -6,26 +6,26 @@
 # License. See LICENSE.TXT for details.
 #
 #===----------------------------------------------------------------------===//
-# Produces an image that compiles and archives clang, based on nvidia/cuda
-# image.
-FROM nvidia/cuda:8.0-devel
-
+# Stage 1. Check out LLVM source code and run the build.
+FROM nvidia/cuda:8.0-devel as builder
 LABEL maintainer "LLVM Developers"
-
-# Arguments to pass to build_install_clang.sh.
-ARG buildscript_args
-
 # Install llvm build dependencies.
 RUN apt-get update && \
     apt-get install -y --no-install-recommends ca-certificates cmake python \
-		    subversion ninja-build && \
+        subversion ninja-build && \
     rm -rf /var/lib/apt/lists/*
 
 ADD checksums /tmp/checksums
 ADD scripts /tmp/scripts
-
 # Arguments passed to build_install_clang.sh.
 ARG buildscript_args
-
-# Run the build. Results of the build will be available as /tmp/clang.tar.gz.
+# Run the build. Results of the build will be available at /tmp/clang-install/.
 RUN /tmp/scripts/build_install_llvm.sh ${buildscript_args}
+
+
+# Stage 2. Produce a minimal release image with build results.
+FROM nvidia/cuda:8.0-devel
+LABEL maintainer "LLVM Developers"
+# Copy clang installation into this container.
+COPY --from=builder /tmp/clang-install/ /usr/local/
+# C++ standard library and binutils are already included in the base package.
diff --git a/llvm/utils/docker/nvidia-cuda/release/Dockerfile b/llvm/utils/docker/nvidia-cuda/release/Dockerfile
diff --git a/llvm/utils/docker/scripts/build_install_llvm.sh b/llvm/utils/docker/scripts/build_install_llvm.sh
@@ -16,8 +16,8 @@ Usage: build_install_llvm.sh [options] -- [cmake-args]
 
 Checkout svn sources and run cmake with the specified arguments. Used
 inside docker container.
-Passes additional -DCMAKE_INSTALL_PREFIX and archives the contents of
-the directory to /tmp/clang.tar.gz.
+Passes additional -DCMAKE_INSTALL_PREFIX and puts the build results into
+/tmp/clang-install/ directory.
 
 Available options:
   -h|--help           show this help message
@@ -244,12 +244,7 @@ ninja $CMAKE_INSTALL_TARGETS
 
 popd
 
-# Pack the installed clang into an archive.
-echo "Archiving clang installation to /tmp/clang.tar.gz"
-cd "$CLANG_INSTALL_DIR"
-tar -czf /tmp/clang.tar.gz *
-
 # Cleanup.
-rm -rf "$CLANG_BUILD_DIR" "$CLANG_INSTALL_DIR"
+rm -rf "$CLANG_BUILD_DIR"
 
 echo "Done"
