|
| 1 | +========================================= |
| 2 | +A guide to Dockerfiles for building LLVM |
| 3 | +========================================= |
| 4 | + |
| 5 | +Introduction |
| 6 | +============ |
| 7 | +You can find a number of sources to build docker images with LLVM components in |
| 8 | +``llvm/utils/docker``. They can be used by anyone who wants to build the docker |
| 9 | +images for their own use, or as a starting point for someone who wants to write |
| 10 | +their own Dockerfiles. |
| 11 | + |
| 12 | +We currently provide Dockerfiles with ``debian8`` and ``nvidia-cuda`` base images. |
| 13 | +We also provide an ``example`` image, which contains placeholders that one would need |
| 14 | +to fill out in order to produce Dockerfiles for a new docker image. |
| 15 | + |
| 16 | +Why? |
| 17 | +---- |
| 18 | +Docker images provide a way to produce binary distributions of |
| 19 | +software inside a controlled environment. Having Dockerfiles to builds docker images |
| 20 | +inside LLVM repo makes them much more discoverable than putting them into any other |
| 21 | +place. |
| 22 | + |
| 23 | +Docker basics |
| 24 | +------------- |
| 25 | +If you've never heard about Docker before, you might find this section helpful |
| 26 | +to get a very basic explanation of it. |
| 27 | +`Docker <https://www.docker.com/>`_ is a popular solution for running programs in |
| 28 | +an isolated and reproducible environment, especially to maintain releases for |
| 29 | +software deployed to large distributed fleets. |
| 30 | +It uses linux kernel namespaces and cgroups to provide a lightweight isolation |
| 31 | +inside currently running linux kernel. |
| 32 | +A single active instance of dockerized environment is called a *docker |
| 33 | +container*. |
| 34 | +A snapshot of a docker container filesystem is called a *docker image*. |
| 35 | +One can start a container from a prebuilt docker image. |
| 36 | + |
| 37 | +Docker images are built from a so-called *Dockerfile*, a source file written in |
| 38 | +a specialized language that defines instructions to be used when build |
| 39 | +the docker image (see `official |
| 40 | +documentation <https://docs.docker.com/engine/reference/builder/>`_ for more |
| 41 | +details). A minimal Dockerfile typically contains a base image and a number |
| 42 | +of RUN commands that have to be executed to build the image. When building a new |
| 43 | +image, docker will first download your base image, mount its filesystem as |
| 44 | +read-only and then add a writable overlay on top of it to keep track of all |
| 45 | +filesystem modifications, performed while building your image. When the build |
| 46 | +process is finished, a diff between your image's final filesystem state and the |
| 47 | +base image's filesystem is stored in the resulting image. |
| 48 | + |
| 49 | +Overview |
| 50 | +======== |
| 51 | +The ``llvm/utils/docker`` folder contains Dockerfiles and simple bash scripts to |
| 52 | +serve as a basis for anyone who wants to create their own Docker image with |
| 53 | +LLVM components, compiled from sources. The sources are checked out from the |
| 54 | +upstream svn repository when building the image. |
| 55 | + |
| 56 | +Inside each subfolder we host Dockerfiles for two images: |
| 57 | + |
| 58 | +- ``build/`` image is used to compile LLVM, it installs a system compiler and all |
| 59 | + build dependencies of LLVM. After the build process is finished, the build |
| 60 | + image will have an archive with compiled components at ``/tmp/clang.tar.gz``. |
| 61 | +- ``release/`` image usually only contains LLVM components, compiled by the |
| 62 | + ``build/`` image, and also libstdc++ and binutils to make image minimally |
| 63 | + useful for C++ development. The assumption is that you usually want clang to |
| 64 | + be one of the provided components. |
| 65 | + |
| 66 | +To build both of those images, use ``build_docker_image.sh`` script. |
| 67 | +It will checkout LLVM sources and build clang in the ``build`` container, copy results |
| 68 | +of the build to the local filesystem and then build the ``release`` container using |
| 69 | +those. The ``build_docker_image.sh`` accepts a list of LLVM repositories to |
| 70 | +checkout, and arguments for CMake invocation. |
| 71 | + |
| 72 | +If you want to write your own docker image, start with an ``example/`` subfolder. |
| 73 | +It provides incomplete Dockerfiles with (very few) FIXMEs explaining the steps |
| 74 | +you need to take in order to make your Dockerfiles functional. |
| 75 | + |
| 76 | +Usage |
| 77 | +===== |
| 78 | +The ``llvm/utils/build_docker_image.sh`` script provides a rather high degree of |
| 79 | +control on how to run the build. It allows you to specify the projects to |
| 80 | +checkout from svn and provide a list of CMake arguments to use during when |
| 81 | +building LLVM inside docker container. |
| 82 | + |
| 83 | +Here's a very simple example of getting a docker image with clang binary, |
| 84 | +compiled by the system compiler in the debian8 image: |
| 85 | + |
| 86 | +.. code-block:: bash |
| 87 | +
|
| 88 | + ./llvm/utils/docker/build_docker_image.sh \ |
| 89 | + --source debian8 \ |
| 90 | + --docker-repository clang-debian8 --docker-tag "staging" \ |
| 91 | + -- \ |
| 92 | + -p clang -i install-clang -i install-clang-headers \ |
| 93 | + -- \ |
| 94 | + -DCMAKE_BUILD_TYPE=Release |
| 95 | +
|
| 96 | +Note there are two levels of ``--`` indirection. First one separates |
| 97 | +``build_docker_image.sh`` arguments from ``llvm/utils/build_install_llvm.sh`` |
| 98 | +arguments. Second one separates CMake arguments from ``build_install_llvm.sh`` |
| 99 | +arguments. Note that build like that doesn't use a 2-stage build process that |
| 100 | +you probably want for clang. Running a 2-stage build is a little more intricate, |
| 101 | +this command will do that: |
| 102 | + |
| 103 | +.. code-block:: bash |
| 104 | +
|
| 105 | + # Run a 2-stage build. |
| 106 | + # LLVM_TARGETS_TO_BUILD=Native is to reduce stage1 compile time. |
| 107 | + # Options, starting with BOOTSTRAP_* are passed to stage2 cmake invocation. |
| 108 | + ./build_docker_image.sh \ |
| 109 | + --source debian8 \ |
| 110 | + --docker-repository clang-debian8 --docker-tag "staging" \ |
| 111 | + -- \ |
| 112 | + -p clang -i stage2-install-clang -i stage2-install-clang-headers \ |
| 113 | + -- \ |
| 114 | + -DLLVM_TARGETS_TO_BUILD=Native -DCMAKE_BUILD_TYPE=Release \ |
| 115 | + -DBOOTSTRAP_CMAKE_BUILD_TYPE=Release \ |
| 116 | + -DCLANG_ENABLE_BOOTSTRAP=ON -DCLANG_BOOTSTRAP_TARGETS="install-clang;install-clang-headers" |
| 117 | + |
| 118 | +This will produce two images, a release image ``clang-debian8:staging`` and a |
| 119 | +build image ``clang-debian8-build:staging`` from the latest upstream revision. |
| 120 | +After the image is built you can run bash inside a container based on your |
| 121 | +image like this: |
| 122 | + |
| 123 | +.. code-block:: bash |
| 124 | +
|
| 125 | + docker run -ti clang-debian8:staging bash |
| 126 | +
|
| 127 | +Now you can run bash commands as you normally would: |
| 128 | + |
| 129 | +.. code-block:: bash |
| 130 | +
|
| 131 | + root@80f351b51825:/# clang -v |
| 132 | + clang version 5.0.0 (trunk 305064) |
| 133 | + Target: x86_64-unknown-linux-gnu |
| 134 | + Thread model: posix |
| 135 | + InstalledDir: /bin |
| 136 | + Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.8 |
| 137 | + Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.8.4 |
| 138 | + Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.9 |
| 139 | + Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.9.2 |
| 140 | + Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.9 |
| 141 | + Candidate multilib: .;@m64 |
| 142 | + Selected multilib: .;@m64 |
| 143 | +
|
| 144 | +
|
| 145 | +Which image should I choose? |
| 146 | +============================ |
| 147 | +We currently provide two images: debian8-based and nvidia-cuda-based. They |
| 148 | +differ in the base image that they use, i.e. they have a different set of |
| 149 | +preinstalled binaries. Debian8 is very minimal, nvidia-cuda is larger, but has |
| 150 | +preinstalled CUDA libraries and allows to access a GPU, installed on your |
| 151 | +machine. |
| 152 | + |
| 153 | +If you need a minimal linux distribution with only clang and libstdc++ included, |
| 154 | +you should try debian8-based image. |
| 155 | + |
| 156 | +If you want to use CUDA libraries and have access to a GPU on your machine, |
| 157 | +you should choose nvidia-cuda-based image and use `nvidia-docker |
| 158 | +<https://github.com/NVIDIA/nvidia-docker>`_ to run your docker containers. Note |
| 159 | +that you don't need nvidia-docker to build the images, but you need it in order |
| 160 | +to have an access to GPU from a docker container that is running the built |
| 161 | +image. |
| 162 | + |
| 163 | +If you have a different use-case, you could create your own image based on |
| 164 | +``example/`` folder. |
| 165 | + |
| 166 | +Any docker image can be built and run using only the docker binary, i.e. you can |
| 167 | +run debian8 build on Fedora or any other Linux distribution. You don't need to |
| 168 | +install CMake, compilers or any other clang dependencies. It is all handled |
| 169 | +during the build process inside Docker's isolated environment. |
| 170 | + |
| 171 | +Stable build |
| 172 | +============ |
| 173 | +If you want a somewhat recent and somewhat stable build, use the |
| 174 | +``branches/google/stable`` branch, i.e. the following command will produce a |
| 175 | +debian8-based image using the latest ``google/stable`` sources for you: |
| 176 | + |
| 177 | +.. code-block:: bash |
| 178 | +
|
| 179 | + ./llvm/utils/docker/build_docker_image.sh \ |
| 180 | + -s debian8 --d clang-debian8 -t "staging" \ |
| 181 | + -- \ |
| 182 | + --branch branches/google/stable \ |
| 183 | + -p clang -i install-clang -i install-clang-headers \ |
| 184 | + -- \ |
| 185 | + -DCMAKE_BUILD_TYPE=Release |
| 186 | +
|
| 187 | +
|
| 188 | +Minimizing docker image size |
| 189 | +============================ |
| 190 | +Due to Docker restrictions we use two images (i.e., build and release folders) |
| 191 | +for the release image to be as small as possible. It's much easier to achieve |
| 192 | +that using two images, because Docker would store a filesystem layer for each |
| 193 | +command in the Dockerfile, i.e. if you install some packages in one command, |
| 194 | +then remove those in a separate command, the size of the resulting image will |
| 195 | +still be proportinal to the size of an image with installed packages. |
| 196 | +Therefore, we strive to provide a very simple release image which only copies |
| 197 | +compiled clang and does not do anything else. |
| 198 | + |
| 199 | +Docker 1.13 added a ``--squash`` flag that allows to flatten the layers of the |
| 200 | +image, i.e. remove the parts that were actually deleted. That is an easier way |
| 201 | +to produce the smallest images possible by using just a single image. We do not |
| 202 | +use it because as of today the flag is in experimental stage and not everyone |
| 203 | +may have the latest docker version available. When the flag is out of |
| 204 | +experimental stage, we should investigate replacing two images approach with |
| 205 | +just a single image, built using ``--squash`` flag. |
0 commit comments