Untiered docker images for easy Jenking setup

facebookresearch · Mar 2, 2018 · 63337a4 · 63337a4
1 parent d267bff
commit 63337a4
Show file tree

Hide file tree

Showing 14 changed files with 249 additions and 391 deletions.
diff --git a/docker/README.md b/docker/README.md
@@ -14,53 +14,40 @@ You can verify your docker installation is fine by running a docker test:
 docker run hello-world
 ```
 
-1. First, you need to clone TensorComprehensions repo
+1. Clone TensorComprehensions repo
 
 ```Shell
 git clone --recursive git@github.com:facebookresearch/TensorComprehensions.git
 ```
 
-and go to the TensorComprehensions repo
+2. Build the docker image
 
-```Shell
-cd TensorComprehensions/docker
-```
-Now, we are going to start building docker image for TC in various steps. We will
-build docker image for TC on linux trusty with gcc4.8, cuda8, cudnn6 and conda environment settings. If you need to build images for some other combination,
-the steps are still valid, just make sure to change the paths below to fit your case.
-
-2. We will first build the docker image of trusty and all the dependencies of TC. Since the dependencies don't change, we can build this image once and keep reusing it.
-
-```Shell
-cd linux-trusty                    # go to the relevant directory
-docker build -t tensorcomprehensions/linux-trusty:2 .   # build image with tag 2 (this can be any number)
-```
-
-3. Now, we will build the image using the base image built in step 1 and we will add gcc4.8 and clang+llvm-tapir5.0 to the image
-
-```Shell
-cd TensorComprehensions/docker/linux-trusty-gcc4.8-tapir5.0
-docker build --build-arg BUILD_ID=2 -t tensorcomprehensions/linux-trusty-gcc4.8-tapir5.0:2 .
-```
-
-4. Now, let's use the above image and add cuda8-cudnn6 to the image
+Go to the TensorComprehensions repo
 
 ```Shell
 cd TensorComprehensions/docker
-docker build --build-arg BUILD_ID=2 --build-arg BUILD=gcc48-cuda8-cudnn6 -f linux-trusty-gcc4.8-tapir5.0-cuda8-cudnn6/Dockerfile -t tensorcomprehensions/linux-trusty-gcc4.8-tapir5.0-cuda8-cudnn6:2 .
 ```
 
-5. Now, using above image, let's build the final image with python3 + conda
+Now, we will build a docker image for TC. We will build docker image for TC on
+linux trusty with gcc4.8, cuda8, cudnn6 and conda environment settings. If you
+need to build images for some other combination, the steps are still valid, just
+make sure to change the paths below to fit your case.
 
 ```Shell
-cd TensorComprehensions/docker/linux-trusty-gcc4.8-tapir5.0-cuda8-cudnn6-py3-conda/
-docker build --build-arg BUILD_ID=2 -t tensorcomprehensions/linux-trusty-gcc4.8-tapir5.0-cuda8-cudnn6-py3-conda:2 .
+docker build -t tensorcomprehensions/linux-trusty-gcc4.8-cuda8-cudnn6-py3-conda .
 ```
 
-6. Finally, let's run the above image, and install TC to verify the image built
-is correct.
+This will build the image for the above permutation and then we can test this image
+
+6. Test the image
 
 ```Shell
-docker run -i -t linux-trusty-gcc4.8-tapir5.0-cuda8-cudnn6-py3-conda:2
-# now clone the TC repo and run the build, at the end run test_cpu.sh to verify
+docker run --runtime=nvidia -i -t tensorcomprehensions/linux-trusty-gcc4.8-cuda8-cudnn6-py3-conda:2
+# now clone the TC repo
+cd $HOME && git clone https://github.com/facebookresearch/TensorComprehensions.git --recursive && cd TensorComprehensions
+# build TC
+WITH_PYTHON_C2=OFF CLANG_PREFIX=/usr/local/clang+llvm-tapir5.0 ./build.sh --all
+# Test the TC build is fine
+./test_cpu.sh
+./test.sh
 ```
diff --git a/docker/common/install_cuda.sh b/docker/common/install_cuda.sh
diff --git a/docker/linux-trusty-gcc4.8-cuda8-cudnn6-py3-conda/Dockerfile b/docker/linux-trusty-gcc4.8-cuda8-cudnn6-py3-conda/Dockerfile
@@ -1,25 +1,57 @@
-ARG BUILD_ID
-FROM tensorcomprehensions/linux-trusty-gcc4.8-cuda8-cudnn6:${BUILD_ID}
+FROM nvidia/cuda:8.0-cudnn6-devel-ubuntu14.04
 
-ENV LD_LIBRARY_PATH /usr/lib/:/usr/local/cuda/lib64:/usr/local/cuda/targets/x86_64-linux/lib/stubs/:$LD_LIBRARY_PATH
+ENV DEBIAN_FRONTEND noninteractive
+
+RUN apt-get update
+
+RUN apt-get install -y --no-install-recommends make git ssh realpath wget unzip cmake3 vim
+RUN apt-get install -y --no-install-recommends libgoogle-glog-dev libyaml-dev
+RUN apt-get install -y --no-install-recommends libgtest-dev libz-dev libgmp3-dev
+RUN apt-get install -y --no-install-recommends automake libtool valgrind subversion
+RUN apt-get install -y --no-install-recommends ca-certificates software-properties-common
+
+RUN cmake --version
+
+# GCC 4.8.*
+RUN add-apt-repository ppa:ubuntu-toolchain-r/test
+RUN apt-get update
+RUN apt-get install -y --no-install-recommends libcilkrts5 gcc-4.8 g++-4.8
+RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.8 50
+RUN update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-4.8 50
+
+ENV CC /usr/bin/gcc
+ENV CXX /usr/bin/g++
+
+# LLVM+Clang-Tapir5.0
+ENV LLVM_SOURCES /tmp/llvm_sources-tapir5.0
+WORKDIR $LLVM_SOURCES
+
+ENV CLANG_PREFIX /usr/local/clang+llvm-tapir5.0
+ENV CMAKE_VERSION cmake
+
+RUN git clone --recursive https://github.com/wsmoses/Tapir-LLVM llvm && \
+    mkdir -p ${LLVM_SOURCES}/llvm_build && cd ${LLVM_SOURCES}/llvm_build && \
+    ${CMAKE_VERSION} -DLLVM_ENABLE_EH=ON -DLLVM_ENABLE_OCAMLDOC=OFF -DLLVM_INSTALL_OCAMLDOC_HTML_DIR=/tmp -DLLVM_OCAML_INSTALL_PATH=/tmp -DCMAKE_INSTALL_PREFIX=${CLANG_PREFIX} -DLLVM_TARGETS_TO_BUILD=X86 -DCOMPILER_RT_BUILD_CILKTOOLS=OFF -DLLVM_ENABLE_CXX1Y=ON -DLLVM_ENABLE_TERMINFO=OFF -DLLVM_BUILD_TESTS=OFF -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_BUILD_TYPE=Release -DLLVM_BUILD_LLVM_DYLIB=ON  -DLLVM_ENABLE_RTTI=ON ../llvm/ && \
+    make -j"$(nproc)" -s && make install -j"$(nproc)" -s
+
+RUN rm -Rf ${LLVM_SOURCES}
+
+# CUDA paths
+RUN ln -s /usr/local/cuda/targets/x86_64-linux/lib/stubs/libcuda.so /usr/local/cuda/targets/x86_64-linux/lib/stubs/libcuda.so.1
+ENV LD_LIBRARY_PATH /usr/local/cuda/lib64:/usr/local/cuda/targets/x86_64-linux/lib/stubs/:$LD_LIBRARY_PATH
 ENV PATH /usr/local/bin:/usr/local/cuda/bin:$PATH
 
+# Anaconda3
 WORKDIR /conda-install
 RUN echo 'export PATH=/opt/conda/bin:$PATH' > /etc/profile.d/conda.sh &&\
     wget --quiet https://repo.continuum.io/archive/Anaconda3-5.0.1-Linux-x86_64.sh -O anaconda.sh && \
-    chmod +x anaconda.sh && \
-    ./anaconda.sh -b -p /opt/conda && \
-    rm anaconda.sh
+    chmod +x anaconda.sh && ./anaconda.sh -b -p /opt/conda && rm anaconda.sh
 
 ENV PATH /opt/conda/bin:$PATH
 
-RUN conda install \
-         numpy\
-         decorator\
-         six\
-         future\
-         cmake
+RUN conda install numpy decorator six future cmake pyyaml
 
+# Protobuf 3.4*
 WORKDIR /proto-install
 RUN wget --quiet https://github.com/google/protobuf/archive/v3.4.0.zip -O proto.zip && unzip -qq proto.zip -d /
 
@@ -33,4 +65,8 @@ RUN protoc --version
 RUN which python
 RUN python --version
 
+# Cleanup the package manager
+RUN apt-get autoclean && apt-get clean
+RUN rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
+
 CMD ["bash"]
diff --git a/docker/linux-trusty-gcc4.8-cuda8-cudnn6-py3/Dockerfile b/docker/linux-trusty-gcc4.8-cuda8-cudnn6-py3/Dockerfile
@@ -1,27 +1,52 @@
-ARG BUILD_ID
-FROM tensorcomprehensions/linux-trusty-gcc4.8-cuda8-cudnn6:${BUILD_ID}
+FROM nvidia/cuda:8.0-cudnn6-devel-ubuntu14.04
 
 ENV DEBIAN_FRONTEND noninteractive
 
 RUN apt-get update
 
-RUN apt-get install -y --no-install-recommends curl
-RUN apt-get install -y --no-install-recommends protobuf-compiler libprotobuf-dev
-RUN apt-get install -y --no-install-recommends python3-dev python3-pip python3-setuptools
+# System dependencies
+RUN apt-get install -y --no-install-recommends curl make git ssh realpath wget unzip cmake3 vim
+RUN apt-get install -y --no-install-recommends libgoogle-glog-dev libyaml-dev
+RUN apt-get install -y --no-install-recommends libgtest-dev libz-dev libgmp3-dev
+RUN apt-get install -y --no-install-recommends automake libtool valgrind subversion
+RUN apt-get install -y --no-install-recommends ca-certificates software-properties-common
+RUN cmake --version
+
+# GCC 4.8.*
+RUN add-apt-repository ppa:ubuntu-toolchain-r/test
+RUN apt-get update
+RUN apt-get install -y --no-install-recommends libcilkrts5 gcc-4.8 g++-4.8
+RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.8 50
+RUN update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-4.8 50
+
+ENV CC /usr/bin/gcc
+ENV CXX /usr/bin/g++
+
+# LLVM+Clang-Tapir5.0
+ENV LLVM_SOURCES /tmp/llvm_sources-tapir5.0
+WORKDIR $LLVM_SOURCES
 
-ENV LD_LIBRARY_PATH /usr/lib/:/usr/local/cuda/lib64:/usr/local/cuda/targets/x86_64-linux/lib/stubs/:$LD_LIBRARY_PATH
+ENV CLANG_PREFIX /usr/local/clang+llvm-tapir5.0
+ENV CMAKE_VERSION cmake
+
+RUN git clone --recursive https://github.com/wsmoses/Tapir-LLVM llvm && \
+    mkdir -p ${LLVM_SOURCES}/llvm_build && cd ${LLVM_SOURCES}/llvm_build && \
+    ${CMAKE_VERSION} -DLLVM_ENABLE_EH=ON -DLLVM_ENABLE_OCAMLDOC=OFF -DLLVM_INSTALL_OCAMLDOC_HTML_DIR=/tmp -DLLVM_OCAML_INSTALL_PATH=/tmp -DCMAKE_INSTALL_PREFIX=${CLANG_PREFIX} -DLLVM_TARGETS_TO_BUILD=X86 -DCOMPILER_RT_BUILD_CILKTOOLS=OFF -DLLVM_ENABLE_CXX1Y=ON -DLLVM_ENABLE_TERMINFO=OFF -DLLVM_BUILD_TESTS=OFF -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_BUILD_TYPE=Release -DLLVM_BUILD_LLVM_DYLIB=ON  -DLLVM_ENABLE_RTTI=ON ../llvm/ && \
+    make -j"$(nproc)" -s && make install -j"$(nproc)" -s
+
+RUN rm -Rf ${LLVM_SOURCES}
+
+# CUDA paths
+RUN ln -s /usr/local/cuda/targets/x86_64-linux/lib/stubs/libcuda.so /usr/local/cuda/targets/x86_64-linux/lib/stubs/libcuda.so.1
+ENV LD_LIBRARY_PATH /usr/local/cuda/lib64:/usr/local/cuda/targets/x86_64-linux/lib/stubs/:$LD_LIBRARY_PATH
 ENV PATH /usr/local/bin:/usr/local/cuda/bin:$PATH
 
+# Python3
+RUN apt-get install -y --no-install-recommends python3-dev python3-pip python3-setuptools
 RUN pip3 install --upgrade pip
-RUN pip3 install \
-         numpy\
-         decorator\
-         six\
-         future\
-         protobuf\
-         setuptools\
-         pyyaml
+RUN pip3 install numpy decorator six future setuptools pyyaml
 
+# Protobuf 3.4*
 WORKDIR /proto-install
 RUN wget --quiet https://github.com/google/protobuf/archive/v3.4.0.zip -O proto.zip && unzip -qq proto.zip -d /
 
@@ -34,4 +59,8 @@ RUN python3 -c 'import yaml'
 RUN which protoc
 RUN protoc --version
 
+# Cleanup the package manager
+RUN apt-get autoclean && apt-get clean
+RUN rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
+
 CMD ["bash"]
diff --git a/docker/linux-trusty-gcc4.8-cuda8-cudnn6/Dockerfile b/docker/linux-trusty-gcc4.8-cuda8-cudnn6/Dockerfile
diff --git a/docker/linux-trusty-gcc4.8/Dockerfile b/docker/linux-trusty-gcc4.8/Dockerfile