Skip to content

Commit

Permalink
added Dockerfile for CPU backend
Browse files Browse the repository at this point in the history
Summary:
Dockerfiles update:
- added dockerfile with flashligth CPU backend
- switch arrayfire to 3.6.2 version for CUDA and CPU dockerfiles

Reviewed By: jacobkahn

Differential Revision: D13632520

fbshipit-source-id: a77845a6fbe48e7230bcb5622099ddb8a49da97f
  • Loading branch information
Tatiana Likhomanenko authored and facebook-github-bot committed Jan 14, 2019
1 parent 0949e3d commit 4ec2fa6
Show file tree
Hide file tree
Showing 3 changed files with 114 additions and 16 deletions.
97 changes: 97 additions & 0 deletions Dockerfile-CPU
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
# ==================================================================
# module list
# ------------------------------------------------------------------
# Ubuntu 16.04
# OpenMPI latest (apt)
# cmake 3.10 (git)
# MKL 2018.4-057 (apt)
# arrayfire 3.6.2 (git, CPU backend)
# MKLDNN 4bdffc2 (git)
# Gloo b7e0906 (git)
# flashlight master (git, CPU backend)
# ==================================================================

FROM ubuntu:16.04

RUN APT_INSTALL="apt-get install -y --no-install-recommends" && \
apt-get update && \
DEBIAN_FRONTEND=noninteractive $APT_INSTALL \
build-essential \
ca-certificates \
wget \
git \
vim \
emacs \
nano \
htop \
g++ \
# for MKL
apt-transport-https \
# for arrayfire CPU backend
# OpenBLAS
libopenblas-dev libfftw3-dev liblapacke-dev \
# ATLAS
libatlas3-base libatlas-dev libfftw3-dev liblapacke-dev \
# ssh for OpenMPI
openssh-server openssh-client \
# OpenMPI
libopenmpi-dev libomp-dev && \
# ==================================================================
# cmake 3.10 (for MKLDNN)
# ------------------------------------------------------------------
apt-get purge -y cmake && \
# for cmake
DEBIAN_FRONTEND=noninteractive $APT_INSTALL zlib1g-dev libcurl4-openssl-dev && \
cd /tmp && wget https://cmake.org/files/v3.10/cmake-3.10.3.tar.gz && \
tar -xzvf cmake-3.10.3.tar.gz && cd cmake-3.10.3 && \
./bootstrap --system-curl && \
make -j8 && make install && cmake --version && \
# ==================================================================
# MKL https://software.intel.com/en-us/mkl
# ------------------------------------------------------------------
cd /tmp && wget https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS-2019.PUB && \
apt-key add GPG-PUB-KEY-INTEL-SW-PRODUCTS-2019.PUB && \
wget https://apt.repos.intel.com/setup/intelproducts.list -O /etc/apt/sources.list.d/intelproducts.list && \
sh -c 'echo deb https://apt.repos.intel.com/mkl all main > /etc/apt/sources.list.d/intel-mkl.list' && \
apt-get update && DEBIAN_FRONTEND=noninteractive $APT_INSTALL intel-mkl-64bit-2018.4-057 && \
export MKLROOT=/opt/intel/mkl && \
# ==================================================================
# arrayfire with CPU backend https://github.com/arrayfire/arrayfire/wiki/
# ------------------------------------------------------------------
cd /tmp && git clone --recursive https://github.com/arrayfire/arrayfire.git && \
cd arrayfire && git checkout v3.6.2 && \
mkdir build && cd build && \
CXXFLAGS=-DOS_LNX cmake .. -DCMAKE_BUILD_TYPE=Release -DAF_BUILD_CUDA=OFF -DAF_BUILD_OPENCL=OFF -DAF_BUILD_EXAMPLES=OFF && \
make -j8 && make install && \
# ==================================================================
# MKLDNN https://github.com/intel/mkl-dnn/
# ------------------------------------------------------------------
cd /tmp && git clone https://github.com/intel/mkl-dnn.git && \
cd mkl-dnn && git checkout 4bdffc2 && mkdir -p build && cd build && \
cmake .. -DMKLDNN_USE_MKL=FULL && \
make -j8 && make install && \
# ==================================================================
# Gloo https://github.com/facebookincubator/gloo.git
# ------------------------------------------------------------------
cd /tmp && git clone https://github.com/facebookincubator/gloo.git && \
cd gloo && git checkout b7e0906 && mkdir build && cd build && \
cmake .. -DUSE_MPI=ON && \
make -j8 && make install && \
# ==================================================================
# config & cleanup
# ------------------------------------------------------------------
ldconfig && \
apt-get clean && \
apt-get autoremove && \
rm -rf /var/lib/apt/lists/* /tmp/* ~/*

RUN mkdir /root/flashlight
COPY . /root/flashlight

# ==================================================================
# flashlight with CPU backend
# ------------------------------------------------------------------
RUN export MKLROOT=/opt/intel/mkl && \
cd /root/flashlight && mkdir -p build && cd build && \
cmake .. -DCMAKE_BUILD_TYPE=Release -DFLASHLIGHT_BACKEND=CPU && \
make -j8 && make install
19 changes: 6 additions & 13 deletions Dockerfile-CUDA
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@
# Ubuntu 16.04
# CUDA 9.2
# CuDNN 7-dev
# arrayfire 3.6.1 (git, CUDA backend)
# OpenMPI 4.0.0 (.gz)
# arrayfire 3.6.2 (git, CUDA backend)
# OpenMPI latest (apt)
# flashlight master (git, CUDA backend)
# ==================================================================

Expand All @@ -29,29 +29,22 @@ RUN APT_INSTALL="apt-get install -y --no-install-recommends" && \
g++ \
# ssh for OpenMPI
openssh-server openssh-client \
# OpenMPI
libopenmpi-dev libomp-dev \
# nccl: for flashlight
libnccl2 libnccl-dev && \
# ==================================================================
# arrayfire https://github.com/arrayfire/arrayfire/wiki/
# ------------------------------------------------------------------
cd /tmp && git clone --recursive https://github.com/arrayfire/arrayfire.git && \
cd arrayfire && git checkout b443e14 && \
cd arrayfire && git checkout v3.6.2 && \
mkdir build && cd build && \
cmake .. -DCMAKE_BUILD_TYPE=Release -DAF_BUILD_CPU=OFF -DAF_BUILD_OPENCL=OFF -DAF_BUILD_EXAMPLES=OFF && \
CXXFLAGS=-DOS_LNX cmake .. -DCMAKE_BUILD_TYPE=Release -DAF_BUILD_CPU=OFF -DAF_BUILD_OPENCL=OFF -DAF_BUILD_EXAMPLES=OFF && \
make -j8 && \
make install && \
# ==================================================================
# OpenMPI https://www.open-mpi.org/
# ------------------------------------------------------------------
cd /tmp && wget https://download.open-mpi.org/release/open-mpi/v4.0/openmpi-4.0.0.tar.gz && \
gunzip -c openmpi-4.0.0.tar.gz | tar xf - && \
cd openmpi-4.0.0 && \
./configure --prefix=/usr/local && \
make all install && \
# ==================================================================
# config & cleanup
# ------------------------------------------------------------------
# at least to link correctly OpenMPI
ldconfig && \
apt-get clean && \
apt-get autoremove && \
Expand Down
14 changes: 11 additions & 3 deletions docs/source/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -110,15 +110,20 @@ Building flashlight on Windows is not supported at this time (coming soon).

Building/Running flashlight with Docker
---------------------------------------
flashlight and its dependencies can also be built with the provided Dockerfile. Only the CUDA backend is supported with Docker at this time.
flashlight and its dependencies can also be built with the provided Dockerfile.

To build flashlight with Docker:
- Install `Docker <https://docs.docker.com/engine/installation/>`_ and `nvidia-docker <https://github.com/NVIDIA/nvidia-docker/>`_

- Install `Docker <https://docs.docker.com/engine/installation/>`_
- For CUDA backend install `nvidia-docker <https://github.com/NVIDIA/nvidia-docker/>`_
- Run the given Dockerfile in a new container:

.. code-block:: shell
# for CUDA backend
sudo docker run --runtime=nvidia --rm -itd --ipc=host --name flashlight flml/flashlight:cuda-latest
# for CPU backend
sudo docker run --rm -itd --name flashlight flml/flashlight:cpu-latest
# go to terminal in the container
sudo docker exec -it flashlight bash
- to run tests inside a container
Expand All @@ -133,7 +138,10 @@ To build flashlight with Docker:
git clone --recursive https://github.com/facebookresearch/flashlight.git
cd flashlight
# for CUDA backend
sudo docker build -f ./Dockerfile-CUDA -t flashlight .
# for CPU backend
sudo docker build -f ./Dockerfile-CPU -t flashlight .
Building Your Project with flashlight
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Expand Down

0 comments on commit 4ec2fa6

Please sign in to comment.