Caffe2 Installation inside Pytorch

[ghost]

i am getting this error message (please see the attached) after run the command "python setup.py install":
.
I followed this office guide of pytorch: https://caffe2.ai/docs/getting-started.html?platform=ubuntu&configuration=compile.
But the weird thing is I can print the coffee2 sucessfully

i would like to ask if does anybody knows how to fix this?

[mhubii]

Maybe you have already installed caffe2 from anaconda's pre-built binaries like so https://caffe2.ai/docs/getting-started.html?platform=ubuntu&configuration=prebuilt

Please provide the full error message instead of a printscreen.

[ghost]

Hi @mhubii, thank for your reply. Yes I have already installed caffe2 from anaconda's pre-built binaries. But I wasn't aware of it. Now how can I fix the issue?

here is my full error message:

Building wheel torch-1.0.0a0+6dd7194
running install
setup.py::run()
running build_deps
setup.py::build_deps::run()

• SYNC_COMMAND=cp
  ++ command -v rsync
• '[' -x /usr/bin/rsync ']'
• SYNC_COMMAND='rsync -lptgoD'
• USE_CUDA=0
• USE_ROCM=0
• USE_NNPACK=0
• USE_MKLDNN=0
• USE_GLOO_IBVERBS=0
• CAFFE2_STATIC_LINK_CUDA=0
• RERUN_CMAKE=1
• [[ 8 -gt 0 ]]
• case "$1" in
• USE_CUDA=1
• shift
• [[ 7 -gt 0 ]]
• case "$1" in
• USE_NNPACK=1
• shift
• [[ 6 -gt 0 ]]
• case "$1" in
• break
• CMAKE_INSTALL='make install'
• BUILD_SHARED_LIBS=ON
• USER_CFLAGS=
• USER_LDFLAGS=
• [[ -n '' ]]
• [[ -n '' ]]
• [[ -n '' ]]
  ++ uname
• '[' Linux == Darwin ']'
  +++ dirname ../tools/build_pytorch_libs.sh
  ++ cd ../tools/..
  +++ pwd
  ++ printf '%q\n' /home/bapvn/work/caffe2-pytorch
• BASE_DIR=/home/bapvn/work/caffe2-pytorch
• TORCH_LIB_DIR=/home/bapvn/work/caffe2-pytorch/torch/lib
• INSTALL_DIR=/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install
• THIRD_PARTY_DIR=/home/bapvn/work/caffe2-pytorch/third_party
• CMAKE_VERSION=cmake
• C_FLAGS=' -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include" -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include/TH" -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include/THC" -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include/THS" -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include/THCS" -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include/THNN" -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include/THCUNN"'
• C_FLAGS=' -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include" -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include/TH" -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include/THC" -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include/THS" -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include/THCS" -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include/THNN" -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include/THCUNN" -DOMPI_SKIP_MPICXX=1'
• LDFLAGS='-L"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/lib" '
• LD_POSTFIX=.so
  ++ uname
• [[ Linux == \D\a\r\w\i\n ]]
• [[ 0 -eq 1 ]]
• LDFLAGS='-L"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/lib" -Wl,-rpath,$ORIGIN'
• CPP_FLAGS=' -std=c++11 '
• GLOO_FLAGS='-DBUILD_TEST=OFF '
• THD_FLAGS=
• NCCL_ROOT_DIR=/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install
• [[ 1 -eq 1 ]]
• GLOO_FLAGS+='-DUSE_CUDA=1 -DNCCL_ROOT_DIR=/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install'
• [[ 0 -eq 1 ]]
• CWRAP_FILES='/home/bapvn/work/caffe2-pytorch/torch/lib/ATen/Declarations.cwrap;/home/bapvn/work/caffe2-pytorch/torch/lib/THNN/generic/THNN.h;/home/bapvn/work/caffe2-pytorch/torch/lib/THCUNN/generic/THCUNN.h;/home/bapvn/work/caffe2-pytorch/torch/lib/ATen/nn.yaml'
• CUDA_NVCC_FLAGS=' -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include" -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include/TH" -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include/THC" -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include/THS" -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include/THCS" -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include/THNN" -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include/THCUNN" -DOMPI_SKIP_MPICXX=1'
• [[ -z '' ]]
• CUDA_DEVICE_DEBUG=0
• '[' -z '' ']'
  ++ getconf _NPROCESSORS_ONLN
• MAX_JOBS=6
• BUILD_TYPE=Release
• [[ -n '' ]]
• [[ -n '' ]]
• echo 'Building in Release mode'
  Building in Release mode
• mkdir -p /home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install
• for arg in '"$@"'
• [[ nccl == \n\c\c\l ]]
• pushd /home/bapvn/work/caffe2-pytorch/third_party
  ~/work/caffe2-pytorch/third_party ~/work/caffe2-pytorch/build
• build_nccl
• mkdir -p build/nccl
• pushd build/nccl
  ~/work/caffe2-pytorch/third_party/build/nccl ~/work/caffe2-pytorch/third_party ~/work/caffe2-pytorch/build
• [[ 1 -eq 1 ]]
• cmake ../../nccl -DCMAKE_MODULE_PATH=/home/bapvn/work/caffe2-pytorch/cmake/Modules_CUDA_fix -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install '-DCMAKE_C_FLAGS= -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include" -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include/TH" -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include/THC" -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include/THS" -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include/THCS" -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include/THNN" -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include/THCUNN" -DOMPI_SKIP_MPICXX=1 ' '-DCMAKE_CXX_FLAGS= -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include" -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include/TH" -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include/THC" -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include/THS" -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include/THCS" -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include/THNN" -I"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include/THCUNN" -DOMPI_SKIP_MPICXX=1 -std=c++11 ' -DCMAKE_SHARED_LINKER_FLAGS= -DCMAKE_UTILS_PATH=/home/bapvn/work/caffe2-pytorch/cmake/public/utils.cmake -DNUM_JOBS=6
  -- Autodetected CUDA architecture(s): 6.1
  -- Set NVCC_GENCODE for building NCCL: -gencode=arch=compute_61,code=sm_61
  -- Configuring done
  -- Generating done
  -- Build files have been written to: /home/bapvn/work/caffe2-pytorch/third_party/build/nccl
• make install -j6
  [100%] Built target nccl
  Install the project...
  -- Install configuration: "Release"
  -- Up-to-date: /home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/include/nccl.h
• mkdir -p /home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/lib
• find lib -name 'libnccl.so*'
• xargs -I '{}' rsync -lptgoD '{}' /home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/lib/
• popd
  ~/work/caffe2-pytorch/third_party ~/work/caffe2-pytorch/build
• '[' -f ./nccl/nccl/src/nccl.h ']'
• popd
  ~/work/caffe2-pytorch/build
• for arg in '"$@"'
• [[ caffe2 == \n\c\c\l ]]
• [[ caffe2 == \g\l\o\o ]]
• [[ caffe2 == \c\a\f\f\e\2 ]]
• build_caffe2
• [[ -z '' ]]
• EXTRA_CAFFE2_CMAKE_FLAGS=()
• [[ -n '' ]]
• [[ -n /home/bapvn/anaconda3/lib/python3.6/site-packages ]]
• EXTRA_CAFFE2_CMAKE_FLAGS+=("-DCMAKE_PREFIX_PATH=$CMAKE_PREFIX_PATH")
• [[ 1 -eq 1 ]]
• cmake /home/bapvn/work/caffe2-pytorch -DPYTHON_EXECUTABLE=/home/bapvn/anaconda3/bin/python -DPYTHON_LIBRARY=/home/bapvn/anaconda3/lib/libpython3.6m.so.1.0 -DPYTHON_INCLUDE_DIR=/home/bapvn/anaconda3/include/python3.6m -DBUILDING_WITH_TORCH_LIBS=ON -DTORCH_BUILD_VERSION=1.0.0a0+6dd7194 -DCMAKE_BUILD_TYPE=Release -DBUILD_TORCH=ON -DBUILD_PYTHON=ON -DBUILD_SHARED_LIBS=ON -DBUILD_BINARY=OFF -DBUILD_TEST=ON -DINSTALL_TEST=ON -DBUILD_CAFFE2_OPS=ON -DONNX_NAMESPACE=onnx_torch -DUSE_CUDA=1 -DUSE_NUMPY= -DCAFFE2_STATIC_LINK_CUDA=0 -DUSE_ROCM=0 -DUSE_NNPACK=1 -DUSE_LEVELDB=OFF -DUSE_LMDB=OFF -DUSE_OPENCV=OFF -DUSE_FFMPEG=OFF -DUSE_GLOG=OFF -DUSE_GFLAGS=OFF -DUSE_SYSTEM_EIGEN_INSTALL=OFF -DCUDNN_INCLUDE_DIR=/usr/include/ -DCUDNN_LIB_DIR=/usr/lib/x86_64-linux-gnu/ -DCUDNN_LIBRARY=/usr/lib/x86_64-linux-gnu/libcudnn.so.7 -DUSE_MKLDNN=0 -DMKLDNN_INCLUDE_DIR= -DMKLDNN_LIB_DIR= -DMKLDNN_LIBRARY= -DCMAKE_INSTALL_PREFIX=/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install -DCMAKE_C_FLAGS= -DCMAKE_CXX_FLAGS= '-DCMAKE_EXE_LINKER_FLAGS=-L"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/lib" -Wl,-rpath,$ORIGIN ' '-DCMAKE_SHARED_LINKER_FLAGS=-L"/home/bapvn/work/caffe2-pytorch/torch/lib/tmp_install/lib" -Wl,-rpath,$ORIGIN ' -DCMAKE_PREFIX_PATH=/home/bapvn/anaconda3/lib/python3.6/site-packages
  -- std::exception_ptr is supported.
  -- NUMA is available
  -- Current compiler supports avx2 extention. Will build perfkernels.
  -- Building using own protobuf under third_party per request.
  -- Use custom protobuf build.
  -- Caffe2 protobuf include directory: $<BUILD_INTERFACE:/home/bapvn/work/caffe2-pytorch/third_party/protobuf/src>$<INSTALL_INTERFACE:include>
  -- The BLAS backend of choice:MKL
  -- Checking for [mkl_intel_lp64 - mkl_gnu_thread - mkl_core - gomp - pthread - m - dl]
  -- Library mkl_intel_lp64: /home/bapvn/anaconda3/lib/libmkl_intel_lp64.so
  -- Library mkl_gnu_thread: /home/bapvn/anaconda3/lib/libmkl_gnu_thread.so
  -- Library mkl_core: /home/bapvn/anaconda3/lib/libmkl_core.so
  -- Library gomp: -fopenmp
  -- Library pthread: /usr/lib/x86_64-linux-gnu/libpthread.so
  -- Library m: /usr/lib/x86_64-linux-gnu/libm.so
  -- Library dl: /usr/lib/x86_64-linux-gnu/libdl.so
  -- Checking for [mkl_intel_lp64 - mkl_gnu_thread - mkl_core - gomp - pthread - m - dl]
  -- Library mkl_intel_lp64: /home/bapvn/anaconda3/lib/libmkl_intel_lp64.so
  -- Library mkl_gnu_thread: /home/bapvn/anaconda3/lib/libmkl_gnu_thread.so
  -- Library mkl_core: /home/bapvn/anaconda3/lib/libmkl_core.so
  -- Library gomp: -fopenmp
  -- Library pthread: /usr/lib/x86_64-linux-gnu/libpthread.so
  -- Library m: /usr/lib/x86_64-linux-gnu/libm.so
  -- Library dl: /usr/lib/x86_64-linux-gnu/libdl.so
  CMake Warning at cmake/Dependencies.cmake:76 (message):
  MKL could not be found. Defaulting to Eigen
  Call Stack (most recent call first):
  CMakeLists.txt:202 (include)

-- Brace yourself, we are building NNPACK
-- Failed to find LLVM FileCheck
-- git Version: v1.4.0-505be96a
-- Version: 1.4.0
-- Performing Test HAVE_STD_REGEX -- success
-- Performing Test HAVE_GNU_POSIX_REGEX -- failed to compile
-- Performing Test HAVE_POSIX_REGEX -- success
-- Performing Test HAVE_STEADY_CLOCK -- success
-- Found Numa (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libnuma.so)
-- Using third party subdirectory Eigen.
Python 3.6.3 :: Anaconda custom (64-bit)
-- Found PythonInterp: /home/bapvn/anaconda3/bin/python (found suitable version "3.6.3", minimum required is "2.7")
-- NumPy ver. 1.13.3 found (include: /home/bapvn/anaconda3/lib/python3.6/site-packages/numpy/core/include)
-- Found PythonInterp: /home/bapvn/anaconda3/bin/python (found version "3.6.3")
-- System pybind11 found
-- pybind11l include dirs: /home/bapvn/anaconda3/include/python3.6m/home/bapvn/anaconda3/include/python3.6m
-- MPI support found
-- MPI compile flags:
-- MPI include path: /usr/lib/openmpi/include/openmpi/opal/mca/event/libevent2021/libevent/usr/lib/openmpi/include/openmpi/opal/mca/event/libevent2021/libevent/include/usr/lib/openmpi/include/usr/lib/openmpi/include/openmpi
-- MPI LINK flags path: -Wl,-rpath -Wl,/usr/lib/openmpi/lib -Wl,--enable-new-dtags
-- MPI libraries: /usr/lib/openmpi/lib/libmpi_cxx.so/usr/lib/openmpi/lib/libmpi.so
CMake Warning at cmake/Dependencies.cmake:455 (message):
OpenMPI found, but it is not built with CUDA support.
Call Stack (most recent call first):
CMakeLists.txt:202 (include)

-- Caffe2: CUDA detected: 9.0
-- Caffe2: CUDA nvcc is: /usr/local/cuda/bin/nvcc
-- Caffe2: CUDA toolkit directory: /usr/local/cuda
-- Caffe2: Header version is: 9.0
-- Found cuDNN: v7.3.1 (include: /usr/include/, library: /usr/lib/x86_64-linux-gnu/libcudnn.so.7)
CMake Error at cmake/public/cuda.cmake:313 (message):
CUDA 9.0 is not compatible with std::tuple from GCC version >= 6. Please
upgrade to CUDA 9.2 or use the following option to use another version (for
example):

-DCUDA_HOST_COMPILER=/usr/bin/gcc-5

Call Stack (most recent call first):
cmake/Dependencies.cmake:492 (include)
CMakeLists.txt:202 (include)

-- Configuring incomplete, errors occurred!
See also "/home/bapvn/work/caffe2-pytorch/build/CMakeFiles/CMakeOutput.log".
See also "/home/bapvn/work/caffe2-pytorch/build/CMakeFiles/CMakeError.log".
Failed to run 'bash ../tools/build_pytorch_libs.sh --use-cuda --use-nnpack nccl caffe2 libshm gloo c10d THD'

[ghost]

@zou3519 thank for adding the label man.

[mhubii]

Hm I don't know. Maybe try what the compiler is complaining about, run:

export CUDA_HOST_COMPILER=/usr/bin/gcc-5
python setup.py install 

[ghost]

@mhubii yes, I did try it but It did not work.

Thank you for your help.

[mhubii]

What is the output of ls /usr/bin | grep gcc?

[ghost]

@mhubii here is it:
c89-gcc
c99-gcc
gcc
gcc-4.8
gcc-4.9
gcc-5
gcc-6
gcc-ar
gcc-ar-4.8
gcc-ar-4.9
gcc-ar-5
gcc-ar-6
gcc-nm
gcc-nm-4.8
gcc-nm-4.9
gcc-nm-5
gcc-nm-6
gcc-ranlib
gcc-ranlib-4.8
gcc-ranlib-4.9
gcc-ranlib-5
gcc-ranlib-6
x86_64-linux-gnu-gcc
x86_64-linux-gnu-gcc-4.8
x86_64-linux-gnu-gcc-4.9
x86_64-linux-gnu-gcc-5
x86_64-linux-gnu-gcc-6
x86_64-linux-gnu-gcc-ar
x86_64-linux-gnu-gcc-ar-4.8
x86_64-linux-gnu-gcc-ar-4.9
x86_64-linux-gnu-gcc-ar-5
x86_64-linux-gnu-gcc-ar-6
x86_64-linux-gnu-gcc-nm
x86_64-linux-gnu-gcc-nm-4.8
x86_64-linux-gnu-gcc-nm-4.9
x86_64-linux-gnu-gcc-nm-5
x86_64-linux-gnu-gcc-nm-6
x86_64-linux-gnu-gcc-ranlib
x86_64-linux-gnu-gcc-ranlib-4.8
x86_64-linux-gnu-gcc-ranlib-4.9
x86_64-linux-gnu-gcc-ranlib-5
x86_64-linux-gnu-gcc-ranlib-6

Thank you so much for your support!

[mhubii]

Hm I don't know if I can help you there. It seems like you somehow need to compile your code with gcc-5. Maybe try to remove the build folder from your pytorch folder. Change your default compiler and rebuild pytorch.