[Caffe2/Bug] Cannot enable MKL-DNN

[jiecaoyu]

Solution

A solution is found for the problem, but maybe a bug in Caffe2

MKL-DNN can be enabled by first compiling the PyTorch. Then turn on MKL by

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 827121b1f..235d00bd6 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -111,6 +111,7 @@ option(USE_SYSTEM_EIGEN_INSTALL
 option(USE_TENSORRT "Using Nvidia TensorRT library" OFF)
 option(USE_ZMQ "Use ZMQ" OFF)
 option(USE_ZSTD "Use ZSTD" OFF)
+option(USE_MKL "Use MKL" ON)
 option(USE_MKLDNN "Use MKLDNN" OFF)
 option(USE_IDEEP "Use IDEEP interface in MKL BLAS" ON)
 option(USE_MKLML "Use MKLML interface in MKL BLAS" ON)

and recompile PyTorch. Then MKL-DNN is enabled.

Issue description

Trying to accelerate caffe2 inference with MKL-DNN.

The MKL-DNN lib can be detected:

-- Found MKLDNN: /home/jiecaoyu/.local/include  
-- Found MKLDNN      (include: /home/jiecaoyu/.local/include, library: /home/jiecaoyu/.local/lib/libmkldnn.so)

But mkl operators are not compiled

-- Excluding mkl operators as we are not using mkl

And MKL-DNN cannot be found after installation since

from caffe2.python import workspace
workspace.C.has_mkldnn

returns False.

Also, I try to enable MKL by changing the CMakeLists.txt:

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 827121b1f..235d00bd6 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -111,6 +111,7 @@ option(USE_SYSTEM_EIGEN_INSTALL
 option(USE_TENSORRT "Using Nvidia TensorRT library" OFF)
 option(USE_ZMQ "Use ZMQ" OFF)
 option(USE_ZSTD "Use ZSTD" OFF)
+option(USE_MKL "Use MKL" ON)
 option(USE_MKLDNN "Use MKLDNN" OFF)
 option(USE_IDEEP "Use IDEEP interface in MKL BLAS" ON)
 option(USE_MKLML "Use MKLML interface in MKL BLAS" ON)

It acutally cause an error:

...
/home/jiecaoyu/LIBS/pytorch/aten/src/TH/THAllocator.cpp:397:1: error: prototype for ‘THMapAllocator::THMapAllocator(WithFd, const char*, int, int)’ does not match any in class ‘THMapAllocator’
 THMapAllocator::THMapAllocator(WithFd, const char *filename, int fd, int flags) {
 ^~~~~~~~~~~~~~
In file included from /home/jiecaoyu/LIBS/pytorch/aten/src/TH/THAllocator.cpp:1:0:
/home/jiecaoyu/LIBS/pytorch/aten/src/TH/THAllocator.h:41:3: error: candidates are: THMapAllocator::THMapAllocator(THMapAllocator&&)
   THMapAllocator(THMapAllocator&&) = delete;
   ^~~~~~~~~~~~~~
/home/jiecaoyu/LIBS/pytorch/aten/src/TH/THAllocator.h:39:3: error:                 THMapAllocator::THMapAllocator(const THMapAllocator&)
   THMapAllocator(const THMapAllocator&) = delete;
   ^~~~~~~~~~~~~~
/home/jiecaoyu/LIBS/pytorch/aten/src/TH/THAllocator.h:38:3: error:                 THMapAllocator::THMapAllocator(WithFd, const char*, int, int, size_t)
   THMapAllocator(WithFd, const char *filename, int fd, int flags, size_t size);
   ^~~~~~~~~~~~~~
/home/jiecaoyu/LIBS/pytorch/aten/src/TH/THAllocator.cpp:393:1: error:                 THMapAllocator::THMapAllocator(const char*, int, size_t)
 THMapAllocator::THMapAllocator(const char *filename, int flags, size_t size) {
 ^~~~~~~~~~~~~~
/home/jiecaoyu/LIBS/pytorch/aten/src/TH/THAllocator.cpp:401:52: error: destructors may not have parameters
 THMapAllocator::~THMapAllocator(THMapAllocator* ctx) {}
                                                    ^
/home/jiecaoyu/LIBS/pytorch/aten/src/TH/THAllocator.cpp:401:1: error: redefinition of ‘THMapAllocator::~THMapAllocator()’
 THMapAllocator::~THMapAllocator(THMapAllocator* ctx) {}
 ^~~~~~~~~~~~~~
In file included from /home/jiecaoyu/LIBS/pytorch/aten/src/TH/THAllocator.cpp:1:0:
...

System Info

PyTorch version: 1.0.0a0+98aebed
Is debug build: No
CUDA used to build PyTorch: None

OS: Ubuntu 18.04.1 LTS
GCC version: (Ubuntu 7.3.0-16ubuntu3) 7.3.0
CMake version: version 3.10.2

Python version: 2.7
Is CUDA available: No
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA

Versions of relevant libraries:
[pip] numpy (1.15.1)
[pip] torch (1.0.0a0+98aebed)
[conda] Could not collect

[deepali-c]

Please check with -DBLAS="MKL".

[jiecaoyu]

Hi @deepali-c , Thanks for the response!

-DBLAS="MKL" seems does not change the compilation.

I add this option in the caffe2 build instruction:

diff --git a/tools/build_pytorch_libs.sh b/tools/build_pytorch_libs.sh
index 37d816775..65e639359 100755
--- a/tools/build_pytorch_libs.sh
+++ b/tools/build_pytorch_libs.sh
@@ -300,6 +300,7 @@ function build_caffe2() {
                       -DCUDNN_INCLUDE_DIR=$CUDNN_INCLUDE_DIR \
                       -DCUDNN_LIB_DIR=$CUDNN_LIB_DIR \
                       -DCUDNN_LIBRARY=$CUDNN_LIBRARY \
+                      -DBLAS="MKL" \
                       -DUSE_MKLDNN=$USE_MKLDNN \
                       -DMKLDNN_INCLUDE_DIR=$MKLDNN_INCLUDE_DIR \
                       -DMKLDNN_LIB_DIR=$MKLDNN_LIB_DIR \

[jiecaoyu]

Hi @deepali-c I have solved the problem, but it might be a bug in Caffe2.

[jiapei100]

I guess the problem is as follows, resided in Caffe2 ...

 CMake Warning at caffe2/CMakeLists.txt:198 (add_library):
   Cannot generate a safe runtime search path for target caffe2 because there
   is a cycle in the constraint graph:

     dir 0 is [/usr/local/cuda/lib64]
     dir 1 is [/usr/local/lib]
       dir 4 must precede it due to runtime library [libmkldnn.so.0]
     dir 2 is [/usr/lib/x86_64-linux-gnu/openmpi/lib]
     dir 3 is [/opt/intel/mkl/lib/intel64]
     dir 4 is [....../pytorch/build/lib]
       dir 1 must precede it due to runtime library [libmkldnn.so.0]

   Some of these libraries may not be found correctly.

[jiecaoyu]

@jiapei100 Truly, I think some contributors are working on solving this problem, e.g., this issue. I am trying to do some performance profiling of MKL-DNN so I am now writing my own code based on MKL-DNN directly.