add MKL for fluid static and shared library #8887

luotao1 · 2018-03-08T12:46:29Z

add MKL for fluid static and shared library， the structure is:

third_party/install/mklml/
├── include
└── lib

remove PADDLE_USE_ATLAS which is forgot by remove ATLAS library #6530
Examples: https://github.com/luotao1/fluid_inference_example.

Xreki

https://github.com/luotao1/fluid_inference_example/blob/f97cd04e6e51753ea7fbe91aab286585c51466b4/CMakeLists.txt#L14-L16

这个示例里面，我认为链接libpaddle_fluid.so的时候也是需要链接libmklml_intel.so的，我后面会测试一下。

======Update======
我本地测试过了，fluid_inference_example项目make时会出如下错误：

/home/liuyiqun01/.jumbo/opt/gcc48/bin/g++      CMakeFiles/example.dir/example.cc.o  -o example -rdynamic /home/liuyiqun01/PaddlePaddle/Paddle/build_paddle/dist/paddle/fluid/inference/libpaddle_fluid.so /home/liuyiqun01/PaddlePaddle/Paddle/build_paddle/dist/third_party/install/gflags/lib/libgflags.a -lrt -ldl -lpthread -Wl,-rpath,/home/liuyiqun01/PaddlePaddle/Paddle/build_paddle/dist/paddle/fluid/inference 
/home/liuyiqun01/.jumbo/opt/binutils/bin/ld: warning: libiomp5.so, needed by /home/liuyiqun01/PaddlePaddle/Paddle/build_paddle/third_party/install/mklml/lib/libmklml_intel.so, not found (try using -rpath or -rpath-link)
/home/liuyiqun01/PaddlePaddle/Paddle/build_paddle/third_party/install/mklml/lib/libmklml_intel.so: undefined reference to `__kmpc_atomic_float4_max@VERSION'
/home/liuyiqun01/PaddlePaddle/Paddle/build_paddle/third_party/install/mklml/lib/libmklml_intel.so: undefined reference to `__kmpc_master@VERSION'
/home/liuyiqun01/PaddlePaddle/Paddle/build_paddle/third_party/install/mklml/lib/libmklml_intel.so: undefined reference to `__kmpc_single@VERSION'
/home/liuyiqun01/PaddlePaddle/Paddle/build_paddle/third_party/install/mklml/lib/libmklml_intel.so: undefined reference to `__kmpc_end_serialized_parallel@VERSION'
/home/liuyiqun01/PaddlePaddle/Paddle/build_paddle/third_party/install/mklml/lib/libmklml_intel.so: undefined reference to `omp_in_parallel@VERSION'
...

需要链接mkl_intel和iomp5两个动态库。

Xreki · 2018-03-21T12:11:58Z

cmake/inference_lib.cmake

+    copy(mklml_lib
+      SRCS ${MKLML_LIB_DIR} ${MKLML_INC_DIR}
+      DSTS ${dst_dir} ${dst_dir}
+    )


这里应该加if(WITH_MKLML)判断？

~~如果链接了mklml库，貌似还需要链接iomp库？~~看错了，已经全部拷贝过去了。另外问一句，libmklml_gnu.so是不是用不上？

Paddle/cmake/util.cmake

Lines 118 to 120 in 4f9c996

if(WITH_MKLML AND MKLML_LIB_DIR AND MKLML_IOMP_LIB)

target_link_libraries(${TARGET_NAME} "-L${MKLML_LIB_DIR} -liomp5 -Wl,--as-needed")

endif()

这里应该加if(WITH_MKLML)判断

不需要加，if(CBLAS_FOUND)时能正确拷贝MKLML相关的库：
https://paddleci.ngrok.io/viewLog.html?buildId=30302&buildTypeId=Paddle_PrCi&tab=buildLog&_focus=13970#_state=71

libmklml_gnu.so是不是用不上

对的，这个库用不上。所以需要把这个库给去掉么？

需要加if(WITH_MKLML)判断的。如果用户cmake设置WITH_MKL=OFF并且设置了OPEBLAS_ROOT，这里不加判断应该会出错。

所以需要把这个库给去掉么？

我觉得可以去掉，至少make inference_lib_dist的时候不要拷贝过去吧。

Xreki · 2018-03-21T12:17:14Z

cmake/external/mklml.cmake

@@ -46,7 +46,7 @@ INCLUDE_DIRECTORIES(${MKLML_INC_DIR})
 FILE(WRITE ${MKLML_DOWNLOAD_DIR}/CMakeLists.txt
  "PROJECT(MKLML)\n"
  "cmake_minimum_required(VERSION 3.0)\n"
-  "install(DIRECTORY ${MKLML_VER}\n"
+  "install(DIRECTORY ${MKLML_VER}/include ${MKLML_VER}/lib \n"
  "        DESTINATION ${MKLML_DST_DIR})\n")


mklml的依赖是不是不太对？我把third_party和third_party/install下面的mklml删掉了之后，make -j12出现了如下编译错误：

[ 13%] Building NVCC (Device) object paddle/fluid/operators/math/CMakeFiles/selected_rows_functor.dir/selected_rows_functor_generated_selected_rows_functor.cu.o In file included from /home/liuyiqun01/PaddlePaddle/Paddle/paddle/fluid/operators/math/selected_rows_functor.cu:17:0: /home/liuyiqun01/PaddlePaddle/Paddle/paddle/fluid/operators/math/math_function.h:17:23: fatal error: mkl_cblas.h: No such file or directory #include <mkl_cblas.h> ^ compilation terminated. CMake Error at selected_rows_functor_generated_selected_rows_functor.cu.o.cmake:202 (message): Error generating /home/liuyiqun01/PaddlePaddle/Paddle/build_paddle/build/paddle/fluid/operators/math/CMakeFiles/selected_rows_functor.dir//./selected_rows_functor_generated_selected_rows_functor.cu.o make[2]: *** [paddle/fluid/operators/math/CMakeFiles/selected_rows_functor.dir/selected_rows_functor_generated_selected_rows_functor.cu.o] Error 1 make[1]: *** [paddle/fluid/operators/math/CMakeFiles/selected_rows_functor.dir/all] Error 2 make[1]: *** Waiting for unfinished jobs....

======Update======
develop分支下载hang住了，但似乎没出现这个错误。

mklml的依赖是不是不太对

依赖是对的，teamcity上的编译能正常通过。

develop分支下载hang住了

是在厂内测试机上下载mklml的时候hang住了么？厂内测试机的下载会受到网络或SSL因素影响，导致下载失败，可以参考这里进行手动安装。

手动安装就没法正确测试这个依赖了。。。依赖从直观上来看觉得没什么问题，但是想不通为啥会遇到这个错误。。。

手动安装的时候，只是在third_party/mklml中手动下载和解压了mklml.tgz这个包，后面install到third_party/install/mklml的时候还需要使用make命令，所以是能正确测试依赖的。

这个其实不一定，因为没有了下载这个过程，解压的过程很快，很可能因为这个任务很快就完成了，而其他地方即使没有正确地依赖，也不会出错。不过因为cmake我看不出来问题，develop分支也没有遇到这个错误，可能是解决了吧。

luotao1 · 2018-03-26T04:49:41Z

libmklml_gnu.so已删除安装，if(WITH_MKLML)判断已加，fluid_inference_example例子已更新。

Xreki

LGTM

Xreki · 2018-03-26T11:59:11Z

cmake/external/mklml.cmake

@@ -46,7 +46,7 @@ INCLUDE_DIRECTORIES(${MKLML_INC_DIR})
 FILE(WRITE ${MKLML_DOWNLOAD_DIR}/CMakeLists.txt
  "PROJECT(MKLML)\n"
  "cmake_minimum_required(VERSION 3.0)\n"
-  "install(DIRECTORY ${MKLML_VER}\n"
+  "install(DIRECTORY ${MKLML_VER}/include ${MKLML_VER}/lib \n"
  "        DESTINATION ${MKLML_DST_DIR})\n")


这个其实不一定，因为没有了下载这个过程，解压的过程很快，很可能因为这个任务很快就完成了，而其他地方即使没有正确地依赖，也不会出错。不过因为cmake我看不出来问题，develop分支也没有遇到这个错误，可能是解决了吧。

luotao1 added 4 commits March 8, 2018 20:41

add MKL for fluid static and shared library

5030681

remove PADDLE_USE_ATLAS

bc0cfb2

Merge branch 'develop' into infer_mkl

de13f0e

fix mklml install path

e42b8f8

luotao1 requested a review from Xreki March 13, 2018 08:41

luotao1 added the 预测原名Inference，包含Capi预测问题等 label Mar 15, 2018

Xreki reviewed Mar 21, 2018

View reviewed changes

luotao1 added 3 commits March 23, 2018 11:54

Merge branch 'develop' into infer_mkl

ae820a3

Merge branch 'develop' into infer_mkl

6332bd1

rm libmklml_gnu.so

d573195

Xreki approved these changes Mar 26, 2018

View reviewed changes

luotao1 merged commit c858f48 into PaddlePaddle:develop Mar 26, 2018

luotao1 deleted the infer_mkl branch March 26, 2018 12:00

Xreki added this to Basic Usage (DONE) in Inference Framework Apr 3, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add MKL for fluid static and shared library #8887

add MKL for fluid static and shared library #8887

luotao1 commented Mar 8, 2018

Xreki left a comment •

edited

Xreki Mar 21, 2018 •

edited

luotao1 Mar 23, 2018

Xreki Mar 23, 2018 •

edited

Xreki Mar 21, 2018 •

edited

luotao1 Mar 23, 2018

Xreki Mar 23, 2018

luotao1 Mar 26, 2018

Xreki Mar 26, 2018

luotao1 commented Mar 26, 2018

Xreki left a comment

Xreki Mar 26, 2018

	if(WITH_MKLML AND MKLML_LIB_DIR AND MKLML_IOMP_LIB)
	target_link_libraries(${TARGET_NAME} "-L${MKLML_LIB_DIR} -liomp5 -Wl,--as-needed")
	endif()

add MKL for fluid static and shared library #8887

add MKL for fluid static and shared library #8887

Conversation

luotao1 commented Mar 8, 2018

Xreki left a comment • edited

Choose a reason for hiding this comment

Xreki Mar 21, 2018 • edited

Choose a reason for hiding this comment

luotao1 Mar 23, 2018

Choose a reason for hiding this comment

Xreki Mar 23, 2018 • edited

Choose a reason for hiding this comment

Xreki Mar 21, 2018 • edited

Choose a reason for hiding this comment

luotao1 Mar 23, 2018

Choose a reason for hiding this comment

Xreki Mar 23, 2018

Choose a reason for hiding this comment

luotao1 Mar 26, 2018

Choose a reason for hiding this comment

Xreki Mar 26, 2018

Choose a reason for hiding this comment

luotao1 commented Mar 26, 2018

Xreki left a comment

Choose a reason for hiding this comment

Xreki Mar 26, 2018

Choose a reason for hiding this comment

Xreki left a comment •

edited

Xreki Mar 21, 2018 •

edited

Xreki Mar 23, 2018 •

edited

Xreki Mar 21, 2018 •

edited