Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add MKL for fluid static and shared library #8887

Merged
merged 7 commits into from
Mar 26, 2018
Merged

add MKL for fluid static and shared library #8887

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
5030681
add MKL for fluid static and shared library
luotao1 Mar 8, 2018
bc0cfb2
remove PADDLE_USE_ATLAS
luotao1 Mar 8, 2018
de13f0e
Merge branch 'develop' into infer_mkl
luotao1 Mar 13, 2018
e42b8f8
fix mklml install path
luotao1 Mar 13, 2018
ae820a3
Merge branch 'develop' into infer_mkl
luotao1 Mar 23, 2018
6332bd1
Merge branch 'develop' into infer_mkl
luotao1 Mar 26, 2018
d573195
rm libmklml_gnu.so
luotao1 Mar 26, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions cmake/external/mklml.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ SET(MKLML_DOWNLOAD_DIR "${MKLML_SOURCE_DIR}/src/${MKLML_PROJECT}")
SET(MKLML_DST_DIR "mklml")
SET(MKLML_INSTALL_ROOT "${THIRD_PARTY_PATH}/install")
SET(MKLML_INSTALL_DIR ${MKLML_INSTALL_ROOT}/${MKLML_DST_DIR})
SET(MKLML_ROOT ${MKLML_INSTALL_DIR}/${MKLML_VER})
SET(MKLML_ROOT ${MKLML_INSTALL_DIR})
SET(MKLML_INC_DIR ${MKLML_ROOT}/include)
SET(MKLML_LIB_DIR ${MKLML_ROOT}/lib)
SET(MKLML_LIB ${MKLML_LIB_DIR}/libmklml_intel.so)
Expand All @@ -46,7 +46,7 @@ INCLUDE_DIRECTORIES(${MKLML_INC_DIR})
FILE(WRITE ${MKLML_DOWNLOAD_DIR}/CMakeLists.txt
"PROJECT(MKLML)\n"
"cmake_minimum_required(VERSION 3.0)\n"
"install(DIRECTORY ${MKLML_VER}\n"
"install(DIRECTORY ${MKLML_VER}/include ${MKLML_VER}/lib \n"
" DESTINATION ${MKLML_DST_DIR})\n")
Copy link
Contributor

@Xreki Xreki Mar 21, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mklml的依赖是不是不太对？我把third_party和third_party/install下面的mklml删掉了之后，make -j12出现了如下编译错误：

[ 13%] Building NVCC (Device) object paddle/fluid/operators/math/CMakeFiles/selected_rows_functor.dir/selected_rows_functor_generated_selected_rows_functor.cu.o
In file included from /home/liuyiqun01/PaddlePaddle/Paddle/paddle/fluid/operators/math/selected_rows_functor.cu:17:0:
/home/liuyiqun01/PaddlePaddle/Paddle/paddle/fluid/operators/math/math_function.h:17:23: fatal error: mkl_cblas.h: No such file or directory
 #include <mkl_cblas.h>
                       ^
compilation terminated.
CMake Error at selected_rows_functor_generated_selected_rows_functor.cu.o.cmake:202 (message):
  Error generating
  /home/liuyiqun01/PaddlePaddle/Paddle/build_paddle/build/paddle/fluid/operators/math/CMakeFiles/selected_rows_functor.dir//./selected_rows_functor_generated_selected_rows_functor.cu.o


make[2]: *** [paddle/fluid/operators/math/CMakeFiles/selected_rows_functor.dir/selected_rows_functor_generated_selected_rows_functor.cu.o] Error 1
make[1]: *** [paddle/fluid/operators/math/CMakeFiles/selected_rows_functor.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

======Update======
develop分支下载hang住了，但似乎没出现这个错误。

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mklml的依赖是不是不太对

依赖是对的，teamcity上的编译能正常通过。

develop分支下载hang住了

是在厂内测试机上下载mklml的时候hang住了么？厂内测试机的下载会受到网络或SSL因素影响，导致下载失败，可以参考这里进行手动安装。

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

手动安装就没法正确测试这个依赖了。。。依赖从直观上来看觉得没什么问题，但是想不通为啥会遇到这个错误。。。

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

手动安装的时候，只是在third_party/mklml中手动下载和解压了mklml.tgz这个包，后面install到third_party/install/mklml的时候还需要使用make命令，所以是能正确测试依赖的。

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个其实不一定，因为没有了下载这个过程，解压的过程很快，很可能因为这个任务很快就完成了，而其他地方即使没有正确地依赖，也不会出错。不过因为cmake我看不出来问题，develop分支也没有遇到这个错误，可能是解决了吧。


ExternalProject_Add(
Expand Down
6 changes: 6 additions & 0 deletions cmake/inference_lib.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,12 @@ if(NOT CBLAS_FOUND)
SRCS ${CBLAS_INSTALL_DIR}/lib ${CBLAS_INSTALL_DIR}/include
DSTS ${dst_dir} ${dst_dir}
)
elseif (WITH_MKLML)
set(dst_dir "${CMAKE_INSTALL_PREFIX}/third_party/install/mklml")
copy(mklml_lib
SRCS ${MKLML_LIB} ${MKLML_IOMP_LIB} ${MKLML_INC_DIR}
DSTS ${dst_dir}/lib ${dst_dir}/lib ${dst_dir}
)
endif()

# paddle fluid module
Expand Down
7 changes: 0 additions & 7 deletions paddle/fluid/operators/math/math_function.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,6 @@ limitations under the License. */
#include <mkl_vml_functions.h>
#endif

#ifdef PADDLE_USE_ATLAS
extern "C" {
#include <cblas.h>
#include <clapack.h>
}
#endif

#ifdef PADDLE_USE_OPENBLAS
#include <cblas.h>
#include <lapacke.h>
Expand Down
15 changes: 4 additions & 11 deletions paddle/math/MathFunctions.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,17 +59,10 @@ void* lapack_dso_handle = nullptr;
} __name; // struct DynLoad__##__name
#endif

#ifdef PADDLE_USE_ATLAS
#define PADDLE_SGETRF clapack_sgetrf
#define PADDLE_DGETRF clapack_dgetrf
#define PADDLE_SGETRI clapack_sgetri
#define PADDLE_DGETRI clapack_dgetri
#else
#define PADDLE_SGETRF LAPACKE_sgetrf
#define PADDLE_DGETRF LAPACKE_dgetrf
#define PADDLE_SGETRI LAPACKE_sgetri
#define PADDLE_DGETRI LAPACKE_dgetri
#endif
#define PADDLE_SGETRF LAPACKE_sgetrf
#define PADDLE_DGETRF LAPACKE_dgetrf
#define PADDLE_SGETRI LAPACKE_sgetri
#define PADDLE_DGETRI LAPACKE_dgetri

#define LAPACK_ROUTINE_EACH(__macro) \
__macro(PADDLE_SGETRF) \
Expand Down
2 changes: 1 addition & 1 deletion paddle/math/MathFunctions.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ limitations under the License. */
#include <mkl_vml_functions.h>
#endif

#if defined(PADDLE_USE_ATLAS) || defined(PADDLE_USE_VECLIB)
#if defined(PADDLE_USE_VECLIB)
extern "C" {
#include <cblas.h>
#include <clapack.h>
Expand Down