fail to find MKL #17791

hubutui · 2020-03-09T05:47:10Z

Description

could not find MKL from #17751

Error Message

-- The C compiler identification is GNU 9.2.1
-- The CXX compiler identification is GNU 9.2.1
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- CMAKE_CROSSCOMPILING FALSE
-- CMAKE_HOST_SYSTEM_PROCESSOR x86_64
-- CMAKE_SYSTEM_PROCESSOR x86_64
-- CMAKE_SYSTEM_NAME Linux
-- CMake version '3.16.5' using generator 'Unix Makefiles'
-- Could not find CCache. Consider installing CCache to speed up compilation.
-- Performing Test SUPPORT_CXX11
-- Performing Test SUPPORT_CXX11 - Success
-- Performing Test SUPPORT_CXX0X
-- Performing Test SUPPORT_CXX0X - Success
-- CMAKE_BUILD_TYPE is unset, defaulting to Release
-- MKL-DNN compat: set DNNL_BUILD_EXAMPLES to MKLDNN_BUILD_EXAMPLES with value `OFF`
-- MKL-DNN compat: set DNNL_BUILD_TESTS to MKLDNN_BUILD_TESTS with value `OFF`
-- MKL-DNN compat: set DNNL_ENABLE_JIT_PROFILING to MKLDNN_ENABLE_JIT_PROFILING with value `OFF`
-- MKL-DNN compat: set DNNL_LIBRARY_TYPE to MKLDNN_LIBRARY_TYPE with value `STATIC`
-- MKL-DNN compat: set DNNL_ARCH_OPT_FLAGS to MKLDNN_ARCH_OPT_FLAGS with value ``
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Found OpenMP_C: -fopenmp (found version "4.5") 
-- Found OpenMP_CXX: -fopenmp (found version "4.5") 
-- Found OpenMP: TRUE (found version "4.5")  
-- GPU support is disabled
-- Found Doxygen: /usr/bin/doxygen (found version "1.8.17") found components: doxygen dot 
-- Found Git: /usr/bin/git (found version "2.25.1") 
-- Intel(R) VTune(TM) Amplifier JIT profiling disabled
CMake Error at /usr/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:146 (message):
  Could NOT find MKL (missing: MKL_INTEL_LP64_LIBRARY
  MKL_INTEL_THREAD_LIBRARY MKL_CORE_LIBRARY IOMP_LIBRARY)
Call Stack (most recent call first):
  /usr/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:393 (_FPHSA_FAILURE_MESSAGE)
  cmake/Modules/FindMKL.cmake:156 (find_package_handle_standard_args)
  cmake/ChooseBlas.cmake:49 (find_package)
  CMakeLists.txt:310 (include)


-- Configuring incomplete, errors occurred!
See also "/home/USER/Projects/archlinuxcn-repo/archlinuxcn/mxnet-git/src-mk/build/CMakeFiles/CMakeOutput.log".
See also "/home/USER/Projects/archlinuxcn-repo/archlinuxcn/mxnet-git/src-mk/build/CMakeFiles/CMakeError.log".

To Reproduce

git clone https://github.com/apache/incubator-mxnet.git
git checkout 636897718f8468be0dd5c4077258eb14c7416717 -b pkg
git submodule update --init --recursive
cd incubator-mxnet
mkdir build
cd build
cmake .. -DUSE_CUDA=0 -DUSE_MKLDNN=1 -DUSE_MKL_IF_AVAILABLE=1 -DUSE_BLAS=MKL

What have you tried to solve it?

I try commit 5cffa74 with no errors.

Environment

I'm using ArchLinux with gcc 9, intel-mkl. intel-mkl is installed to /opt/intel.

The text was updated successfully, but these errors were encountered:

pengzhao-intel · 2020-03-09T06:13:45Z

@wuxun-zhang could you take a look?

wuxun-zhang · 2020-03-09T06:38:24Z

Sure. I will look into this issue.

TaoLv · 2020-03-09T09:27:37Z

cc @leezu

wuxun-zhang · 2020-03-09T10:15:04Z

Actually I cannot reproduce this issue with cmake 3.16/gcc 7.

 ~/softwares/cmake_3.16/bin/cmake .. -DUSE_CUDA=0 -DUSE_MKLDNN=1 -DUSE_MKL_IF_AVAILABLE=1 -DUSE_BLAS=MKL                                             [636897718] 
-- The C compiler identification is GNU 7.2.1
-- The CXX compiler identification is GNU 7.2.1
-- Check for working C compiler: /opt/rh/devtoolset-7/root/usr/bin/cc
-- Check for working C compiler: /opt/rh/devtoolset-7/root/usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /opt/rh/devtoolset-7/root/usr/bin/c++
-- Check for working CXX compiler: /opt/rh/devtoolset-7/root/usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- CMAKE_CROSSCOMPILING FALSE
-- CMAKE_HOST_SYSTEM_PROCESSOR x86_64
-- CMAKE_SYSTEM_PROCESSOR x86_64
-- CMAKE_SYSTEM_NAME Linux
-- CMake version '3.16.5' using generator 'Unix Makefiles'
-- Could not find CCache. Consider installing CCache to speed up compilation.
-- Performing Test SUPPORT_CXX11
-- Performing Test SUPPORT_CXX11 - Success
-- Performing Test SUPPORT_CXX0X
-- Performing Test SUPPORT_CXX0X - Success
-- CMAKE_BUILD_TYPE is unset, defaulting to Release
-- MKL-DNN compat: set DNNL_BUILD_EXAMPLES to MKLDNN_BUILD_EXAMPLES with value `OFF`
-- MKL-DNN compat: set DNNL_BUILD_TESTS to MKLDNN_BUILD_TESTS with value `OFF`
-- MKL-DNN compat: set DNNL_ENABLE_JIT_PROFILING to MKLDNN_ENABLE_JIT_PROFILING with value `OFF`
-- MKL-DNN compat: set DNNL_LIBRARY_TYPE to MKLDNN_LIBRARY_TYPE with value `STATIC`
-- MKL-DNN compat: set DNNL_ARCH_OPT_FLAGS to MKLDNN_ARCH_OPT_FLAGS with value ``
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Found OpenMP_C: -fopenmp (found version "4.5") 
-- Found OpenMP_CXX: -fopenmp (found version "4.5") 
-- Found OpenMP: TRUE (found version "4.5")  
-- GPU support is disabled
-- Found Doxygen: /usr/bin/doxygen (found version "1.8.5") found components: doxygen dot 
-- Found Git: /home/wuxunzha/anaconda3/bin/git (found version "2.23.0") 
-- Intel(R) VTune(TM) Amplifier JIT profiling disabled
-- Found MKL: /opt/intel/mkl/include  
-- Found MKL (include: /opt/intel/mkl/include, lib: -Wl,--start-group;/opt/intel/mkl/lib/intel64/libmkl_intel_lp64.a;/opt/intel/mkl/lib/intel64/libmkl_intel_thread.a;/opt/intel/mkl/lib/intel64/libmkl_core.a;/opt/intel/lib/intel64/libiomp5.so;-Wl,--end-group
-- OpenCV 3.1.0 found (/home/wuxunzha/anaconda3/share/OpenCV)
--  OpenCV_LIBS=opencv_core;opencv_highgui;opencv_imgproc;opencv_imgcodecs
-- Found OpenMP_C: -fopenmp  
-- Found OpenMP_CXX: -fopenmp  
-- Found OpenMP: TRUE

hubutui · 2020-03-09T10:28:03Z

cmake 3.16.5, gcc 9.2.1, intel-mkl 2020.0.166 are available in ArchLinux, could you try in a docker container?

TaoLv · 2020-03-09T12:31:12Z

@hubutui Could you please share the directory structure of MKL and what are installed in the mkl/lib/intel64 folder?

sl1pkn07 · 2020-03-09T13:56:08Z

you can found it here https://www.archlinux.org/packages/community/x86_64/intel-mkl/files/ and here https://www.archlinux.org/packages/community/x86_64/intel-dnnl/files/

TaoLv · 2020-03-09T15:17:19Z

@sl1pkn07 Thanks! Seems static libraries are not released in the package. I don't see the files below which I think are needed for the changes in PR #17751.

libmkl_core.a
libmkl_gnu_thread.a
libmkl_intel_thread.a
libmkl_intel_lp64.a
libmkl_intel_ilp64.a
libmkl_sequential.a

leezu · 2020-03-09T16:25:00Z

@TaoLv the motivation of #17751 is to prevent the situation where MKLDNN dlopens intel omp after another omp has already been opened by the dynamic linker. Static MKL is not needed for that, so let's disable MKL_USE_STATIC_LIBS by default again.

TaoLv · 2020-03-10T01:00:54Z

@hubutui @sl1pkn07 Who is releasing the package for ArchLinux?

sl1pkn07 · 2020-03-10T01:03:18Z

official archlinux member

@svenstaro

TaoLv · 2020-03-10T01:32:06Z

Hope the package on ArchLinux can include those missing files so we can still use static linking.

sl1pkn07 · 2020-03-10T01:35:01Z

normally, archlinux always avoid ship static libs if exist a shared counterpart

this case is not a exception

TaoLv · 2020-03-10T01:42:09Z

Then one may need to turn MKL_USE_STATIC_LIBS off explicitly in the cmake command line.

svenstaro · 2020-03-10T11:04:40Z

I'll consider adding the static libs as some projects using mkl seem to expect it but I'm 70% 'no' on this as the dynamic libs fulfill the same purpose while being more space efficient. Make a bug report on our tracker with a good case and I'll consider it. For now I'm not convinced.

…

On Tue, Mar 10, 2020, 08:42 Tao Lv ***@***.***> wrote: Then one may need to turn MKL_USE_STATIC_LIBS off explicitly in the cmake command line. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#17791>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAANACQII36NYBEWSZZHB3RGWLHFANCNFSM4LEBOGAQ> .

svenstaro · 2020-07-04T09:50:39Z

Just to follow up on this, our intel-mkl package now has static libs so consider retesting/closing this.

TaoLv · 2020-07-28T00:33:13Z

I'm closing this issue as the target platform changed to provide static libraries of MKL. @sl1pkn07 please feel free to re-open the issue if you have more questions. Please also note that there is an open PR (#17794 ) to disable the static linking.

hubutui added the Bug label Mar 9, 2020

pengzhao-intel added this to To do in CPU Performance and Quantization via automation Mar 9, 2020

pengzhao-intel added the MKL label Mar 9, 2020

leezu mentioned this issue Mar 9, 2020

Disable MKL_USE_STATIC_LIBS by default #17794

Open

szha assigned TaoLv Jul 25, 2020

TaoLv closed this as completed Jul 28, 2020

CPU Performance and Quantization automation moved this from To do to Done Jul 28, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fail to find MKL #17791

fail to find MKL #17791

hubutui commented Mar 9, 2020 •

edited

Loading

pengzhao-intel commented Mar 9, 2020

wuxun-zhang commented Mar 9, 2020

TaoLv commented Mar 9, 2020

wuxun-zhang commented Mar 9, 2020

hubutui commented Mar 9, 2020

TaoLv commented Mar 9, 2020

sl1pkn07 commented Mar 9, 2020 •

edited

Loading

TaoLv commented Mar 9, 2020

leezu commented Mar 9, 2020

TaoLv commented Mar 10, 2020

sl1pkn07 commented Mar 10, 2020 •

edited

Loading

TaoLv commented Mar 10, 2020

sl1pkn07 commented Mar 10, 2020

TaoLv commented Mar 10, 2020

svenstaro commented Mar 10, 2020 via email

svenstaro commented Jul 4, 2020

TaoLv commented Jul 28, 2020

fail to find MKL #17791

fail to find MKL #17791

Comments

hubutui commented Mar 9, 2020 • edited Loading

Description

Error Message

To Reproduce

What have you tried to solve it?

Environment

pengzhao-intel commented Mar 9, 2020

wuxun-zhang commented Mar 9, 2020

TaoLv commented Mar 9, 2020

wuxun-zhang commented Mar 9, 2020

hubutui commented Mar 9, 2020

TaoLv commented Mar 9, 2020

sl1pkn07 commented Mar 9, 2020 • edited Loading

TaoLv commented Mar 9, 2020

leezu commented Mar 9, 2020

TaoLv commented Mar 10, 2020

sl1pkn07 commented Mar 10, 2020 • edited Loading

TaoLv commented Mar 10, 2020

sl1pkn07 commented Mar 10, 2020

TaoLv commented Mar 10, 2020

svenstaro commented Mar 10, 2020 via email

svenstaro commented Jul 4, 2020

TaoLv commented Jul 28, 2020

hubutui commented Mar 9, 2020 •

edited

Loading

sl1pkn07 commented Mar 9, 2020 •

edited

Loading

sl1pkn07 commented Mar 10, 2020 •

edited

Loading