Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

[MXNET-83] Fix CMake build issue with MKL. #10075

Merged
merged 1 commit into from
Mar 14, 2018

Conversation

KellenSunderland
Copy link
Contributor

Description

Fixes issue #10072
I'd like to do a test run in CI to make sure this doesn't have unintended side effects.

@KellenSunderland KellenSunderland changed the title WIP: Fix default cmake builds Fix CMake build issue with MKL. Mar 12, 2018
@KellenSunderland
Copy link
Contributor Author

Looks like this is building properly in CI and locally for me. @cjolivier01 would you be able to have a look and see if there's any issues with the change?

@cjolivier01
Copy link
Member

Not sure. At this point, I really don't know what's going on with the mkl build, what works with what, does mkldnn use mklml or mkl? can mkldnn operate on its own, what combinations work, etc.

@@ -278,8 +278,6 @@ build_ubuntu_gpu_cmake() {
cmake \
-DUSE_CUDA=1 \
-DUSE_CUDNN=1 \
-DUSE_MKLML_MKL=0 \
-DUSE_MKLDNN=0 \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this one was to explicitely test without MKLDNN.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see the value in explicitly turning off MKL, but what we want to do here is test the default build. I think CI should reflect how users are likely to actually use a project, and I don't think they're likely to explicitly turn off all the settings they're not using. We can certainly do both if you prefer, but in my mind it's more important to ensure basic builds work.

If you feel strongly these should stay in I'll amend the commit, after this patch it should work in either case.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This case is handled by build_ubuntu_gpu_cmake_mkldnn. The idea here is to create a variety of environments and different build configurations.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

build_ubuntu_gpu_cmake_mkldnn explicitly turns on MKL, which is fine because if you're opting into a feature you know what that feature is. My intent here was to test the default behaviour (what a user is likely to do). I.e. what happens when you don't explicitly change any other option other than CUDA.

@pengzhao-intel
Copy link
Contributor

pengzhao-intel commented Mar 13, 2018

Not sure. At this point, I really don't know what's going on with the mkl build, what works with what, does mkldnn use mklml or mkl? can mkldnn operate on its own, what combinations work, etc.

@cjolivier01 MKL-DNN can work standalone (you can build it without MKL/MKLML and it will perform great). MKL-DNN uses MKL/MKLML only to accelerate inner product via gemm from MKL/MKLML. MKLML is a small library, which contains part of MKL functionality (ML related).

update: information from MKL-DNN team for MKLML.

MKLML includes all single precision BLAS functions, a subset of vector math functions, and a subset of LAPACK. The library is created to support self-contained distribution of open source deep learning applications and for this scenario can be used instead of full MKL.

@KellenSunderland
Copy link
Contributor Author

KellenSunderland commented Mar 13, 2018

@pengzhao-intel Would also be great if you could review the PR. Does it look ok from your point of view?

@KellenSunderland KellenSunderland changed the title Fix CMake build issue with MKL. [MXNET-83] Fix CMake build issue with MKL. Mar 13, 2018
@anirudh2290
Copy link
Member

@pengzhao-intel thank you for the explanation. Is this information documented somewhere on the Intel or mxnet docs ?

@cjolivier01
Copy link
Member

I am not sure if the mkl flags in cmake still follow those, but I don't think these changes affect that behavior (allowable overlap of mkl libraries).
At some point, someone will have to check the combinations with cmake configs (possibly renaming config items, changing their descriptions) and make it all work again. Maybe i works now, but I kind of doubt it based upon what I have seen changed before. Maybe I am wrong, however.

@pengzhao-intel
Copy link
Contributor

pengzhao-intel commented Mar 14, 2018

@anirudh2290 it's in the MKL-DNN readme by "Intel MKL small libraries" and I am working on improving the explanations. I will update these descriptions into MXNET doc later.
https://github.com/intel/mkl-dnn/blob/master/README.md#installation

@KellenSunderland I am looking the whole logic. And as @cjolivier01 mentioned, the current building logic is not very clear. The CMake and Makefile flow is not consistency.
So, I am asking @jinhuang415 to write up a proposal to make a clear building approach of MKL/MKL-DNN/MKLML.

@KellenSunderland
Copy link
Contributor Author

@pengzhao-intel great to have a look over the whole system, but what is clear to me given the current setup is:

(Assuming MKL flags left at default values)
1 - MKLDNN will always be found in 3rdparty. This is good as it will always be there.
2 - MKL_FOUND always gets set to true, which is bad because it indicates a full mkl installation is available, which will not be true for most users.

This change won't affect MKLDNN usage, but it makes sure we're only setting MKL_FOUND if the full MKL installation is present.

@pengzhao-intel
Copy link
Contributor

pengzhao-intel commented Mar 14, 2018

@KellenSunderland Agree, this change is good. We can merge it.

@marcoabreu marcoabreu merged commit faa4f9f into apache:master Mar 14, 2018
jinhuang415 pushed a commit to jinhuang415/incubator-mxnet that referenced this pull request Mar 30, 2018
rahul003 pushed a commit to rahul003/mxnet that referenced this pull request Jun 4, 2018
zheng-da pushed a commit to zheng-da/incubator-mxnet that referenced this pull request Jun 28, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants