MKL building troubles on v1.2.0

[kiranshila]

I apologize in advance that this issue seems to be beaten to death, but I believe building Julia with MKL is still rather broken, at least on MacOS.

My am running MacOS Catalina, although it also didn't work on Mavericks.

How to reproduce

Install the latest MKL from Intel - then clone Julia - then checkout v1.2.0

git clone git://github.com/JuliaLang/julia.git
cd julia
git checkout v1.2.0

As per the MKL instructions, I source the environment variables from MKL. On macOS, this install is in the following location.

source /opt/intel/bin/compilervars.sh intel64

Finally, I make the Make.user file and build

echo "USE_INTEL_MKL = 1" >> Make.user
make -j4

So, the build gets reasonably far, but quits here

Got SONAMES of
    PERL base/pcre_h.jl
    CC usr/lib/libccalltest.dylib
    CC usr/lib/libllvmcalltest.dylib
    PERL base/errno_h.jl
    PERL base/build_h.jl.phony
    PERL base/file_constants.jl
    PERL base/uv_constants.jl
    CC src/support/hashing.o
make[1]: *** [/Users/kiranshila/Desktop/Projects.nosync/Julia/julia/usr/lib/julia/libmkl_rt.dylib] Error 1
make[1]: *** Waiting for unfinished jobs....

It seems there is a problem with usr/lib/julia/libmkl_rt.dylib
I look at that directory and it is completely empty

ls usr/lib/julia

How I got it to build

I read in another issue that I need to source a different file, so I tried repeating the above steps with this modification to the call to source

source /opt/intel/mkl/bin/mklvars.sh intel64

But I get the exact same issue

I then tried just symlinking the missing libs directly into usr/lib/julia from a fresh clone using the source command from the instructions

cd ..
rm -rd julia
git clone git://github.com/JuliaLang/julia.git
cd julia
git checkout v1.2.0
source /opt/intel/bin/compilervars.sh intel64
echo "USE_INTEL_MKL = 1" >> Make.user
mkdir -p usr/lib/julia # This didn't exist yet in fresh clone
ln -s /opt/intel/mkl/lib/libmkl_rt.dylib usr/lib/julia/libmkl_rt.dylib
make -j4

This gets past the first roadblock as now that library is visible. Now, after the build steps get to building with Julia, I get this error during Generating precompile statements...

error during bootstrap:
LoadError("sysimg.jl", 16, LoadError("/Users/kiranshila/Desktop/Projects.nosync/Julia/julia/usr/share/julia/stdlib/v1.2/SuiteSparse/src/SuiteSparse.jl", 24, LoadError("/Users/kiranshila/Desktop/Projects.nosync/Julia/julia/usr/share/julia/stdlib/v1.2/SuiteSparse/src/umfpack.jl", 66, ErrorException("error compiling top-level scope: could not load library \"libsuitesparse_wrapper\"\ndlopen(libsuitesparse_wrapper.dylib, 1): Library not loaded: @rpath/libirc.dylib\n  Referenced from: /Users/kiranshila/Desktop/Projects.nosync/Julia/julia/usr/lib/libcholmod.3.0.13.dylib\n  Reason: image not found"))))

Which to me reads that libirc.dylib isn't found. I found it and symlinked it in as before

ln -s /opt/intel/compilers_and_libraries_2019.5.281/mac/compiler/lib/libirc.dylib usr/lib/julia/libirc.dylib

Finally Julia built! Checking the build

julia> versioninfo()
Julia Version 1.2.0
Commit c6da87ff4b (2019-08-20 00:03 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin19.0.0)
  CPU: Intel(R) Core(TM) i7-4750HQ CPU @ 2.00GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, haswell)

julia> LinearAlgebra.versioninfo()
BLAS: libmkl_rt
LAPACK: libmkl_rt

So, it looks like it worked, but that was many hoops to jump through. The issue then is how can we fix the build scripts such that the symlinking isn't broken so others on MacOS don't have to spend a few days digging and troubleshooting.

[ViralBShah]

Did you try using MKL.jl?

[kiranshila]

@ViralBShah I did not as I am interested in building from source.

[ViralBShah]

Yes, but we are hoping to move in the general direction of MKL.jl. You can still build Julia from source, and then use MKL.jl. That said - there is no reason this should break at this point, and it would certainly be nice to fix it.

MKL on macOS is perhaps the least tested configuration.

[kiranshila]

That would indeed be a more portable solution. Thanks Viral for looking into this!

[kiranshila]

It might be beneficial to update the build docs to suggest people try MKL.jl first if that is their sole reason for building from source.

[ViralBShah]

I just tried out MKL.jl on mac, and it seemed to work fine.

[kiranshila]

Yep, just tried and it works for me as well. This seems like the solution.

The same problem occurs however when trying to use Intel's libm, where all the libraries are missing from usr/lib/julia. I think there is a problem with the symlinking in the makefile. The following commands fix those as well.

ln -s /opt/intel/compilers_and_libraries_2019.5.281/mac/compiler/lib/libimf.dylib usr/lib/julia/libimf.dylib
ln -s /opt/intel/compilers_and_libraries_2019.5.281/mac/compiler/lib/libintlc.dylib usr/lib/julia/libintlc.dylib

[ViralBShah]

The use of MKL.jl is documented in the FAQ: https://github.com/JuliaLang/julia/pull/31570/files

[carstenbauer]

I can confirm that the instructions by @kiranshila work for Julia 1.4.0 on macOS Catalina 10.15.4.

However, I'm a bit worried by messages like this during the build process:

ld: warning: directory not found for option '-L/opt/intel/compilers_and_libraries_2020.1.216/mac/mkl/lib/intel64'

The intel64 subfolder doesn't exist in MKL for Mac.

It really is a pity that building with MKL is much more bumpy than on linux. While MKL.jl is great, since I need to compile Julia from source anyways (for Intel LIBM and JIT Flags etc) I'd much prefer integrating MKL directly.

[ViralBShah]

I think this issue probably needs to be filled in mkl.jl.

[KristofferC]

Why? This is about building Julia itself with MKL directly and nothing about using MKL.jl:

While MKL.jl is great, since I need to compile Julia from source anyways (for Intel LIBM and JIT Flags etc) I'd much prefer integrating MKL directly.

[ViralBShah]

The ld warning - I thought it comes from mkl.jl

[kiranshila]

To be honest, even on a fresh copy of mkl and julia on Linux, I can't get Julia+MKL to build. It looks like Intel updated their linker environment variables script, and it doesn't even run on my system. I have found some people's script to fix it, but I gave up a while ago and just use MKL.jl.

I've been recommending all my friends do the same, because fixing all the linker errors is more hassle than its worth.

I think part of the problem stems from Intel moving around and changing their script, but in any case, adding a bit to the build docs to recommend trying MKL.jl first I think would help a lot of people who just want to build Julia to add MKL and potentially don't know about MKL.jl.