Building MKL.jl fails in v1.6.0

[mihalybaci]

I am running into errors building MKL.jl in Julia v1.6.0-beta1, and I have confirmed that it builds fine with Julia v1.5.3 ( no errors and BLAS.vendor() returns :mkl. The full message error I get is:

(@v1.6) pkg> build MKL
    Building MKL → `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/e01175ff53c062c52b6cbe441a4382e132f7c82e/build.log`
ERROR: Error building `MKL`, showing the last 100 of log: 
best_platform = select_platform(Dict(p => triplet(p) for p in platforms))
dlopen("libjulia", RTLD_LAZY | RTLD_DEEPBIND)

'`, ProcessExited(1)) [1]

Stacktrace:
  [1] pipeline_error
    @ ./process.jl:525 [inlined]
  [2] run(::Cmd; wait::Bool)
    @ Base ./process.jl:440
  [3] run
    @ ./process.jl:438 [inlined]
  [4] (::Main.anonymous.var"#1#5"{Set{String}, String})(prec_path::String)
    @ Main.anonymous /tmp/jl_Z4lpKw:212
  [5] mktempdir(fn::Main.anonymous.var"#1#5"{Set{String}, String}, parent::String; prefix::String)
    @ Base.Filesystem ./file.jl:729
  [6] mktempdir
    @ ./file.jl:727 [inlined]
  [7] generate_precompile_statements()
    @ Main.anonymous /tmp/jl_Z4lpKw:195
  [8] top-level scope
    @ /tmp/jl_Z4lpKw:339
  [9] eval(m::Module, e::Any)
    @ Core ./boot.jl:360
 [10] top-level scope
    @ /tmp/jl_Z4lpKw:6
 [11] include(fname::String)
    @ Base.MainInclude ./client.jl:444
 [12] top-level scope
    @ none:33
in expression starting at /tmp/jl_Z4lpKw:3
[ Info: Splicing in code to load MKL in /home/michael/.local/julia-1.6.0-beta1/bin/../share/julia/base/sysimg.jl
[ Info: getting precompile script from: https://raw.githubusercontent.com/JuliaLang/julia/release-1.6/contrib/generate_precompile.jl
Precompiling project...
  ✓ MKL
1 dependency successfully precompiled in 1 seconds (7 already precompiled)
[ Info: PackageCompiler: creating base system image (incremental=false)...
[ Info: PackageCompiler: creating system image object file, this might take a while...
ERROR: LoadError: failed process: Process(`/home/michael/.local/julia-1.6.0-beta1/bin/julia --color=yes --startup-file=no --cpu-target=native --sysimage=/tmp/jl_jPBxgE/sys.ji --project=/tmp/jl_BwAXby --output-o=/tmp/jl_tTeUfB.o -e 'Base.reinit_stdio()
@eval Sys BINDIR = ccall(:jl_get_julia_bindir, Any, ())::String
@eval Sys STDLIB = "/home/michael/.local/julia-1.6.0-beta1/share/julia/stdlib/v1.6"
Base.init_load_path()
if isdefined(Base, :init_active_project)
    Base.init_active_project()
end
Base.init_depot_path()
# This @eval prevents symbols from being put into Main
@eval Module() begin
    PrecompileStagingArea = Module()
    for (_pkgid, _mod) in Base.loaded_modules
        if !(_pkgid.name in ("Main", "Core", "Base"))
            eval(PrecompileStagingArea, :(const $(Symbol(_mod)) = $_mod))
        end
    end
    precompile_files = String[
        "/tmp/jl_b29gfy"
    ]
    for file in precompile_files, statement in eachline(file)
        # println(statement)
        # The compiler has problem caching signatures with \`Vararg{?, N}\`. Replacing
        # N with a large number seems to work around it.
        statement = replace(statement, r"Vararg{(.*?), N} where N" => s"Vararg{\1, 100}")
        try
            Base.include_string(PrecompileStagingArea, statement)
        catch
            # See julia issue #28808
            @debug "failed to execute $statement"
        end
    end
end # module
include("/tmp/jl_Z4lpKw")
empty!(LOAD_PATH)
empty!(DEPOT_PATH)
'`, ProcessExited(1)) [1]

Stacktrace:
  [1] pipeline_error
    @ ./process.jl:525 [inlined]
  [2] run(::Cmd; wait::Bool)
    @ Base ./process.jl:440
  [3] run
    @ ./process.jl:438 [inlined]
  [4] create_sysimg_object_file(object_file::String, packages::Vector{String}; project::String, base_sysimage::String, precompile_execution_file::Vector{String}, precompile_statements_file::Vector{String}, cpu_target::String, script::String, isapp::Bool)
    @ PackageCompiler ~/.julia/packages/PackageCompiler/3BsME/src/PackageCompiler.jl:306
  [5] create_sysimage(packages::Vector{Symbol}; sysimage_path::Nothing, project::String, precompile_execution_file::Vector{String}, precompile_statements_file::Vector{String}, incremental::Bool, filter_stdlibs::Bool, replace_default::Bool, cpu_target::String, script::String, base_sysimage::Nothing, isapp::Bool)
    @ PackageCompiler ~/.julia/packages/PackageCompiler/3BsME/src/PackageCompiler.jl:442
  [6] top-level scope
    @ ~/.julia/packages/MKL/1aXoN/src/install.jl:128
  [7] eval
    @ ./boot.jl:360 [inlined]
  [8] change_blas_library(libblas::String)
    @ Main ~/.julia/packages/MKL/1aXoN/src/install.jl:126
  [9] enable_mkl_startup()
    @ Main ~/.julia/packages/MKL/1aXoN/src/install.jl:103
 [10] top-level scope
    @ ~/.julia/packages/MKL/1aXoN/deps/build.jl:9
 [11] include(fname::String)
    @ Base.MainInclude ./client.jl:444
 [12] top-level scope
in expression starting at /home/michael/.julia/packages/MKL/1aXoN/deps/build.jl:9

My versioninfo() is:

Julia Version 1.6.0-beta1
Commit b84990e1ac (2021-01-08 12:42 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.0 (ORCJIT, haswell)
Environment:
  JULIA_NUM_THREADS = 8

Let me know if there is any other information you need.

[mtfishman]

I get a similar error with:

julia> versioninfo()
Julia Version 1.6.0-beta1
Commit b84990e1ac (2021-01-08 12:42 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Xeon(R) E-2176M  CPU @ 2.70GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.0 (ORCJIT, skylake)

[pablosanjose]

I confirm the same error arises with

julia> versioninfo()
Julia Version 1.7.0-DEV.320
Commit 7647ab574f (2021-01-15 22:52 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin18.7.0)
  CPU: Intel(R) Core(TM) i7-8559U CPU @ 2.70GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.0 (ORCJIT, skylake)
Environment:
  JULIA_EDITOR = code

(I'm using macos Big Sur 11.1. Before arriving at this error I get a permission error when trying to write to the share folder inside the Julia-1.7.app package, where the sysimg.jl sits. That is easily overcome by giving my user write permission to the share folder)

[ViralBShah]

Is this in /Applications? I usually always install julia in my home directory - where I suppose I have permissions to write and hence didn't run into this issue.

[pablosanjose]

The problem is writing inside the .app package. It doesn't matter whether I install it in /Applications or /Users/Shared. I still have to enable write permissions to my user for the files inside the package.

EDIT: I believe the issue happens in Big Sur, but I think it didn't happen in Mojave. I certainly did not encounter it until I upgraded.

[pablosanjose]

Friendly bump

[ViralBShah]

What if you install the app in your own home directory? Not really an MKL.jl issue, I suppose.

[pablosanjose]

Nope, same error. I also tried building MKL.jl in a julia build from source, not packagaged into an app. Same issue.

[mihalybaci]

I just downloaded the julia-1.6.0-rc1 binaries, and MKL.jl still fails to build. The first two lines in the log are:

ERROR: ArgumentError: Package Artifacts not found in current path:
- Run `import Pkg; Pkg.add("Artifacts")` to install the Artifacts package.

The rest of the log is the stacktrace for that error. I checked Pkg and everything appears normal

(@v1.6) pkg> st
      Status `~/.julia/environments/v1.6/Project.toml`
  [33e6dc65] MKL v0.3.0
  [56f22d72] Artifacts

I am not sure why it can't find the package. Everything is in local, non-root directories (and this is my home computer so I do have root access in any case).

[dmbates]

I am encountering the same problem under Linux (Ubuntu 20.10) with v1.6.0-rc1 and with v1.7.0-DEV.593. Could the problem be related to the Artifacts package being a stdlib in v1.6+ and not a submodule of Pkg?

[ViralBShah]

@dmbates 1.7 is going to make this really easy once JuliaLang/julia#39455 is merged. Will no longer need to be built into the system image. We naturally need to fix the issue in 1.6, and perhaps @KristofferC may know what's going on.

[dmbates]

The relevant section in sysimg.jl after the insertion of the MKL loading code is, in my case starting at line 88

        Base._track_dependencies[] = true
# START MKL INSERT
pushfirst!(LOAD_PATH, "@v#.#")
pushfirst!(LOAD_PATH, "@")
MKL = Base.require(Base, :MKL)
MKL.MKL_jll.__init__()
MKL.__init__()
popfirst!(LOAD_PATH)
popfirst!(LOAD_PATH)
# END MKL INSERT
        tot_time_stdlib = @elapsed for stdlib in stdlibs
            tt = @elapsed Base.require(Base, stdlib)
            print_time(stdlib, tt)
        end

So MKL is going to be loaded before the stdlibs, which is necessary so that LinearAlgebra already has MKL loaded. However, MKL will require MKL_jll which will require Artifacts to ensure that the dll has been installed. Is there a sequencing problem here because of that?

[dmbates]

That's great news, @ViralBShah . I look forward to working with it under v1.7 and, I hope, also under v1.6.

The reason I am excited about it is I recently bought a laptop that has a 11th gen Intel core processor which supports avx512. In my typical benchmark for the MixedModels package MKL would give maybe a 30% speed boost relative to OpenBLAS on older processors. On this processor the benchmark evaluation of an objective went from 9.6 ms to 3.2 ms when using MKL. At first I thought it was a mistake and couldn't possibly be that fast but it is.

[ViralBShah]

Can you try 1.7 master? We were quite behind on openblas versions (due to bugs in openblas), but yesterday, we merged the move to 0.3.13 on 1.7 master. @chriselrod may also be interested in understanding the performance issues and if any of his packages could help.