Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MKL isn't built on Windows #24

Closed
aminya opened this issue Dec 31, 2019 · 17 comments · Fixed by #29
Closed

MKL isn't built on Windows #24

aminya opened this issue Dec 31, 2019 · 17 comments · Fixed by #29

Comments

@aminya
Copy link
Contributor

aminya commented Dec 31, 2019

CI on my V0.1 branch:
https://travis-ci.com/aminya/MKL.jl/builds/142809478
https://ci.appveyor.com/project/andreasnoack/mkl-jl/builds/29837534

Travis on my patch-3 branch (before REPL-Startup fix) https://travis-ci.com/aminya/MKL.jl/builds/142824221

This shows that the problem is something external like PackageCompiler because I didn't have any issue building patch-3 at that time.

Local build spoiler:
(v1.3) pkg> add https://github.com/JuliaComputing/MKL.jl
  Updating git-repo `https://github.com/JuliaComputing/MKL.jl`
  Updating git-repo `https://github.com/JuliaComputing/MKL.jl`
 Resolving package versions...
  Updating `C:\Users\yahyaaba\.julia\environments\v1.3\Project.toml`
  [33e6dc65] + MKL v0.1.0 #master (https://github.com/JuliaComputing/MKL.jl)
  Updating `C:\Users\yahyaaba\.julia\environments\v1.3\Manifest.toml`
  [85c772de] + AbstractNumbers v0.2.0
  [537997a7]  AbstractPlotting v0.9.10  v0.9.16
  [39de3d68] - AxisArrays v0.3.3
  [e1450e63] + BufferedStreams v1.0.0
  [aafaddc9] - CatIndices v0.2.0
  [35d6a980] + ColorSchemes v3.5.0
  [ed09eef8] - ComputationalResources v0.3.0
  [150eb455] - CoordinateTransformations v0.5.0
  [dc8bdbbb] - CustomUnitRanges v0.2.0
  [2e619515] + Expat_jll v2.2.7+0
  [4f61f5a4] - FFTViews v0.3.0
  [e9467ef8]  GLMakie v0.0.8  v0.0.12
  [0862f596] + HTTPClient v0.2.1
  [bbac6d45] - IdentityRanges v0.3.0
  [2803e5a7] - ImageAxes v0.6.2
  [6a3955dd] - ImageFiltering v0.6.9
  [02fcd773] - ImageTransformations v0.8.1
  [9b13fd28] - IndirectArrays v0.5.1
  [c8e1da08] - IterTools v1.3.0
  [b27032c2] + LibCURL v0.5.2
  [522f3ed2] + LibExpat v0.6.0
  [2ec943e9] + Libz v1.0.1
  [33e6dc65] + MKL v0.1.0 #master (https://github.com/JuliaComputing/MKL.jl)
  [9b87118b] + PackageCompiler v0.6.5
  [b3c3ace0] - RangeArrays v0.3.2
  [6038ab10] - Rotations v0.13.0
  [699a6c99] - SimpleTraits v0.9.1
  [06e1c1a7] - TiledIteration v0.2.3
  [c17dfb99] + WinRPM v0.4.3

(v1.3) pkg> build MKL
  Building LibCURL ──────── `C:\Users\yahyaaba\.julia\packages\LibCURL\lWJxD\deps\build.log`
  Building WinRPM ───────── `C:\Users\yahyaaba\.julia\packages\WinRPM\BfpQI\deps\build.log`
  Building PackageCompiler  `C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\deps\build.log`
  Building MKL ──────────── `C:\Users\yahyaaba\.julia\packages\MKL\9dWbC\deps\build.log`
┌ Error: Error building `MKL`, showing the last 100 of log:
│ CRC32c  ─────────  0.009568 seconds
│ SHA  ────────────  0.175667 seconds
│ FileWatching  ───  0.087281 seconds
│ Unicode  ────────  0.010049 seconds
│ Mmap  ───────────  0.072874 seconds
│ Serialization  ──  0.880602 seconds
│ Libdl  ──────────  0.028785 seconds
│ Markdown  ───────  1.004954 seconds
│ LibGit2  ────────  2.635718 seconds
│ Logging  ────────  0.436265 seconds
│ Sockets  ────────  1.678002 seconds
│ Printf  ─────────  0.013321 seconds
│ Profile  ────────  0.223180 seconds
│ Dates  ──────────  2.114574 seconds
│ DelimitedFiles  ─  0.115828 seconds
│ Random  ─────────  0.495196 seconds
│ UUIDs  ──────────  0.019967 seconds
│ Future  ─────────  0.011228 seconds
│ LinearAlgebra  ── 10.850114 seconds
│ SparseArrays  ───  4.239812 seconds
│ SuiteSparse  ────  1.437989 seconds
│ Distributed  ────  4.812321 seconds
│ SharedArrays  ───  0.163400 seconds
│ Pkg  ──────────── 12.755205 seconds
│ Test  ───────────  0.912637 seconds
│ REPL  ───────────  0.853262 seconds
│ Statistics  ─────  0.160607 seconds
│ Stdlibs total  ── 49.844954 seconds
│ Sysimage built. Summary:
│ Total ───────  74.055858 seconds
│ Base: ───────  24.207745 seconds 32.6885%
│ Stdlibs: ────  49.844954 seconds 67.3072%
│ c:/users/yahyaaba/.julia/packages/winrpm/bfpqi/deps/usr/x86_64-w64-mingw32/sys-root/mingw/bin/../lib/gcc/x86_64-w64-mingw32/8.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: <unknown-file>:0: syntax error
│ Warning: .drectve `-export:ccalllib_C:\Users\yahyaaba\.julia\packages\MKL\9dWbC\deps\usr\bin\mkl_rt.dll,data ' unrecognized
│ c:/users/yahyaaba/.julia/packages/winrpm/bfpqi/deps/usr/x86_64-w64-mingw32/sys-root/mingw/bin/../lib/gcc/x86_64-w64-mingw32/8.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot export MKL: symbol not found
│ c:/users/yahyaaba/.julia/packages/winrpm/bfpqi/deps/usr/x86_64-w64-mingw32/sys-root/mingw/bin/../lib/gcc/x86_64-w64-mingw32/8.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot export Users: symbol not found
│ c:/users/yahyaaba/.julia/packages/winrpm/bfpqi/deps/usr/x86_64-w64-mingw32/sys-root/mingw/bin/../lib/gcc/x86_64-w64-mingw32/8.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot export ccalllib_C:: symbol not found
│ c:/users/yahyaaba/.julia/packages/winrpm/bfpqi/deps/usr/x86_64-w64-mingw32/sys-root/mingw/bin/../lib/gcc/x86_64-w64-mingw32/8.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot export packages: symbol not found
│ collect2.exe: error: ld returned 1 exit status
│ ┌ Warning: On Windows, creating file symlinks requires Administrator privileges
│ └ @ Base.Filesystem file.jl:848
│ [ Info: Replacing libblas_name in C:\Julia-1.3.1-MKL\bin\..\share\julia\base\build_h.jl
│ [ Info: Checking if we need to update PATH...
│ [ Info: Checking sysimg.jl
│ [ Info: Could not find init function in sysimg.jl
│ [ Info: Checking Base.jl
│ [ Info: Successfully modified Base.jl
│ ┌ Info: Building `inference.o`:
│ └ `'C:\Julia-1.3.1-MKL\bin\julia.exe' -C native --output-ji 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\basecompiler.ji' --output-o 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\basecompiler.o' compiler/compiler.jl`
│ ┌ Info: Building `sys.o`:
│ └ `'C:\Julia-1.3.1-MKL\bin\julia.exe' -C native --output-ji 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\sys.ji' --output-o 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\sys.o' -J 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\basecompiler.ji' --startup-file=no sysimg.jl`
│ ERROR: LoadError: failed process: Process(setenv(`'C:\Users\yahyaaba\.julia\packages\WinRPM\BfpQI\deps\usr\x86_64-w64-mingw32\sys-root\mingw\bin\gcc.exe' --sysroot 'C:\Users\yahyaaba\.julia\packages\WinRPM\BfpQI\deps\usr\x86_64-w64-mingw32\sys-root' -shared '-DJULIAC_PROGRAM_LIBNAME="C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\sys.dll"' -o 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\sys.dll' -Wl,--whole-archive 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\sys.o' -Wl,--no-whole-archive -std=gnu99 '-IC:\Julia-1.3.1-MKL\include\julia' -DJULIA_ENABLE_THREADING=1 '-LC:\Julia-1.3.1-MKL\bin' -Wl,--stack,8388608 -ljulia -lopenlibm -m64 -Wl,--export-all-symbols`,["PATH=C:\\Users\\yahyaaba\\.julia\\packages\\WinRPM\\BfpQI\\deps\\usr\\x86_64-w64-mingw32\\sys-root\\mingw\\bin;"]), ProcessExited(1)) [1]

│ Stacktrace:
│  [1] pipeline_error at .\process.jl:525 [inlined]
│  [2] #run#565(::Bool, ::typeof(run), ::Cmd) at .\process.jl:440
│  [3] run at .\process.jl:438 [inlined]
│  [4] run_PATH(::Cmd) at C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\src\static_julia.jl:23
│  [5] (::PackageCompiler.var"#16#18")() at C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\src\static_julia.jl:309
│  [6] cd(::PackageCompiler.var"#16#18", ::String) at .\file.jl:93
│  [7] build_shared(::String, ::String, ::Bool, ::String, ::Bool, ::Nothing, ::Nothing, ::Cmd, ::Nothing) at C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\src\static_julia.jl:308
│  [8] (::PackageCompiler.var"#40#41"{Bool,String})() at C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\src\system_image.jl:46
│  [9] cd(::PackageCompiler.var"#40#41"{Bool,String}, ::String) at .\file.jl:93
│  [10] #compile_system_image#39(::Bool, ::typeof(PackageCompiler.compile_system_image), ::String, ::String) at C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\src\system_image.jl:15
│  [11] #compile_system_image at .\none:0 [inlined]
│  [12] get_backup!(::Bool, ::String) at C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\src\system_image.jl:63
│  [13] force_native_image! at C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\src\api.jl:123 [inlined] (repeats 2 times)
│  [14] enable_mkl_startup(::String) at C:\Users\yahyaaba\.julia\packages\MKL\9dWbC\src\install.jl:138
│  [15] top-level scope at C:\Users\yahyaaba\.julia\packages\MKL\9dWbC\deps\build.jl:9
│  [16] include at .\boot.jl:328 [inlined]
│  [17] include_relative(::Module, ::String) at .\loading.jl:1105
│  [18] include(::Module, ::String) at .\Base.jl:31
│  [19] include(::String) at .\client.jl:424
│  [20] top-level scope at none:5
│ in expression starting at C:\Users\yahyaaba\.julia\packages\MKL\9dWbC\deps\build.jl:9
│ caused by [exception 1]
│ IOError: symlink: operation not permitted (EPERM)
│ Stacktrace:
│  [1] uv_error at .\libuv.jl:97 [inlined]
│  [2] symlink(::String, ::String) at .\file.jl:851
│  [3] probe_symlink_creation(::String) at C:\Users\yahyaaba\.julia\packages\BinaryProvider\kcGxO\src\PlatformEngines.jl:121
│  [4] #probe_platform_engines!#30(::Bool, ::typeof(BinaryProvider.probe_platform_engines!)) at C:\Users\yahyaaba\.julia\packages\BinaryProvider\kcGxO\src\PlatformEngines.jl:175
│  [5] probe_platform_engines! at C:\Users\yahyaaba\.julia\packages\BinaryProvider\kcGxO\src\PlatformEngines.jl:169 [inlined]
│  [6] __init__() at C:\Users\yahyaaba\.julia\packages\BinaryProvider\kcGxO\src\BinaryProvider.jl:28
│  [7] _include_from_serialized(::String, ::Array{Any,1}) at .\loading.jl:692
│  [8] _require_search_from_serialized(::Base.PkgId, ::String) at .\loading.jl:776
│  [9] _require(::Base.PkgId) at .\loading.jl:1001
│  [10] require(::Base.PkgId) at .\loading.jl:922
│  [11] require(::Module, ::Symbol) at .\loading.jl:917
│  [12] include at .\boot.jl:328 [inlined]
│  [13] include_relative(::Module, ::String) at .\loading.jl:1105
│  [14] include(::Module, ::String) at .\Base.jl:31
│  [15] include(::String) at .\client.jl:424
│  [16] top-level scope at C:\Users\yahyaaba\.julia\packages\MKL\9dWbC\deps\build.jl:4
│  [17] include at .\boot.jl:328 [inlined]
│  [18] include_relative(::Module, ::String) at .\loading.jl:1105
│  [19] include(::Module, ::String) at .\Base.jl:31
│  [20] include(::String) at .\client.jl:424
│  [21] top-level scope at none:5
│ Build shared library "C:\\Users\\yahyaaba\\.julia\\packages\\PackageCompiler\\4yNnV\\sysimg\\backup\\native\\sys.dll":
`'C:\Users\yahyaaba\.julia\packages\WinRPM\BfpQI\deps\usr\x86_64-w64-mingw32\sys-root\mingw\bin\gcc.exe' --sysroot 'C:\Users\yahyaaba\.julia\packages\WinRPM\BfpQI\deps\usr\x86_64-w64-mingw32\sys-root' -shared '-DJULIAC_PROGRAM_LIBNAME="C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\sys.dll"' -o 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\sys.dll' -Wl,--whole-archive 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\sys.o' -Wl,--no-whole-archive -std=gnu99 '-IC:\Julia-1.3.1-MKL\include\julia' -DJULIA_ENABLE_THREADING=1 '-LC:\Julia-1.3.1-MKL\bin' -Wl,--stack,8388608 -ljulia -lopenlibm -m64 -Wl,--export-all-symbols`

│ Full log at C:\Users\yahyaaba\.julia\packages\MKL\9dWbC\deps\build.log
└ @ Pkg.Operations D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.3\Pkg\src\backwards_compatible_isolation.jl:649

Full Log

essentials.jl
ctypes.jl
generator.jl
reflection.jl
options.jl
promotion.jl
tuple.jl
pair.jl
traits.jl
range.jl
expr.jl
error.jl
bool.jl
number.jl
int.jl
operators.jl
pointer.jl
refvalue.jl
indices.jl
array.jl
abstractarray.jl
bitarray.jl
bitset.jl
abstractdict.jl
abstractset.jl
iterators.jl
namedtuple.jl
docs/core.jl
ordering.jl
sort.jl
compiler/utilities.jl
compiler/validation.jl
compiler/inferenceresult.jl
compiler/params.jl
compiler/inferencestate.jl
compiler/typeutils.jl
compiler/typelimits.jl
compiler/typelattice.jl
compiler/tfuncs.jl
compiler/abstractinterpretation.jl
compiler/typeinfer.jl
compiler/optimize.jl
compiler/ssair/driver.jl
compiler/ssair/ir.jl
compiler/ssair/domtree.jl
compiler/ssair/slot2ssa.jl
compiler/ssair/queries.jl
compiler/ssair/passes.jl
compiler/ssair/inlining.jl
compiler/ssair/verify.jl
compiler/ssair/legacy.jl
compiler/bootstrap.jl
coreio.jl
exports.jl
essentials.jl
ctypes.jl
gcutils.jl
generator.jl
reflection.jl
options.jl
promotion.jl
tuple.jl
expr.jl
pair.jl
traits.jl
range.jl
error.jl
bool.jl
number.jl
int.jl
operators.jl
pointer.jl
refvalue.jl
refpointer.jl
checked.jl
indices.jl
array.jl
abstractarray.jl
subarray.jl
views.jl
baseext.jl
ntuple.jl
abstractdict.jl
iterators.jl
namedtuple.jl
hashing.jl
rounding.jl
float.jl
twiceprecision.jl
complex.jl
rational.jl
multinverses.jl
abstractarraymath.jl
arraymath.jl
simdloop.jl
reduce.jl
reshapedarray.jl
reinterpretarray.jl
bitarray.jl
bitset.jl
multimedia.jl
some.jl
dict.jl
abstractset.jl
set.jl
char.jl
strings/basic.jl
strings/string.jl
strings/substring.jl
build_h.jl
version_git.jl
osutils.jl
c.jl
io.jl
iobuffer.jl
intfuncs.jl
strings/strings.jl
strings/search.jl
strings/unicode.jl
strings/util.jl
strings/io.jl
parse.jl
shell.jl
regex.jl
pcre.jl
show.jl
arrayshow.jl
methodshow.jl
cartesian.jl
multidimensional.jl
permuteddimsarray.jl
broadcast.jl
missing.jl
version.jl
sysinfo.jl
libc.jl
env.jl
linked_list.jl
condition.jl
threads.jl
lock.jl
task.jl
weakkeydict.jl
logging.jl
libuv.jl
uv_constants.jl
asyncevent.jl
iostream.jl
stream.jl
filesystem.jl
cmd.jl
process.jl
grisu/grisu.jl
secretbuffer.jl
floatfuncs.jl
math.jl
reducedim.jl
accumulate.jl
ordering.jl
sort.jl
fastmath.jl
Enums.jl
gmp.jl
mpfr.jl
combinatorics.jl
hashing2.jl
irrationals.jl
mathconstants.jl
printf.jl
meta.jl
channels.jl
deepcopy.jl
download.jl
summarysize.jl
errorshow.jl
stacktraces.jl
initdefs.jl
threadcall.jl
uuid.jl
loading.jl
util.jl
asyncmap.jl
experimental.jl
deprecated.jl
docs\basedocs.jl
client.jl
docs\Docs.jl
Base  ─────────── 24.207745 seconds
Base64  ─────────  3.632059 seconds
CRC32c  ─────────  0.009568 seconds
SHA  ────────────  0.175667 seconds
FileWatching  ───  0.087281 seconds
Unicode  ────────  0.010049 seconds
Mmap  ───────────  0.072874 seconds
Serialization  ──  0.880602 seconds
Libdl  ──────────  0.028785 seconds
Markdown  ───────  1.004954 seconds
LibGit2  ────────  2.635718 seconds
Logging  ────────  0.436265 seconds
Sockets  ────────  1.678002 seconds
Printf  ─────────  0.013321 seconds
Profile  ────────  0.223180 seconds
Dates  ──────────  2.114574 seconds
DelimitedFiles  ─  0.115828 seconds
Random  ─────────  0.495196 seconds
UUIDs  ──────────  0.019967 seconds
Future  ─────────  0.011228 seconds
LinearAlgebra  ── 10.850114 seconds
SparseArrays  ───  4.239812 seconds
SuiteSparse  ────  1.437989 seconds
Distributed  ────  4.812321 seconds
SharedArrays  ───  0.163400 seconds
Pkg  ──────────── 12.755205 seconds
Test  ───────────  0.912637 seconds
REPL  ───────────  0.853262 seconds
Statistics  ─────  0.160607 seconds
Stdlibs total  ── 49.844954 seconds
Sysimage built. Summary:
Total ───────  74.055858 seconds 
Base: ───────  24.207745 seconds 32.6885%
Stdlibs: ────  49.844954 seconds 67.3072%
c:/users/yahyaaba/.julia/packages/winrpm/bfpqi/deps/usr/x86_64-w64-mingw32/sys-root/mingw/bin/../lib/gcc/x86_64-w64-mingw32/8.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: <unknown-file>:0: syntax error
Warning: .drectve `-export:ccalllib_C:\Users\yahyaaba\.julia\packages\MKL\9dWbC\deps\usr\bin\mkl_rt.dll,data ' unrecognized
c:/users/yahyaaba/.julia/packages/winrpm/bfpqi/deps/usr/x86_64-w64-mingw32/sys-root/mingw/bin/../lib/gcc/x86_64-w64-mingw32/8.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot export MKL: symbol not found
c:/users/yahyaaba/.julia/packages/winrpm/bfpqi/deps/usr/x86_64-w64-mingw32/sys-root/mingw/bin/../lib/gcc/x86_64-w64-mingw32/8.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot export Users: symbol not found
c:/users/yahyaaba/.julia/packages/winrpm/bfpqi/deps/usr/x86_64-w64-mingw32/sys-root/mingw/bin/../lib/gcc/x86_64-w64-mingw32/8.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot export ccalllib_C:: symbol not found
c:/users/yahyaaba/.julia/packages/winrpm/bfpqi/deps/usr/x86_64-w64-mingw32/sys-root/mingw/bin/../lib/gcc/x86_64-w64-mingw32/8.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot export packages: symbol not found
collect2.exe: error: ld returned 1 exit status
┌ Warning: On Windows, creating file symlinks requires Administrator privileges
└ @ Base.Filesystem file.jl:848
[ Info: Replacing libblas_name in C:\Julia-1.3.1-MKL\bin\..\share\julia\base\build_h.jl
[ Info: Checking if we need to update PATH...
[ Info: Checking sysimg.jl
[ Info: Could not find init function in sysimg.jl
[ Info: Checking Base.jl
[ Info: Successfully modified Base.jl
┌ Info: Building `inference.o`:
`'C:\Julia-1.3.1-MKL\bin\julia.exe' -C native --output-ji 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\basecompiler.ji' --output-o 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\basecompiler.o' compiler/compiler.jl`
┌ Info: Building `sys.o`:
`'C:\Julia-1.3.1-MKL\bin\julia.exe' -C native --output-ji 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\sys.ji' --output-o 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\sys.o' -J 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\basecompiler.ji' --startup-file=no sysimg.jl`
ERROR: LoadError: failed process: Process(setenv(`'C:\Users\yahyaaba\.julia\packages\WinRPM\BfpQI\deps\usr\x86_64-w64-mingw32\sys-root\mingw\bin\gcc.exe' --sysroot 'C:\Users\yahyaaba\.julia\packages\WinRPM\BfpQI\deps\usr\x86_64-w64-mingw32\sys-root' -shared '-DJULIAC_PROGRAM_LIBNAME="C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\sys.dll"' -o 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\sys.dll' -Wl,--whole-archive 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\sys.o' -Wl,--no-whole-archive -std=gnu99 '-IC:\Julia-1.3.1-MKL\include\julia' -DJULIA_ENABLE_THREADING=1 '-LC:\Julia-1.3.1-MKL\bin' -Wl,--stack,8388608 -ljulia -lopenlibm -m64 -Wl,--export-all-symbols`,["PATH=C:\\Users\\yahyaaba\\.julia\\packages\\WinRPM\\BfpQI\\deps\\usr\\x86_64-w64-mingw32\\sys-root\\mingw\\bin;C:\\WINDOWS\\system32\\WBEM;C:\\Program Files (x86)\\Common Files\\Oracle\\Java\\javapath;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\;"]), ProcessExited(1)) [1]

Stacktrace:
 [1] pipeline_error at .\process.jl:525 [inlined]
 [2] #run#565(::Bool, ::typeof(run), ::Cmd) at .\process.jl:440
 [3] run at .\process.jl:438 [inlined]
 [4] run_PATH(::Cmd) at C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\src\static_julia.jl:23
 [5] (::PackageCompiler.var"#16#18")() at C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\src\static_julia.jl:309
 [6] cd(::PackageCompiler.var"#16#18", ::String) at .\file.jl:93
 [7] build_shared(::String, ::String, ::Bool, ::String, ::Bool, ::Nothing, ::Nothing, ::Cmd, ::Nothing) at C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\src\static_julia.jl:308
 [8] (::PackageCompiler.var"#40#41"{Bool,String})() at C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\src\system_image.jl:46
 [9] cd(::PackageCompiler.var"#40#41"{Bool,String}, ::String) at .\file.jl:93
 [10] #compile_system_image#39(::Bool, ::typeof(PackageCompiler.compile_system_image), ::String, ::String) at C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\src\system_image.jl:15
 [11] #compile_system_image at .\none:0 [inlined]
 [12] get_backup!(::Bool, ::String) at C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\src\system_image.jl:63
 [13] force_native_image! at C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\src\api.jl:123 [inlined] (repeats 2 times)
 [14] enable_mkl_startup(::String) at C:\Users\yahyaaba\.julia\packages\MKL\9dWbC\src\install.jl:138
 [15] top-level scope at C:\Users\yahyaaba\.julia\packages\MKL\9dWbC\deps\build.jl:9
 [16] include at .\boot.jl:328 [inlined]
 [17] include_relative(::Module, ::String) at .\loading.jl:1105
 [18] include(::Module, ::String) at .\Base.jl:31
 [19] include(::String) at .\client.jl:424
 [20] top-level scope at none:5
in expression starting at C:\Users\yahyaaba\.julia\packages\MKL\9dWbC\deps\build.jl:9
caused by [exception 1]
IOError: symlink: operation not permitted (EPERM)
Stacktrace:
 [1] uv_error at .\libuv.jl:97 [inlined]
 [2] symlink(::String, ::String) at .\file.jl:851
 [3] probe_symlink_creation(::String) at C:\Users\yahyaaba\.julia\packages\BinaryProvider\kcGxO\src\PlatformEngines.jl:121
 [4] #probe_platform_engines!#30(::Bool, ::typeof(BinaryProvider.probe_platform_engines!)) at C:\Users\yahyaaba\.julia\packages\BinaryProvider\kcGxO\src\PlatformEngines.jl:175
 [5] probe_platform_engines! at C:\Users\yahyaaba\.julia\packages\BinaryProvider\kcGxO\src\PlatformEngines.jl:169 [inlined]
 [6] __init__() at C:\Users\yahyaaba\.julia\packages\BinaryProvider\kcGxO\src\BinaryProvider.jl:28
 [7] _include_from_serialized(::String, ::Array{Any,1}) at .\loading.jl:692
 [8] _require_search_from_serialized(::Base.PkgId, ::String) at .\loading.jl:776
 [9] _require(::Base.PkgId) at .\loading.jl:1001
 [10] require(::Base.PkgId) at .\loading.jl:922
 [11] require(::Module, ::Symbol) at .\loading.jl:917
 [12] include at .\boot.jl:328 [inlined]
 [13] include_relative(::Module, ::String) at .\loading.jl:1105
 [14] include(::Module, ::String) at .\Base.jl:31
 [15] include(::String) at .\client.jl:424
 [16] top-level scope at C:\Users\yahyaaba\.julia\packages\MKL\9dWbC\deps\build.jl:4
 [17] include at .\boot.jl:328 [inlined]
 [18] include_relative(::Module, ::String) at .\loading.jl:1105
 [19] include(::Module, ::String) at .\Base.jl:31
 [20] include(::String) at .\client.jl:424
 [21] top-level scope at none:5
Build shared library "C:\\Users\\yahyaaba\\.julia\\packages\\PackageCompiler\\4yNnV\\sysimg\\backup\\native\\sys.dll":
  `'C:\Users\yahyaaba\.julia\packages\WinRPM\BfpQI\deps\usr\x86_64-w64-mingw32\sys-root\mingw\bin\gcc.exe' --sysroot 'C:\Users\yahyaaba\.julia\packages\WinRPM\BfpQI\deps\usr\x86_64-w64-mingw32\sys-root' -shared '-DJULIAC_PROGRAM_LIBNAME="C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\sys.dll"' -o 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\sys.dll' -Wl,--whole-archive 'C:\Users\yahyaaba\.julia\packages\PackageCompiler\4yNnV\sysimg\backup\native\sys.o' -Wl,--no-whole-archive -std=gnu99 '-IC:\Julia-1.3.1-MKL\include\julia' -DJULIA_ENABLE_THREADING=1 '-LC:\Julia-1.3.1-MKL\bin' -Wl,--stack,8388608 -ljulia -lopenlibm -m64 -Wl,--export-all-symbols`
@aminya

This comment has been minimized.

@aminya aminya changed the title MKL isn't built on Windows locally MKL isn't built on Windows Jan 1, 2020
@Crown421
Copy link

Crown421 commented Jan 2, 2020

Currently also dealing with this and I think the problem is more complicated. I have tried pinning PackageCompiler to 0.6.4 and 0.6.3 and it still fails.
I have further tried different commits of MKL, specifically

with the pinned version of PackageCompiler, so the problem does not appear to lie their combination.

Further, as you can see in this CI run, MKL builds successfully on Julia 1.0, but then throws a different error when actually using it with VML.

Run julia-actions/julia-buildpkg@latest
julia --color=yes --project -e "using Pkg; if VERSION >= v\"1.1.0-rc1\"; Pkg.build(verbose=true); else Pkg.build(); end"
Intel MKL FATAL ERROR: Cannot load mkl_intel_thread.dll.
##[error]The process 'julia' failed with exit code 2
##[error]Node run failed with exit code 1

On Julia 1.3 the build itself fails as above.

@Crown421
Copy link

Crown421 commented Jan 2, 2020

Further, building MKL already failed on nightly Julia at the end of November, as seen here.

The last PackageCompiler update was Dec. 18, and it appears back then it was WinRPM that caused problems.

I tried pinning all packages to the versions in the successful Julia 1.0 run, but no luck so far.

@aminya
Copy link
Contributor Author

aminya commented Jan 2, 2020

Further, as you can see in this CI run, MKL builds successfully on Julia 1.0, but then throws a different error when actually using it with VML.

This error was fixed on MKL by a316332. See this successful CI for it: https://ci.appveyor.com/project/andreasnoack/mkl-jl/builds/28053628/job/su5qa4uk4mb5nb3k

@aminya
Copy link
Contributor Author

aminya commented Jan 2, 2020

This combination was the latest successful build on Julia 1 Windows:

Packages
 Installed WinRPM ────────── v0.4.2
 Installed HTTPClient ────── v0.2.1
 Installed Compat ────────── v2.2.0
 Installed Libz ──────────── v1.0.0
 Installed BinDeps ───────── v0.8.10
 Installed BufferedStreams ─ v1.0.0
 Installed BinaryProvider ── v0.5.8
 Installed PackageCompiler ─ v0.6.4
 Installed LibExpat ──────── v0.5.0
 Installed URIParser ─────── v0.4.0
 Installed LibCURL ───────── v0.5.2
  Updating `C:\projects\mkl-jl\Project.toml`
  [b99e7846] + BinaryProvider v0.5.8
  [9b87118b] + PackageCompiler v0.6.4
  [8f399da3] + Libdl 
  [37e2e46d] + LinearAlgebra 
  [8dfed614] + Test 
  Updating `C:\projects\mkl-jl\Manifest.toml`
  [9e28174c] + BinDeps v0.8.10
  [b99e7846] + BinaryProvider v0.5.8
  [e1450e63] + BufferedStreams v1.0.0
  [34da2185] + Compat v2.2.0
  [0862f596] + HTTPClient v0.2.1
  [b27032c2] + LibCURL v0.5.2
  [522f3ed2] + LibExpat v0.5.0
  [2ec943e9] + Libz v1.0.0
  [9b87118b] + PackageCompiler v0.6.4
  [30578b45] + URIParser v0.4.0
  [c17dfb99] + WinRPM v0.4.2
  [2a0f44e3] + Base64 
  [ade2ca70] + Dates 
  [8bb1440f] + DelimitedFiles 
  [8ba89e20] + Distributed 
  [b77e0a4c] + InteractiveUtils 
  [76f85450] + LibGit2 
  [8f399da3] + Libdl 
  [37e2e46d] + LinearAlgebra 
  [56ddb016] + Logging 
  [d6f4376e] + Markdown 
  [a63ad114] + Mmap 
  [44cfe95a] + Pkg 
  [de0858da] + Printf 
  [3fa0cd96] + REPL 
  [9a3f8284] + Random 
  [ea8e919c] + SHA 
  [9e88b42a] + Serialization 
  [1a1011a3] + SharedArrays 
  [6462fe0b] + Sockets 
  [2f01184e] + SparseArrays 
  [10745b16] + Statistics 
  [8dfed614] + Test 
  [cf7118a7] + UUIDs 
  [4ec0a83e] + Unicode

@IanButterworth
Copy link

IanButterworth commented Jan 17, 2020

Given FFTW.jl now depends on MKL_jll (which includes IntelOpenMP_jll), it seems worth it to try a PR that simplifies the build to just depending on MKL_jll.

The caveat is that jll's require julia 1.3.

Or you could just use the binaries that have been wrapped: https://github.com/JuliaBinaryWrappers/MKL_jll.jl
https://github.com/JuliaBinaryWrappers/IntelOpenMP_jll.jl

@KristofferC
Copy link
Contributor

The problem is that you want Julia's own methods (like matrix multiplication) to call into MKL which with the current code structure requires a sysimage rebuild.

@aminya
Copy link
Contributor Author

aminya commented Jan 17, 2020

The problem is that you want Julia's own methods (like matrix multiplication) to call into MKL which with the current code structure requires a sysimage rebuild.

This is the default MKL.jl repository.

@KristofferC
Copy link
Contributor

KristofferC commented Jan 17, 2020

Yes... Sorry I didn't understand the point you were making. Could you elaborate?

@aminya
Copy link
Contributor Author

aminya commented Jan 17, 2020

Yes... Sorry I didn't understand the point you were making. Could you elaborate?

We can't use MKL.jl on Windows:

julia>]add https://github.com/JuliaComputing/MKL.jl

results in the error I wrote in the OP

@KristofferC
Copy link
Contributor

My reply was regarding MKL_jll and that it doesn't change the fact that the sysimage has to be rebuilt.

@Crown421
Copy link

Somewhat off-topic, but who is maintaining MKL_jll/ are packages apart from FFTW.jl using it?

@IanButterworth
Copy link

@Crown421 Like most JLL's it's being "maintained" by the #binarybuilder julia slack channel folks. I'm not sure if packages other than FFTW.jl are using it yet.

@KristofferC My bad, I though it was a MKL build problem rather than a PackageCompiler issue. Should've read the issue in more detail. Sorry for the noise

@aminya
Copy link
Contributor Author

aminya commented Jan 19, 2020

So should we switch to PackageCompilerX or something or we can solve this in an easier way?

@KristofferC

@KristofferC
Copy link
Contributor

I don't think the error Intel MKL FATAL ERROR: Cannot load mkl_intel_thread.dll would be helped by doing that change.

@jlperla
Copy link

jlperla commented Jan 22, 2020

Is moving to require Julia 1.3 so you rely on the artifacts helpful? I think it is a reasonable requirement if it would simplify the setup issues of binary locations?

@aminya
Copy link
Contributor Author

aminya commented Jan 22, 2020

I don't think the error Intel MKL FATAL ERROR: Cannot load mkl_intel_thread.dll would be helped by doing that change.

This is not the error that i get. Please see the log I attached in OP.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants