LLVM not found

[ekurtulus]

Describe the bug

I am using Julia 1.5.3. When I add Cuda package via Pkg.add("CUDA") and use it I am getting the following errors:

julia> using CUDA
[ Info: Precompiling CUDA [052768ef-5323-5732-b1bb-66c8b64840ba]
ERROR: LoadError: InitError: could not load library "C:\Users\Emirhan Kurtulu�\AppData\Local\Programs\Julia 1.5.3\bin\LLVM.dll"
Belirtilen modül bulunamadı.
Stacktrace:
 [1] dlopen at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Libdl\src\Libdl.jl:109 [inlined] (repeats 2 times)
 [2] (::LLVM.var"#14#cache_fptr!#3")() at C:\Users\Emirhan Kurtuluş\.julia\packages\LLVM\7Q46C\src\util.jl:103
 [3] macro expansion at C:\Users\Emirhan Kurtuluş\.julia\packages\LLVM\7Q46C\src\util.jl:111 [inlined]
 [4] runtime_version() at C:\Users\Emirhan Kurtuluş\.julia\packages\LLVM\7Q46C\src\base.jl:9
 [5] __init__() at C:\Users\Emirhan Kurtuluş\.julia\packages\LLVM\7Q46C\src\LLVM.jl:88
 [6] top-level scope at none:2
 [7] eval at .\boot.jl:331 [inlined]
during initialization of module LLVM
in expression starting at C:\Users\Emirhan Kurtuluş\.julia\packages\GPUCompiler\uTpNx\src\GPUCompiler.jl:3
ERROR: LoadError: Failed to precompile GPUCompiler [61eb1bfa-7361-4325-ad38-22787b887f55] to C:\Users\Emirhan Kurtuluş\.julia\compiled\v1.5\GPUCompiler\yPwef_DUzIX.ji.
Stacktrace:
 [1] top-level scope at none:2
 [2] eval at .\boot.jl:331 [inlined]
in expression starting at C:\Users\Emirhan Kurtuluş\.julia\packages\CUDA\wTQsK\src\CUDA.jl:3
ERROR: Failed to precompile CUDA [052768ef-5323-5732-b1bb-66c8b64840ba] to C:\Users\Emirhan Kurtuluş\.julia\compiled\v1.5\CUDA\oWw5k_DUzIX.ji.
Stacktrace:
 [1] run_repl(::REPL.AbstractREPL, ::Any) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\REPL\src\REPL.jl:288

I tried to pin down the problem but could not get any leads. Pytorch and Tensorflow successfully uses my GPU without a problem. I tried using Knet.jl and Flux.jl to see if the problem is on the library but when I saw that I get the same error on both of them, I decided to submit the issue here. I checked "Julia 1.5.3\bin\LLVM.dll" manually to ascertain that the file exists.

Version info

Details on Julia:

julia> versioninfo()
Julia Version 1.5.3
Commit 788b2c77c1 (2020-11-09 13:37 UTC)
Platform Info:
OS: Windows (x86_64-w64-mingw32)
CPU: Intel(R) Core(TM) i7-10875H CPU @ 2.30GHz
WORD_SIZE: 64
LIBM: libopenlibm
LLVM: libLLVM-9.0.1 (ORCJIT, skylake)

[maleadt]

That's strange. Could you post the full output of using Libdl; println.(Libdl.dllist());?

[ekurtulus]

julia> using Libdl; println.(Libdl.dllist());
C:\Users\Emirhan Kurtulu�\AppData\Local\Programs\Julia 1.5.3\bin\julia.exe
C:\Windows\SYSTEM32\ntdll.dll
C:\Windows\System32\KERNEL32.DLL
C:\Windows\System32\KERNELBASE.dll
C:\Windows\System32\msvcrt.dll
C:\Users\Emirhan Kurtulu�\AppData\Local\Programs\Julia 1.5.3\bin\libjulia.dll
C:\Users\Emirhan Kurtulu�\AppData\Local\Programs\Julia 1.5.3\bin\libgcc_s_seh-1.dll
C:\Windows\System32\ADVAPI32.dll
C:\Windows\System32\sechost.dll
C:\Users\Emirhan Kurtulu�\AppData\Local\Programs\Julia 1.5.3\bin\libssp-0.dll
C:\Users\Emirhan Kurtulu�\AppData\Local\Programs\Julia 1.5.3\bin\libwinpthread-1.dll
C:\Windows\System32\RPCRT4.dll
C:\Users\Emirhan Kurtulu�\AppData\Local\Programs\Julia 1.5.3\bin\libstdc++-6.dll
C:\Windows\System32\PSAPI.DLL
C:\Windows\System32\USER32.dll
C:\Windows\System32\win32u.dll
C:\Windows\System32\GDI32.dll
C:\Windows\SYSTEM32\dbghelp.dll
C:\Windows\System32\gdi32full.dll
C:\Windows\System32\ucrtbase.dll
C:\Windows\SYSTEM32\IPHLPAPI.DLL
C:\Windows\System32\msvcp_win.dll
C:\Windows\System32\WS2_32.dll
C:\Windows\SYSTEM32\Secur32.dll
C:\Windows\SYSTEM32\USERENV.dll
C:\Windows\SYSTEM32\WINMM.dll
C:\Users\Emirhan Kurtulu�\AppData\Local\Programs\Julia 1.5.3\bin\LLVM.dll
C:\Windows\System32\ole32.dll
C:\Windows\System32\combase.dll
C:\Windows\System32\SHELL32.dll
C:\Windows\SYSTEM32\SSPICLI.DLL
C:\Windows\SYSTEM32\CRYPTSP.dll
C:\Windows\system32\rsaenh.dll
C:\Windows\System32\bcrypt.dll
C:\Windows\SYSTEM32\CRYPTBASE.dll
C:\Windows\System32\bcryptPrimitives.dll
C:\Windows\System32\IMM32.DLL
C:\Windows\SYSTEM32\powrprof.dll
C:\Windows\SYSTEM32\UMPDC.dll
C:\Windows\system32\uxtheme.dll
C:\Windows\system32\mswsock.dll
C:\Users\Emirhan Kurtulu�\AppData\Local\Programs\Julia 1.5.3\lib\julia\sys.dll
C:\Users\Emirhan Kurtulu�\AppData\Local\Programs\Julia 1.5.3\bin\libpcre2-8.DLL
C:\Users\Emirhan Kurtulu�\AppData\Local\Programs\Julia 1.5.3\bin\libgmp.DLL
C:\Users\Emirhan Kurtulu�\AppData\Local\Programs\Julia 1.5.3\bin\libmpfr.DLL
C:\Users\Emirhan Kurtulu�\AppData\Local\Programs\Julia 1.5.3\bin\libgmp-10.dll
C:\Users\Emirhan Kurtulu�\AppData\Local\Programs\Julia 1.5.3\bin\libopenblas64_.DLL
C:\Users\Emirhan Kurtulu�\AppData\Local\Programs\Julia 1.5.3\bin\libgfortran-5.dll
C:\Users\Emirhan Kurtulu�\AppData\Local\Programs\Julia 1.5.3\bin\libquadmath-0.dll
C:\Users\Emirhan Kurtulu�\AppData\Local\Programs\Julia 1.5.3\bin\libcholmod.DLL
C:\Users\Emirhan Kurtulu�\AppData\Local\Programs\Julia 1.5.3\bin\libcamd.dll
C:\Users\Emirhan Kurtulu�\AppData\Local\Programs\Julia 1.5.3\bin\libccolamd.dll
C:\Users\Emirhan Kurtulu�\AppData\Local\Programs\Julia 1.5.3\bin\libsuitesparseconfig.dll
C:\Users\Emirhan Kurtulu�\AppData\Local\Programs\Julia 1.5.3\bin\libamd.dll
C:\Users\Emirhan Kurtulu�\AppData\Local\Programs\Julia 1.5.3\bin\libcolamd.dll
C:\Users\Emirhan Kurtulu�\AppData\Local\Programs\Julia 1.5.3\bin\libsuitesparse_wrapper.DLL

[maleadt]

Well, that's weird. Can you try the following in a clean session:

using Libdl
Libdl.dlopen("C:\Users\Emirhan Kurtulu�\AppData\Local\Programs\Julia 1.5.3\bin\LLVM.dll")

[ekurtulus]

julia> Libdl.dlopen("C:/Users/Emirhan Kurtuluş/AppData/Local/Programs/Julia 1.5.3/bin/LLVM.dll")
Ptr{Nothing} @0x0000000002700000

� character is "ş" which is non-ascii, is there any chance that this is the problem?

[maleadt]

Maybe; try:

@show Base.libllvm_path()
@show String(Base.libllvm_path())
using Libdl
Libdl.dlopen(String(Base.libllvm_path()))

[ekurtulus]

julia> @show Base.libllvm_path()
ERROR: UndefVarError: libllvm_path not defined
Stacktrace:
 [1] top-level scope at show.jl:641
 [2] run_repl(::REPL.AbstractREPL, ::Any) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\REPL\src\REPL.jl:288

julia> @show String(Base.libllvm_path())
ERROR: UndefVarError: libllvm_path not defined
Stacktrace:
 [1] top-level scope at show.jl:641
 [2] run_repl(::REPL.AbstractREPL, ::Any) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\REPL\src\REPL.jl:288

julia> using Libdl

julia> Libdl.dlopen(String(Base.libllvm_path()))
ERROR: UndefVarError: libllvm_path not defined
Stacktrace:
 [1] top-level scope at REPL[9]:1
 [2] run_repl(::REPL.AbstractREPL, ::Any) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\REPL\src\REPL.jl:288

[maleadt]

Oh right, you're using Julia 1.5 which doesn't have libllvm_path(). No point in executing all those commands, then.
Basically, we go through the list of libraries in Libdl.dllist() and try to find LLVM's here: https://github.com/maleadt/LLVM.jl/blob/1ca9dc611a5506d7be5688c2bee1fe1f93749e1e/src/LLVM.jl#L72-L84
At some point it seems to lose the unicode char. Can you reproduce those steps and figure our where?

[ekurtulus]

julia> using Libdl

julia>     libllvm_paths = filter(Libdl.dllist()) do lib
                   occursin(r"LLVM\b", basename(lib))
               end
1-element Array{String,1}:
 "C:\\Users\\Emirhan Kurtulu\xfe\\AppData\\Local\\Programs\\Julia 1.5.3\\bin\\LLVM.dll"
julia>         if isempty(libllvm_paths)
                   error("""Cannot find the LLVM library loaded by Julia.
                            Please use a version of Julia that has been built with USE_LLVM_SHLIB=1 (like the official binaries).
                            If you are, please file an issue and attach the output of `Libdl.dllist()`.""")
               end

julia>         if length(libllvm_paths) > 1
                   error("""Multiple LLVM libraries loaded by Julia.
                            Please file an issue and attach the output of `Libdl.dllist()`.""")
               end

julia>         first(libllvm_paths)
"C:\\Users\\Emirhan Kurtulu\xfe\\AppData\\Local\\Programs\\Julia 1.5.3\\bin\\LLVM.dll"

It seems to be finding the LLVM.dll but does not work during precompilation. I really could not back trace where the error occurs.

[maleadt]

And you can dlopen the result of that?

[ekurtulus]

julia> dlopen(first(libllvm_paths))
ERROR: could not load library "C:\Users\Emirhan Kurtulu�\AppData\Local\Programs\Julia 1.5.3\bin\LLVM.dll"
Belirtilen modül bulunamadı.
Stacktrace:
 [1] dlopen at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\Libdl\src\Libdl.jl:109 [inlined] (repeats 2 times)
 [2] top-level scope at REPL[7]:1
 [3] run_repl(::REPL.AbstractREPL, ::Any) at C:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.5\REPL\src\REPL.jl:288

It seems I cannot, but Julia IO is capable of finding the file:

julia> isfile("C:/Users/Emirhan Kurtuluş/AppData/Local/Programs/Julia 1.5.3/bin/LLVM.dll")
true

As far as I reviewed the Libdl library it is just an export of the inner C library so it is incapable of dealing with unicode characters. I do not believe that such an extension of Julia functionality is required; however, a warning (or directly an error since this will already result in unexpected behavior) must be given by CUDA library. If you concur, I can prepare a pull request on this.

[maleadt]

FWIW, this isn't a CUDA.jl issue, or CUDA related. It's an issue of Julia or LLVM.jl, but I can't transfer the issue outside of this organization.

Can you try Julia 1.6 to see if the new mechanism there fixes this issue?

[ekurtulus]

It seems to work with 1.6. I also changed the directory names so that such a problem will never exist. Thank you.