Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't install MXNet on windows #257

Open
mikewl opened this issue Jun 14, 2017 · 54 comments
Open

Can't install MXNet on windows #257

mikewl opened this issue Jun 14, 2017 · 54 comments

Comments

@mikewl
Copy link
Contributor

mikewl commented Jun 14, 2017

julia> using MXNet
INFO: Recompiling stale cache file C:\Users\micha\.julia\lib\v0.5\MXNet.ji for module MXNet.
ERROR: LoadError: LoadError: Cannot find or load libmxnet.so. Please see the document on how to build it.
 in error(::String) at .\error.jl:21
 in include_from_node1(::String) at .\loading.jl:488 (repeats 2 times)
 in macro expansion; at .\none:2 [inlined]
 in anonymous at .\<missing>:?
 in eval(::Module, ::Any) at .\boot.jl:234
 in process_options(::Base.JLOptions) at .\client.jl:242
 in _start() at .\client.jl:321
while loading C:\Users\micha\.julia\v0.5\MXNet\src\base.jl, in expression starting on line 26
while loading C:\Users\micha\.julia\v0.5\MXNet\src\MXNet.jl, in expression starting on line 24
ERROR: Failed to precompile MXNet to C:\Users\micha\.julia\lib\v0.5\MXNet.ji.
 in compilecache(::String) at .\loading.jl:593
 in require(::Symbol) at .\loading.jl:393

Can't get any version to run after the GPU release. Files are in the correct position so no idea what is going on!

@pluskid
Copy link
Member

pluskid commented Jun 19, 2017

Did you install the pre-built libmxnet somewhere? It seems the package cannot find the backend.

@vchuravy
Copy link
Collaborator

What is the output of Pkg.build("MXNet")?

@mikewl
Copy link
Contributor Author

mikewl commented Jul 4, 2017

Am away from pc for now but the error was with opening the dll. The file exists in the location expected after manually checking it but Libdl.find_library fails to find the library that it just downloaded.
Copy and pasting the line into the REPL fails as well - even though I can see the files are in that location being checked. I have not compiled a version of mxnet myself however or tried directly grabbing the libraries off of github yet.

@baggepinnen
Copy link

baggepinnen commented Jul 10, 2017

I can not seem to run Pkg.build("MXNet") under Ubuntu
It does a bunch of git clones, then errors with

INFO: Changing Directory to /home/fredrikb/.julia/v0.6/MXNet/deps/src/mxnet
fatal: reference is not a tree: 89de7ab20167909bc2c4f8acd397671c47cf3c0d
Unable to checkout '89de7ab20167909bc2c4f8acd397671c47cf3c0d' in submodule path 'cub'
===========================================[ ERROR: MXNet ]===========================================

LoadError: failed process: Process(`git submodule update --init`, ProcessExited(1)) [1]
while loading /home/fredrikb/.julia/v0.6/MXNet/deps/build.jl, in expression starting on line 58



julia> versioninfo()
Julia Version 0.6.0
Commit 9036443 (2017-06-19 13:05 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i5-2500K CPU @ 3.30GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Sandybridge)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.9.1 (ORCJIT, sandybridge)

Edit: found solution in #263

@mikewl
Copy link
Contributor Author

mikewl commented Jul 28, 2017

ERROR: LoadError: LoadError: Cannot find or load libmxnet.so. Please see the document on how to build it.
Stacktrace:
 [1] error(::String) at .\error.jl:21
 [2] include_from_node1(::String) at .\loading.jl:569
 [3] include(::String) at .\sysimg.jl:14
 [4] include_from_node1(::String) at .\loading.jl:569
 [5] include(::String) at .\sysimg.jl:14
 [6] anonymous at .\<missing>:2
while loading C:\Users\micha\.julia\v0.6\MXNet\src\base.jl, in expression starting on line 26
while loading C:\Users\micha\.julia\v0.6\MXNet\src\MXNet.jl, in expression starting on line 24
ERROR: Failed to precompile MXNet to C:\Users\micha\.julia\lib\v0.6\MXNet.ji.
Stacktrace:
 [1] compilecache(::String) at .\loading.jl:703
 [2] _require(::Symbol) at .\loading.jl:490
 [3] require(::Symbol) at .\loading.jl:398

is the output when trying to run using MXNet
Pkg.build("MXNet") outputs:

julia> Pkg.build("MXNet")
INFO: Building MXNet
INFO: Found a CUDA installation.
WARNING: Compat.UTF8String is deprecated, use String instead.
  likely near C:\Users\micha\.julia\v0.6\MXNet\deps\build.jl:58
WARNING: Compat.UTF8String is deprecated, use String instead.
  likely near C:\Users\micha\.julia\v0.6\MXNet\deps\build.jl:58
WARNING: Compat.UTF8String is deprecated, use String instead.
  likely near C:\Users\micha\.julia\v0.6\MXNet\deps\build.jl:58
WARNING: Compat.UTF8String is deprecated, use String instead.
  likely near C:\Users\micha\.julia\v0.6\MXNet\deps\build.jl:58
WARNING: Compat.UTF8String is deprecated, use String instead.
  likely near C:\Users\micha\.julia\v0.6\MXNet\deps\build.jl:58
WARNING: Compat.UTF8String is deprecated, use String instead.
  likely near C:\Users\micha\.julia\v0.6\MXNet\deps\build.jl:58
WARNING: Compat.UTF8String is deprecated, use String instead.
  likely near C:\Users\micha\.julia\v0.6\MXNet\deps\build.jl:58
WARNING: Compat.UTF8String is deprecated, use String instead.
  likely near C:\Users\micha\.julia\v0.6\MXNet\deps\build.jl:58
WARNING: Compat.UTF8String is deprecated, use String instead.
  likely near C:\Users\micha\.julia\v0.6\MXNet\deps\build.jl:58
WARNING: Compat.UTF8String is deprecated, use String instead.
  likely near C:\Users\micha\.julia\v0.6\MXNet\deps\build.jl:58
WARNING: Compat.UTF8String is deprecated, use String instead.
  likely near C:\Users\micha\.julia\v0.6\MXNet\deps\build.jl:58
WARNING: Compat.UTF8String is deprecated, use String instead.
  likely near C:\Users\micha\.julia\v0.6\MXNet\deps\build.jl:58
WARNING: Compat.UTF8String is deprecated, use String instead.
  likely near C:\Users\micha\.julia\v0.6\MXNet\deps\build.jl:58
WARNING: Compat.UTF8String is deprecated, use String instead.
  likely near C:\Users\micha\.julia\v0.6\MXNet\deps\build.jl:58
WARNING: Compat.UTF8String is deprecated, use String instead.
  likely near C:\Users\micha\.julia\v0.6\MXNet\deps\build.jl:58
INFO: Downloading pre-built packages for Windows.

@vchuravy
Copy link
Collaborator

vchuravy commented Jul 28, 2017 via email

@mikewl
Copy link
Contributor Author

mikewl commented Jul 28, 2017

It's there, I added it as an edit.

julia> Pkg.build("MXNet")
INFO: Building MXNet
INFO: Found a CUDA installation.
WARNING: Compat.UTF8String is deprecated, use String instead.
  likely near C:\Users\micha\.julia\v0.6\MXNet\deps\build.jl:58
WARNING: Compat.UTF8String is deprecated, use String instead.
  likely near C:\Users\micha\.julia\v0.6\MXNet\deps\build.jl:58
WARNING: Compat.UTF8String is deprecated, use String instead.
  likely near C:\Users\micha\.julia\v0.6\MXNet\deps\build.jl:58
WARNING: Compat.UTF8String is deprecated, use String instead.
  likely near C:\Users\micha\.julia\v0.6\MXNet\deps\build.jl:58
WARNING: Compat.UTF8String is deprecated, use String instead.
  likely near C:\Users\micha\.julia\v0.6\MXNet\deps\build.jl:58
WARNING: Compat.UTF8String is deprecated, use String instead.
  likely near C:\Users\micha\.julia\v0.6\MXNet\deps\build.jl:58
WARNING: Compat.UTF8String is deprecated, use String instead.
  likely near C:\Users\micha\.julia\v0.6\MXNet\deps\build.jl:58
WARNING: Compat.UTF8String is deprecated, use String instead.
  likely near C:\Users\micha\.julia\v0.6\MXNet\deps\build.jl:58
WARNING: Compat.UTF8String is deprecated, use String instead.
  likely near C:\Users\micha\.julia\v0.6\MXNet\deps\build.jl:58
WARNING: Compat.UTF8String is deprecated, use String instead.
  likely near C:\Users\micha\.julia\v0.6\MXNet\deps\build.jl:58
WARNING: Compat.UTF8String is deprecated, use String instead.
  likely near C:\Users\micha\.julia\v0.6\MXNet\deps\build.jl:58
WARNING: Compat.UTF8String is deprecated, use String instead.
  likely near C:\Users\micha\.julia\v0.6\MXNet\deps\build.jl:58
WARNING: Compat.UTF8String is deprecated, use String instead.
  likely near C:\Users\micha\.julia\v0.6\MXNet\deps\build.jl:58
WARNING: Compat.UTF8String is deprecated, use String instead.
  likely near C:\Users\micha\.julia\v0.6\MXNet\deps\build.jl:58
WARNING: Compat.UTF8String is deprecated, use String instead.
  likely near C:\Users\micha\.julia\v0.6\MXNet\deps\build.jl:58
INFO: Downloading pre-built packages for Windows.

@iblislin
Copy link
Member

Hmm... the downloading failed?

Anything under the deps/usr/... of Pkg.dir("MXNet") ?

@mikewl
Copy link
Contributor Author

mikewl commented Jul 30, 2017

all the files you would expect!

@iblislin
Copy link
Member

OMG, I installed a Windows VM and I can reproduce this issue....

@iblislin
Copy link
Member

ha, I got it.
You need to install the vc14 redistributable runtime https://www.microsoft.com/en-us/download/details.aspx?id=48145

@iblislin
Copy link
Member

Then, just using MXNet.

@mikewl
Copy link
Contributor Author

mikewl commented Jul 30, 2017

😠 unfortunately this looks like it is going to be a problem with CUDA. CUDA requires that you already have vc14 installed. So when trying to install the redist it errors and the log file tells me nicely Error 0x80070666: Cannot install a product when a newer version is installed.

@mikewl
Copy link
Contributor Author

mikewl commented Jul 30, 2017

I'm going to try compiling MXNet myself and see what happens

@iblislin
Copy link
Member

iblislin commented Jul 30, 2017

Is there a way to list the dependencies of a DLL on windows?
On Linux, we have ldd.

Could you check that on your MXNet/deps/usr/lib/libmxnet.dll?
(I just left my lab and the remote access of vm is not availabe now)

@vchuravy
Copy link
Collaborator

Is this still an issue?

@ylxdzsw
Copy link

ylxdzsw commented Nov 19, 2017

Yes, I have exactly the same problem even after removing my vc14 redistributable runtime and reinstalling using @iamed2 's link.

@iblislin
Copy link
Member

@ylxdzsw which version of MXNet.jl?

@ylxdzsw
Copy link

ylxdzsw commented Nov 19, 2017

Both v0.3.0 and latest master have the same problem.

I tried ldd comes with mingw and it shows the following output:

PS C:\Users\ylxdzsw\.julia\v0.6\MXNet\deps\usr\build> ldd .\libmxnet.dll
        ntdll.dll => /c/WINDOWS/SYSTEM32/ntdll.dll (0x7ffb23b00000)
        KERNEL32.DLL => /c/WINDOWS/System32/KERNEL32.DLL (0x7ffb210e0000)
        KERNELBASE.dll => /c/WINDOWS/System32/KERNELBASE.dll (0x7ffb20020000)
        ConEmuHk64.dll => /c/Program Files/ConEmu/ConEmu/ConEmuHk64.dll (0x7e110000)
        USER32.dll => /c/WINDOWS/System32/USER32.dll (0x7ffb23530000)
        win32u.dll => /c/WINDOWS/System32/win32u.dll (0x7ffb20f00000)
        GDI32.dll => /c/WINDOWS/System32/GDI32.dll (0x7ffb21470000)
        gdi32full.dll => /c/WINDOWS/System32/gdi32full.dll (0x7ffb20d60000)
        msvcp_win.dll => /c/WINDOWS/System32/msvcp_win.dll (0x7ffb1ff80000)
        ucrtbase.dll => /c/WINDOWS/System32/ucrtbase.dll (0x7ffb20c60000)
        IMM32.DLL => /c/WINDOWS/System32/IMM32.DLL (0x7ffb23a40000)
        combase.dll => /c/WINDOWS/System32/combase.dll (0x7ffb236c0000)
        RPCRT4.dll => /c/WINDOWS/System32/RPCRT4.dll (0x7ffb212b0000)
        bcryptPrimitives.dll => /c/WINDOWS/System32/bcryptPrimitives.dll (0x7ffb1ff00000)
        ??? => ??? (0x7ffad68d0000)
        ??? => ??? (0x7ffafda80000)
        ??? => ??? (0x6d7c0000)
        ??? => ??? (0x7ffb233c0000)
        ??? => ??? (0x4560000)
        ??? => ??? (0x7ffb22b00000)
        ??? => ??? (0x7ffad0e10000)
        ??? => ??? (0x7ffad35f0000)
        ??? => ??? (0x7ffac3530000)
        ??? => ??? (0x7ffb214a0000)
        ??? => ??? (0x7ffb20a40000)
        ??? => ??? (0x7ffb21190000)
        ??? => ??? (0x7ffb20290000)
        ??? => ??? (0x7ffac2690000)
        ??? => ??? (0x7ffb22aa0000)
        ??? => ??? (0x7ffb1fec0000)
        ??? => ??? (0x7ffb1fe70000)
        ??? => ??? (0x7ffb1fe50000)
        ??? => ??? (0x7ffb22ea0000)
        ??? => ??? (0x7ffac4260000)

I don't know what ??? means, but it seems it doesn't show the CUDA related dlls and those in the 3rdparty folder inside mxnet. I'm sure all these things are in my Path variable.

@iblislin
Copy link
Member

Here is mine from a windows VM ( without CUDA)

user@user-PC MINGW64 ~/.julia/v0.6/MXNet/deps/usr/lib ((v0.3.0))
$ ldd ./libmxnet.dll
        ntdll.dll => /c/Windows/SYSTEM32/ntdll.dll (0x77930000)
        kernel32.dll => /c/Windows/system32/kernel32.dll (0x77810000)
        KERNELBASE.dll => /c/Windows/system32/KERNELBASE.dll (0x7fefdbc0000)
        libopenblas.dll => /c/Users/user/.julia/v0.6/MXNet/deps/usr/lib/libopenblas.dll (0x6d7c0000)
        ADVAPI32.dll => /c/Windows/system32/ADVAPI32.dll (0x7feffb60000)
        msvcrt.dll => /c/Windows/system32/msvcrt.dll (0x7fefdc50000)
        sechost.dll => /c/Windows/SYSTEM32/sechost.dll (0x7fefe180000)
        RPCRT4.dll => /c/Windows/system32/RPCRT4.dll (0x7fefdd40000)
        libgcc_s_seh-1.dll => /c/Users/user/.julia/v0.6/MXNet/deps/usr/lib/libgcc_s_seh-1.dll (0x61440000)
        libgfortran-3.dll => /c/Users/user/.julia/v0.6/MXNet/deps/usr/lib/libgfortran-3.dll (0x420000)
        libquadmath-0.dll => /c/Users/user/.julia/v0.6/MXNet/deps/usr/lib/libquadmath-0.dll (0x6cf00000)
        USER32.dll => /c/Windows/system32/USER32.dll (0x77710000)
        GDI32.dll => /c/Windows/system32/GDI32.dll (0x7feff0e0000)
        LPK.dll => /c/Windows/system32/LPK.dll (0x7feffad0000)
        USP10.dll => /c/Windows/system32/USP10.dll (0x7fefe0b0000)
        VCOMP140.DLL => /c/Users/user/.julia/v0.6/MXNet/deps/usr/lib/VCOMP140.DLL (0x7feef830000)
        IMM32.DLL => /c/Windows/system32/IMM32.DLL (0x7fefefd0000)
        MSCTF.dll => /c/Windows/system32/MSCTF.dll (0x7fefdfa0000)
        api-ms-win-core-synch-l1-2-0.DLL => /c/Windows/system32/api-ms-win-core-synch-l1-2-0.DLL (0x7fefc870000)

Is there VCOMP140.DLL for you? If not, maybe rm the deps/usr and Pkg.build("MXNet") again could help.

@ylxdzsw
Copy link

ylxdzsw commented Nov 19, 2017

I have a vcomp140.dll in the deps/usr/3rdparty/vc and another one in Windows/system32. Both path are in my $Path but it seems it doesn't find either.

Well, there is yet another one in deps/usr/lib too, and this is also in my $Path

@iblislin
Copy link
Member

Will this throw an error on your machine?

julia> Libdl.dlopen(Pkg.dir("MXNet", "deps", "usr", "lib", "vcomp140.dll"))

Also, you can poke other dlls via dlopen to figure out which is the problematic one.

@iblislin
Copy link
Member

e.g. here is mine

julia> Libdl.dlopen(Pkg.dir("MXNet", "deps", "usr", "lib", "vcomp140.dll"))
Ptr{Void} @0x000007fef7800000

julia> Libdl.dlopen(Pkg.dir("MXNet", "deps", "usr", "lib", "libgfortran-3.dll"))

Ptr{Void} @0x0000000016120000

@ylxdzsw
Copy link

ylxdzsw commented Nov 19, 2017

Well, unfortunately, it seems that Julia can find all of them:

julia> Libdl.dlopen("ntdll.dll")
Ptr{Void} @0x00007ffb23b00000

julia> Libdl.dlopen("kernel32.dll")
Ptr{Void} @0x00007ffb210e0000

julia> Libdl.dlopen("KERNELBASE.dll")
Ptr{Void} @0x00007ffb20020000

julia> Libdl.dlopen("libopenblas.dll")
Ptr{Void} @0x000000003c150000

julia> Libdl.dlopen("ADVAPI32.dll")
Ptr{Void} @0x00007ffb233c0000

julia> Libdl.dlopen("msvcrt.dll")
Ptr{Void} @0x00007ffb213d0000

julia> Libdl.dlopen("sechost.dll")
Ptr{Void} @0x00007ffb22b00000

julia> Libdl.dlopen("RPCRT4.dll")
Ptr{Void} @0x00007ffb212b0000

julia> Libdl.dlopen("libgcc_s_seh-1.dll")
Ptr{Void} @0x0000000061440000

julia> Libdl.dlopen("libgfortran-3.dll")
Ptr{Void} @0x000000006f600000

julia> Libdl.dlopen("libquadmath-0.dll")
Ptr{Void} @0x000000006cf00000

julia> Libdl.dlopen("USER32.dll")
Ptr{Void} @0x00007ffb23530000

julia> Libdl.dlopen("GDI32.dll")
Ptr{Void} @0x00007ffb21470000

julia> Libdl.dlopen("LPK.dll")
Ptr{Void} @0x0000000180000000

julia> Libdl.dlopen("USP10.dll")
Ptr{Void} @0x00007ffaf70d0000

julia> Libdl.dlopen("VCOMP140.DLL")
Ptr{Void} @0x00007ffb0d7a0000

julia> Libdl.dlopen("IMM32.DLL")
Ptr{Void} @0x00007ffb23a40000

julia> Libdl.dlopen("MSCTF.dll")
Ptr{Void} @0x00007ffb22cb0000

julia> Libdl.dlopen("api-ms-win-core-synch-l1-2-0.DLL")
Ptr{Void} @0x00007ffb20020000

@iblislin
Copy link
Member

and.... what happened for dlopen libmxnet.dll ?

@ylxdzsw
Copy link

ylxdzsw commented Nov 19, 2017

My problem was solved by installing CUDA 8.0 😂. My previous installation is CUDA 9.0 since CUDA 8.0 reports that I'm using a newer hardware than the bundled driver. After install CUDA 8.0, it works well.

This is correct ldd output now:

PS C:\Users\ylxdzsw\.julia\v0.6\MXNet\deps\usr\build> ldd .\libmxnet.dll
        ntdll.dll => /c/WINDOWS/SYSTEM32/ntdll.dll (0x7ffaa3110000)
        KERNEL32.DLL => /c/WINDOWS/System32/KERNEL32.DLL (0x7ffaa2590000)
        KERNELBASE.dll => /c/WINDOWS/System32/KERNELBASE.dll (0x7ffaa0120000)
        ConEmuHk64.dll => /c/Program Files/ConEmu/ConEmu/ConEmuHk64.dll (0x7e110000)
        USER32.dll => /c/WINDOWS/System32/USER32.dll (0x7ffaa0d80000)
        win32u.dll => /c/WINDOWS/System32/win32u.dll (0x7ffa9fe30000)
        GDI32.dll => /c/WINDOWS/System32/GDI32.dll (0x7ffaa08a0000)
        gdi32full.dll => /c/WINDOWS/System32/gdi32full.dll (0x7ffaa0390000)
        msvcp_win.dll => /c/WINDOWS/System32/msvcp_win.dll (0x7ffa9fe50000)
        ucrtbase.dll => /c/WINDOWS/System32/ucrtbase.dll (0x7ffa9ffa0000)
        IMM32.DLL => /c/WINDOWS/System32/IMM32.DLL (0x7ffaa28c0000)
        combase.dll => /c/WINDOWS/System32/combase.dll (0x7ffaa2d10000)
        RPCRT4.dll => /c/WINDOWS/System32/RPCRT4.dll (0x7ffaa2b90000)
        bcryptPrimitives.dll => /c/WINDOWS/System32/bcryptPrimitives.dll (0x7ffaa00a0000)
        cudart64_80.dll => /c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v8.0/bin/cudart64_80.dll (0x7ffa7a180000)
        libopenblas.dll => /c/Users/ylxdzsw/.julia/v0.6/MXNet/deps/usr/lib/libopenblas.dll (0x6d7c0000)
        ADVAPI32.dll => /c/WINDOWS/System32/ADVAPI32.dll (0x7ffaa2ae0000)
        msvcrt.dll => /c/WINDOWS/System32/msvcrt.dll (0x7ffaa0f80000)
        sechost.dll => /c/WINDOWS/System32/sechost.dll (0x7ffaa0f10000)
        cublas64_80.dll => /c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v8.0/bin/cublas64_80.dll (0x7ffa540c0000)
        curand64_80.dll => /c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v8.0/bin/curand64_80.dll (0x7ffa50de0000)
        nvcuda.dll => /c/WINDOWS/SYSTEM32/nvcuda.dll (0x7ffa42660000)
        SHELL32.dll => /c/WINDOWS/System32/SHELL32.dll (0x7ffaa10d0000)
        cfgmgr32.dll => /c/WINDOWS/System32/cfgmgr32.dll (0x7ffaa0530000)
        shcore.dll => /c/WINDOWS/System32/shcore.dll (0x7ffaa1020000)
        nvrtc64_80.dll => /c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v8.0/bin/nvrtc64_80.dll (0x7ffa43390000)
        windows.storage.dll => /c/WINDOWS/System32/windows.storage.dll (0x7ffa9f6e0000)
        shlwapi.dll => /c/WINDOWS/System32/shlwapi.dll (0x7ffaa08d0000)
        kernel.appcore.dll => /c/WINDOWS/System32/kernel.appcore.dll (0x7ffa9f4f0000)
        cudnn64_7.dll => /c/Users/ylxdzsw/.julia/v0.6/MXNet/deps/usr/lib/cudnn64_7.dll (0x7ffa44230000)
        powrprof.dll => /c/WINDOWS/System32/powrprof.dll (0x7ffa9f4a0000)
        profapi.dll => /c/WINDOWS/System32/profapi.dll (0x7ffa9f460000)
        SETUPAPI.dll => /c/WINDOWS/System32/SETUPAPI.dll (0x7ffaa0930000)
        VCOMP140.DLL => /c/WINDOWS/SYSTEM32/VCOMP140.DLL (0x7ffa79ef0000)
        libgcc_s_seh-1.dll => /mingw64/bin/libgcc_s_seh-1.dll (0x61440000)
        VERSION.dll => /c/WINDOWS/SYSTEM32/VERSION.dll (0x7ffa997c0000)
        cufft64_80.dll => /c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v8.0/bin/cufft64_80.dll (0x7ffa39af0000)
        cusolver64_80.dll => /c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v8.0/bin/cusolver64_80.dll (0x7ffa36950000)
        dbghelp.dll => /c/WINDOWS/SYSTEM32/dbghelp.dll (0x7ffa92bf0000)
        libgfortran-3.dll => /c/Program Files/Julia/bin/libgfortran-3.dll (0x4930000)
        nvfatbinaryLoader.dll => /c/WINDOWS/SYSTEM32/nvfatbinaryLoader.dll (0x7ffa36890000)
        libwinpthread-1.dll => /mingw64/bin/libwinpthread-1.dll (0x64940000)
        WS2_32.dll => /c/WINDOWS/System32/WS2_32.dll (0x7ffaa2520000)
        libgfortran-3.dll => /c/Program Files/Julia/bin/libgfortran-3.dll (0x4930000)
        libquadmath-0.dll => /c/Program Files/Julia/bin/libquadmath-0.dll (0x6cf00000)

@iblislin
Copy link
Member

🎉 I will add that note on release page.

@iblislin
Copy link
Member

I also think CUDA 9.0 is on master only.

@ylxdzsw
Copy link

ylxdzsw commented Nov 19, 2017

I think we can check the CUDA version here since we call nvcc --version already.

Finally, thanks!

@iblislin
Copy link
Member

what does the output of nvcc --version look like on your machine?

@ylxdzsw
Copy link

ylxdzsw commented Nov 19, 2017

Now it's

PS C:\Users\ylxdzsw\.julia\v0.6\MXNet\deps\usr\build> nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2016 NVIDIA Corporation
Built on Mon_Jan__9_17:32:33_CST_2017
Cuda compilation tools, release 8.0, V8.0.60

It may be changed in CUDA 9.0, but anyway there won't be release 8.0 on other versions...

@iblislin
Copy link
Member

ok, seems its output format is same as Linux.
I will make a PR later.

@traderd65
Copy link

Julia V0.62 on Windows 10, after running "Pkg.add("MXNet")" in the console, I get the following errors. Is there another step necessary (e.g. to copy libmxnet.dll from .julia\v0.6\MXNet\deps\usr\build to somewhere?)

julia> using MXNet
INFO: Precompiling module MXNet.
ERROR: LoadError: LoadError: Cannot find or load libmxnet.dll. Please see the document on how to build it.
Stacktrace:
[1] error(::String) at .\error.jl:21
[2] include_from_node1(::String) at .\loading.jl:576
[3] include(::String) at .\sysimg.jl:14
[4] include_from_node1(::String) at .\loading.jl:576
[5] include(::String) at .\sysimg.jl:14
[6] anonymous at .<missing>:2
while loading C:\Users\doron.julia\v0.6\MXNet\src\base.jl, in expression starting on line 38
while loading C:\Users\doron.julia\v0.6\MXNet\src\MXNet.jl, in expression starting on line 25
ERROR: Failed to precompile MXNet to C:\Users\doron.julia\lib\v0.6\MXNet.ji.
Stacktrace:
[1] compilecache(::String) at .\loading.jl:710
[2] _require(::Symbol) at .\loading.jl:497
[3] require(::Symbol) at .\loading.jl:405

@traderd65
Copy link

traderd65 commented Jan 22, 2018

julia> Libdl.dlopen(Pkg.dir("MXNet", "deps", "usr", "lib", "opencv_ffmpeg320_64.dll"))
Ptr{Void} @0x0000000039640000

julia> Libdl.dlopen(Pkg.dir("MXNet", "deps", "usr", "lib", "libmxnet.dll"))
ERROR: could not load library "C:\Users\doron\.julia\v0.6\MXNet\deps\usr\lib\libmxnet.dll"
The specified module could not be found.


Since libmxnet.dll is present in the directory, that means the DLL is corrupted or...?

@iblislin
Copy link
Member

or there are some dependencies missing.
Do you have ldd available (maybe in Cygwin)?

@iblislin
Copy link
Member

please check the dependencies like this: #257 (comment)

@traderd65
Copy link

Attached the output of Dependency Checker on libmxnet.dll (a lot of these warnings are likely spurious)

libmxnetDLL-dependency-walker-output.txt

@iblislin
Copy link
Member

What's your cuda version?
Perhaps you can try to switch it #257 (comment)

@traderd65
Copy link

Using the latest CUDA version (9.1)

@iblislin
Copy link
Member

MXNet.jl v0.3 only works with CUDA 8.0, and CUDA 9 or later need master build.

@traderd65
Copy link

I'm currently building from latest versions of OpenBLAS (https://sourceforge.net/projects/openblas/files/v0.2.20/), OpenCV (pre-built WinPack, VC15, https://github.com/opencv/opencv/releases/tag/3.4.0) and CUDA/CUDnn (9.1)

@traderd65
Copy link

traderd65 commented Jan 27, 2018

Running cmake -G "Visual Studio 15 2017" in a newly added build folder under C:\MXNet produces the following in CMakeError.log (under build\CMakeFiles folder), which looks to be attempting a 32-bit rather than a 64-bit compilation of CUDA 9.1?

C:\MXNet\build\CMakeFiles\3.10.2\CompilerIdCUDA>"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.1\bin\nvcc.exe" -gencode=arch=compute_30,code="sm_30,compute_30" --use-local-env --cl-version 2017 -ccbin "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.12.25827\bin\HostX64\x86" -x cu -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.1\include" -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.1\include" -G --keep-dir Debug -maxrregcount=0 --machine 32 --compile -cudart static -g -D_MBCS -Xcompiler "/EHsc /W0 /nologo /Od /FS /Zi /RTC1 /MDd " -o Debug\CMakeCUDACompilerId.cu.obj "C:\MXNet\build\CMakeFiles\3.10.2\CompilerIdCUDA\CMakeCUDACompilerId.cu"
nvcc fatal : 32 bit compilation is only supported for Microsoft Visual Studio 2013 and earlier

@iblislin
Copy link
Member

iblislin commented Jan 27, 2018

I'm not a Windows master, but I think there is a flag --machine 32 in your post. Could you switch it?

By the way, if your are okay with prebuilt binary, here it is. https://github.com/yajiedesign/mxnet/releases

@traderd65
Copy link

traderd65 commented Jan 27, 2018

I'm likely still struggling to figure out where to make setting changes. Is it in the cmake command? in the CMakeLists.txt file (I only changed there 2 lines to reference CUDA version 9.1 since the original file didn't):

set(__cuda_toolset "7.5" "8.0" "9.0")
set(CUDA_TOOLSET "8.0" CACHE STRING "Select CUDA Version.")

Perhaps I should only change CMakeCache.txt in the build folder?

I'm not ready to give up just yet on compilation of MXNet from source since all the dependent components (OpenBLAS, OpenCV, CUDA) are the latest and seem to compile just fine. Let MXNet not be the obstacle to success... I feel like I'm close. Attaching here the relevant files.

CMakeLists.txt
CMakeCache.txt
CMakeError.log
CMakeOutput.log

@iblislin
Copy link
Member

From your post,

-ccbin "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.12.25827\bin\HostX64\x86"

Is your VC 32-bit or 64-bit version?

@traderd65
Copy link

This is interesting. Visual Studio is installed under Program Files (x86), has 2 folders Hostx64 and Hostx86 under Tools. Under Hostx64 there are two folders x64 and x86 so it looks like the x64 should be used but instead x86 is used. What setting controls that selection?

@iblislin
Copy link
Member

I just googled a bit, perhaps this can help:
https://cmake.org/cmake/help/latest/variable/CMAKE_GENERATOR_PLATFORM.html

@iblislin
Copy link
Member

-DCMAKE_GENERATOR_PLATFORM=x64 I guess

@traderd65
Copy link

Progress! Now the 64-bit VC compiler is selected, but no CUDA_COMPILER defined.

C:\MXNet\build>cmake -G "Visual Studio 15 2017" -DCMAKE_GENERATOR_PLATFORM=x64 ..
-- The C compiler identification is MSVC 19.12.25834.0
-- The CXX compiler identification is MSVC 19.12.25834.0
-- The CUDA compiler identification is unknown
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.12.25827/bin/Hostx64/x64/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.12.25827/bin/Hostx64/x64/cl.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.12.25827/bin/Hostx64/x64/cl.exe
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.12.25827/bin/Hostx64/x64/cl.exe -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at CMakeLists.txt:20 (project):
No CMAKE_CUDA_COMPILER could be found.

@iblislin
Copy link
Member

set CUDA 9.1 in your CMakeLists.txt already?

@traderd65
Copy link

traderd65 commented Jan 30, 2018

#Some things have order. This must be put in front alone
mxnet_option(USE_CUDA             "Build with CUDA support"   ON)
mxnet_option(USE_OLDCMAKECUDA           "Build with old cmake cuda" OFF)
if(USE_CUDA)
  add_definitions(-DMSHADOW_USE_CUDA=1)
  IF(FIRST_CUDA AND (NOT USE_OLDCMAKECUDA))
    set(__cuda_toolset "9.1")
    set(CUDA_TOOLSET "9.1" CACHE STRING "Select CUDA Version.")
    set_property( CACHE CUDA_TOOLSET PROPERTY STRINGS "" ${__cuda_toolset} )
    set(CMAKE_GENERATOR_TOOLSET "cuda=${CUDA_TOOLSET},host=x64")
    project(mxnet C CXX CUDA)
  else()
    project(mxnet C CXX)
    set(FIRST_CUDA FALSE)
  endif()
else()
  project(mxnet C CXX)
  add_definitions(-DMSHADOW_USE_CUDA=0)
endif()

@iblislin
Copy link
Member

looks legit 🤔

@iblislin
Copy link
Member

Oh, have you tried to remove all cmake generated file then run again, like CMakeCache.txt?

@traderd65
Copy link

traderd65 commented Jan 30, 2018

Yes, deleted CMakeCache.txt and the CMakeFiles folder (both under build folder) before re-running cmake.

Noticed that CMakeCache.txt has these 2 lines:

//Host compiler to be used by nvcc
CMAKE_CUDA_HOST_COMPILER:FILEPATH=

CMakeError.log (attached) has 2 errors which point to a problem with host compiler compatibility.
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.1\include\crt\host_config.h has:

#if defined(_WIN32)

#if _MSC_VER < 1600 || _MSC_VER > 1911

#error -- unsupported Microsoft Visual Studio version! Only the versions 2012, 2013, 2015 and 2017 are supported!

while I'm using the Visual Studio Community Edition 2017 version 15.5.4 (latest)
CMakeError.log

Looks like the compatibility problem is discussed in this thread: https://devtalk.nvidia.com/default/topic/1022648/cuda-9-unsupported-visual-studio-version-error/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants