New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Anaconda Python, CUDA, cuDNN and Theano with PyCharm Install for CPU and GPU on Windows 10 #5348
Comments
Thanks for all the information. I'm pretty sure it will help other people.
…On Sun, Dec 18, 2016 at 8:22 PM, mustgoplay ***@***.***> wrote:
This installation has been a nightmare! So I took copious notes and
attempted to minimize the installs needed. I am a complete newbie python
programmer and so I figured if I could get this to work maybe it'll help
other people and maybe they'll include this in the documentation.
References:
http://deeplearning.net/software/theano/install_windows.htmlhttps://github.com/Lasagne/Lasagne/wiki/From-Zero-to-Lasagne-on-Windows-7-%2864-bit%29https://github.com/titu1994/Neural-Style-Transfer/blob/master/Guide.mdhttp://conda.pydata.org/docs/using/envs.html#https://github.com/philferriere/dlwinhttp://www.islandman93.com/2016/04/tutorial-theano-install-on-windows-7-8.html
1.
Download and install Anaconda. DO NOT install in the default directory
for now. The .theanorc file will not work if there are spaces in the
directory names pointing to the library. Future versions (after theano
0.8.2) may support spaces. Using python 3.4 allows use of nolearn (lasagne)
and pydot-ng (keras). I downloaded the 64-bit installer:
https://www.continuum.io/downloads
Optional - I did not) Downgrade to 3.4:
conda install python==3.4.4
conda update --all
2.
Create new environment for your project. The "anaconda" after the
environment name includes all the core packages:
conda create -n env_name35 anaconda python=3.5
conda create -n env_name34 anaconda python=3.4
3.
Change to the environment of choice and import theano:
conda install theano
4.
Download and run check_blas.py
https://raw.githubusercontent.com/Theano/Theano/master/
theano/misc/check_blas.py
It will say:
Total execution time: 25.74s on CPU (without direct Theano binding to
blas but with numpy/scipy binding to blas).
5.
Create a file called .theanorc and place it in the C:\Users[Your_User]
directory and have it say the following:
[global]
floatx = float32
[blas]
ldflags = -LC:\Anaconda3\Library\bin -lmkl_rt
This will bind theano to the mkl BLAS library that was included in
Anaconda that copied into your environment in step 2. mkl_rt.dll is the
library and the directory is the bin directory. It will also limit calcs to
float32 which is most of the improvement in the speed. Now when you run
check_blas you should have things faster and it should say that theano is
binding to blas:
Total execution time: 11.77s on CPU (with direct Theano binding to blas).
1.
Now we need to install the Microsoft C++ compiler. You need about 7GB
of space and can get it here:
https://www.visualstudio.com/free-developer-offers/
Download the visual studio community edition. You only need to install
Common Tools for Visual C++ 2015.
1.
Now you can install CUDA from NVidia here:
https://developer.nvidia.com/cuda-downloads
I only installed the development, visual studio integration and runtime
files.
1.
Add the following lines to .theanorc:
[nvcc]
flags=--cl-version=2015 -D_FORCE_INLINES
2.
Nvcc was unable to see the Visual Studio include and library files
with the new Kits format. To have it see the includes, add a new
environmental variable called INCLUDE by going to:
My Computer/Right-click/Properties/Advanced System
Settings/Environmental Variables/System Variables - New
Fill in the Variable Value:
C:\Program Files (x86)\Windows Kits\10\Include\10.0.14393.0\um;C:\Program Files (x86)\Windows Kits\10\Include\10.0.14393.0\ucrt
Make sure you are using the 10.0.HighestNumber
1.
While in the environmental variables section, in the system variables
find the PATH variable and add the mingw64 directory:
C:\Anaconda3\envs\corrnet35\Library\mingw-w64\bin
And make sure all of the following are also present in the path:
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\amd64
C:\Anaconda3
C:\Anaconda3\Scripts
C:\Anaconda3\Library\bin
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\bin
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\libnvvp
1.
The only way I could get Nvcc to see the library files was to copy
them into a new directory near the other lib files. This is what worked for
me but if someone can get the libraries in C:\Program Files (x86)\Windows
Kits\10\Lib\10.0.14393.0\ to be seen with either environmental variables or
in nvcc.profile that would be better:
i) Create a new directory lib2 under C:\Program Files\NVIDIA GPU
Computing Toolkit\CUDA\v8.0
ii) Create a subirectory x64 under C:\Program Files\NVIDIA GPU
Computing Toolkit\CUDA\v8.0\lib2
iii) Copy all the files from:
C:\Program Files (x86)\Windows Kits\10\Lib\10.0.14393.0\ucrt\x64
C:\Program Files (x86)\Windows Kits\10\Lib\10.0.14393.0\um\x64
into:
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\lib2\x64
Note: you could directly copy these into the original lib directory
but I kept them separate to be able to easily replace/delete them.
iv) Right-click on Notepad and Run-As-Administrator. Open:
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\bin\nvcc.
profile
After the first LIBRARIES line add:
LIBRARIES =+ $(*SPACE*) "/LIBPATH:$(TOP)/lib2/$(*WIN_PLATFORM*)"
2.
Restart the computer to allow the path changes created by the install
to take effect. Now when you run check_blas it should be faster:
Total execution time: 9.83s on CPU (with direct Theano binding to
blas).
3.
Now add the g++ information to .theanorc in the global flag cxx and
the gcc sections. Mingw-64w was included in the theano install so all you
have to do is link it. You can also throw in the fast_run mode. The
.theanorc file should now look like this:
[global]
floatx = float32
cxx = C:\Anaconda3\envs\corrnet35\Library\mingw-w64\bin\g++.exe
mode = FAST_RUN
[blas]
ldflags = -LC:\Anaconda3\Library\bin -lmkl_rt
[gcc]
cxxflags = -LC:\Anaconda3\envs\corrnet35\Library\mingw-w64\include
-LC:\Anaconda3\envs\corrnet35\Library\mingw-w64\lib -lm
[nvcc]
flags=--cl-version=2015 -D_FORCE_INLINES
This doesn't seem to make any difference in the speed but every
installation guide recommends setting up the g++ link.
1.
Update .theanorc to change global seeting for the device to gpu
instead of cpu. I.e. change it to read:
[global]
floatx = float32
cxx = C:\Anaconda3\envs\corrnet35\Library\mingw-w64\bin\g++.exe
mode = FAST_RUN
device = gpu
Now when you run it you'll see another big increase in speed:
Total execution time: 0.75s on GPU.
NOTE: With the gpu being used I enabled cnMEM with 80% usage by adding
this to .theanorc:
[lib]
cnmem=0.80
and tried the cuBLAS library by copying the file cublas64_80.dll into the
\Anaconda3\Library\bin directory and using that instead of mklBLAS
(mkl_rt.dll) and changing the blas line in .theanorc to:
[blas]
ldflags = -LC:\Anaconda3\Library\bin -lcublas64_80
both of which are supposed to increase performance but I got no
improvement in speed at all. Intel says cuBLAS should be 6-17x faster so
I'm just leaving it that way in case it makes a difference for bigger
networks.
I also signed up for as an NVidia developer and downloaded cuDNN, copied
the files into the appropriate directories in the toolkit and again did not
find any improvement in speed for check_blas but the library is mainly for
convnets.
1.
Install PyCharm
https://www.jetbrains.com/pycharm/download/?gclid=
CIaQwY2q8tACFZ26wAod-ScPwA&gclsrc=aw.ds.ds&dclid=
CJC7042q8tACFRO6TwodaKAAAA#section=windows
<https://www.jetbrains.com/pycharm/download/?gclid=CIaQwY2q8tACFZ26wAod-ScPwA&gclsrc=aw.ds.ds&dclid=CJC7042q8tACFRO6TwodaKAAAA#section=windows>
2.
Open PyCharm, on the bottom right press Configure/Settings and then go
to Project Interpreter. Press the down arrow next to Project Interpreter
and press Show All. Press the + sign and Add Local. Navigate to and select
python.exe in your virtual environment:
C:\Anaconda3\envs\env_name\python.exe
3.
Install keras and/or lasagne and program away. If the keras download
breaks check_blas.py then update all the libraries with:
conda upgrade --all
Good luck!!
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#5348>, or mute the thread
<https://github.com/notifications/unsubscribe-auth/AALC-xeaZ6Uu9PyMI76xx78XdMvmnVeeks5rJdxZgaJpZM4LQS82>
.
|
I think we should update the website with this post, because it actually described every single problem i hit upon my installation (and then some), and fixes every single one with surgical precision. Thank you so much 👍 |
Thank you. I'm glad it's helping. I fine tuned it a little in case it gets included by listing more errors that I've run into or read about and some formatting and directory info. |
Please let me know when/if this gets included so I can stop OCDng and updating. I added a new error for CNMeM running out of memory and rearranged that section a little, updated the title, added a reference and clarified the difference between a cpu and gpu install and added old errors I ran into. Lucked out with a new post that told me the environmental variable for cuda so no need to mess with nvcc profile now. (https://groups.google.com/forum/#!topic/theano-users/_Rl_UeQ7FxU) so a much cleaner install. Only .theanorc and path/environmental variables need to be made now. I think it's finally done now. |
A great help! Thank you very much! |
I could kiss you right now :D |
Following the given step by step tutorial I was (almost) able to install theano with C compiler and full GPU support on Windows 10 for python 3.6. Thank you very much for these crucial hints! In addition I had to do some additional adaptations:
Important to Note:
|
@schlichtanders , thanks for the update for 3.6. I haven't tried it but other installations also recommended the SDK install but it wasn't needed when I used the above method for 3.5 so I went for the minimum install. I think you might actually be able to get them from the install in step 6 when you customize the install. It's weird that it's required now but glad you figured it out. |
For now, the latest cuda 8.0 has supported VS2015 under windows and gcc 5.3 under ubuntu according to their online document. And Ray Donnelly has kindly uploaded his theano 0.9.0rc1 version onto anaconda cloud. To install this package with conda run:
It's same on Ubuntu 16.04. |
Can you try to just run |
Great info! I think i've been going through the same pain, trying various combinations of python version, compilers, etc. Any chance of an uplift of the Theano docs to reflect guidance for windows? http://deeplearning.net/software/theano/install_windows.html#install-windows |
We are waiting for the conda packages to be done for this. You can use the
conda 0.9rc4 packages with this command from memory:
conda install -c rdonnelly Theano
On Tue, Mar 21, 2017 at 2:45 PM Peter Coghill <notifications@github.com> wrote:
Great info! I think i've been going through the same pain, trying various
combinations of python version, compilers, etc.
Any chance of an uplift of the Theano docs to reflect guidance for windows?
http://deeplearning.net/software/theano/install_windows.html#install-windows
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#5348 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AALC-9bLbo3YZCUgB34xUM9k_t6e1x8yks5roBq_gaJpZM4LQS82>
.
|
@nouiz Theano seemed to work with your suggestion above. Tests mostly passed. But continued to have difficultes with PyMC3 which I was hoping to use theano for. Found this guide also. Currenly following. https://github.com/philferriere/dlwin |
The guide you link uses Theano 0.8.2 and the old GPU back-end, though. |
One of your References links has gone stale: the Tutorial: Theano Install on ... |
For the insatallation, I would recommand that people use the conda versoin on windows: conda install theano pygpu I don't know how that interfere with pycharm. No need to install VS anymore. Also see https://github.com/Theano/Theano/wiki/Converting-to-the-new-gpu-back-end%28gpuarray%29 to use the new gpu back-end |
Hi All, AssertionError: AbstractConv2d Theano optimization failed: there is no implementation available supporting the requested options. Did you exclude both "conv_dnn" and "conv_gemm" from the optimizer? If on GPU, is cuDNN available and does the GPU support it? If on CPU, do you have a BLAS library installed Theano can link against? |
I'm not really sure what I'm doing wrong, but there's no errors and yet it shows "Total execution time: 0.00" https://gist.github.com/darthdeus/d96cd056ab7725d42676f2882dc3918b |
Made a similar install on Win 10 relying on similar web tutorials, it was hard actually. All works well now but cuDNN, which is there but not compiling. Getting this error when trying the usual benchmark code:
It should be something messed up with the .theanorc file, many indecisive tweaking attempts already, so asking for help. Any advice? Thanks!
|
Hi,
I strongly recommand that you update to Theano dev version and use the new
GPU back-end. It make the installation much simpler on Windows:
https://github.com/Theano/Theano/wiki/Converting-to-the-new-gpu-back-end(gpuarray)
…On Thu, May 18, 2017 at 9:01 AM PippoIT17 ***@***.***> wrote:
Made a similar install on Win 10 relying on similar web tutorials, it was
hard actually. All works well now but cuDNN, which is there but not
compiling. Getting this error when trying the usual benchmark code:
Using gpu device 0: GeForce GTX 1070 (CNMeM is enabled with initial size: 80.0% of memory, cuDNN Can not compile with cuDNN. We got this error:
b'')
ERROR (theano.gof.opt): SeqOptimizer apply <theano.sandbox.cuda.dnn.NoCuDNNRaise object at 0x000002757D4E4080>
ERROR (theano.gof.opt): Traceback:
ERROR (theano.gof.opt): Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\lib\site-packages\theano\gof\opt.py", line 235, in apply
sub_prof = optimizer.optimize(fgraph)
File "C:\ProgramData\Anaconda3\lib\site-packages\theano\gof\opt.py", line 87, in optimize
ret = self.apply(fgraph, *args, **kwargs)
File "C:\ProgramData\Anaconda3\lib\site-packages\theano\sandbox\cuda\dnn.py", line 3479, in apply
if not dnn_available():
File "C:\ProgramData\Anaconda3\lib\site-packages\theano\sandbox\cuda\__init__.py", line 360, in dnn_available
dnn_available.msg)
RuntimeError: You enabled cuDNN, but we aren't able to use it: Can not compile with cuDNN. We got this error:
b''
It should be something messed up with the .theanorc file, many indecisive
tweaking attempts already, so asking for help. Any advice? Thanks!
[global]
device = gpu
floatX = float32
cuda.disable_gcc_cudnn_check=True
optimizer_including=cudnn
[cuda]
root = C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0
[nvcc]
flags=-D_FORCE_INLINES
fastmath=True
compiler_bindir=C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\amd64
optimizer_including=cudnn
[dnn]
enabled = True
[lib]
cnmem=0.8
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#5348 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AALC-xdz_spWBkO3sawqfEtrP7Q8_DVyks5r7EE1gaJpZM4LQS82>
.
|
Hi, Many thanks for your reply. I have that already from my installation and it is up to date, but if I use
Is that related to my Anaconda using Python 3.6 ? I can read here and there this backend supports up to 3.5? |
I had some trouble with 3.6, sadly. I had better luck with 3.5, although I cannot remember anymore which component did not like 3.6. I also have read conflicting reports about visual studio 2015, as in CUDA 8 was released with support for it, but Microsoft may have changed something that hurt compatibility after the fact? I went with the ancient visual studio 2013 for this reason. (although it was humorous to see the "fabulous new" technologies they were pushing in this version. Silverlight anyone???) Oh how I wish there was just a more minimal compiler built into Windows than the bloated ecosystem of VS, which i know nothing about nor care to. |
This is my .theanorc, and I can currently use CUDA 8 on Windows 10 & VS 2013, if it is any help. Mostly got here through trial and error, so this might not be optimal setup: [global] [nvcc] [cuda] [lib] [dnn] |
This instruction where for the old back-end and don't work for the new back-end. So closing. Thanks for the instruction, it helped many people when it was useful. |
It is already failing with the theano part: Proceed ([y]/n)? Fetching packages ... libpython-2.0- 100% |###############################| Time: 0:00:00 1.27 MB/s theano- 0.8.2-p 100% |###############################| Time: 0:00:01 2.91 MB/s Extracting packages ... [ COMPLETE ]|##################################################| 100% Linking packages ... [ COMPLETE ]|##################################################| 100% (C:\Anaconda3\envs\env_name34) C:\Anaconda3>python check_blas.py Traceback (most recent call last): File "check_blas.py", line 16, in import theano ImportError: No module named 'theano' I tried and I can import theano in spyder on the console but nowhere else |
You probably have multiple Python installation and Theano was installed in only one of them. Why not use the spyder console if that work? Also, this isn't related to this issue, so don't continue the discussion here. |
Hi,
I'm using Windows 10, with Nvidia GeForce 840M with 388 driver, CUDA v 9.0. Updated the driver to the latest one (388), the error persisted. Couldn't find any driver available specifically on the page to which they point me. |
We only support Theano 1.0rc1 and up. We don't have time to support older
version. The instruction in this page is for older version. Check this page
for newer instruction and how to clean up your environment:
https://github.com/Theano/Theano/wiki/Converting-to-the-new-gpu-back-end(gpuarray)
…On Sat, Nov 4, 2017 at 4:35 PM Mateusz Jurewicz ***@***.***> wrote:
Hi,
After following every step I got the error:
Installing the nvidia driver available on the same download page as the
cuda package will fix the problem:
http://developer.nvidia.com/cuda-downloads
I'm using Windows 10, with Nvidia GeForce 840M with 388 driver, CUDA v
9.0. Updated the driver to the latest one (388), the error persisted.
Couldn't find any driver available specifically on the page to which they
point me.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
<#5348 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AALC-xWMVHZU_WX6JHgz61KymE8ZdC9pks5szMp7gaJpZM4LQS82>
.
|
You have:
[dnn]
enabled=False
This explicitly disable cudnn. If you want it, remove those 2 lines.
…On Wed, May 16, 2018 at 9:31 AM Sayantan17 ***@***.***> wrote:
Hello , guys is this right still i m getting something weird , can
somebody sort it out? The .theanorc file is the:-
[global]
floatx = float32
device = cuda
mode = FAST_RUN
cxx = C:\Users\Sayantan\Anaconda3\Library\mingw-w64\bin\g++.exe
[cuda]
enabled=True
[blas]
ldflags = -LC:\Users\Sayantan\Anaconda3\Library\bin -lcublas64_80
[gcc]
cxxflags = -LC:\Users\Sayantan\Anaconda3\Library\mingw-w64\include
-LC:Users\Sayantan\Anaconda3\Library\mingw-w64\lib -lm
[nvcc]
flags=--cl-version=2015 -D_FORCE_INLINES
[dnn]
enabled=False
[lib]
cnmem=0.70
But still not appropriate .
[image: untitled]
<https://user-images.githubusercontent.com/37222844/40119857-66093afa-593b-11e8-8be0-141268f3173f.png>
Why is it written Mapped device ........
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
<#5348 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AALC-024CEIOwUZYgG2CHUf5eoJBH70fks5tzCo-gaJpZM4LQS82>
.
|
@nouiz lemme check , will let you know! |
Anyone get this error: I'm using CUDA 9.2 and my Nvidia driver is v398.26 |
@Nomat Please don't add comments unrelated to the issue. This is not a forum. Open a new issue or post on the theano-users mailing list if you need help. Also, this is apparently a return of an old problem. The solution is probably to downgrade the driver version a bit until the current drivers are fine again. |
Hi! I was having dll load failed problem and I tryied to use the suggestion on this post. I've just adapted for my environment ( I didnt create a virtual environmnet on conda. I just run it from default) and I got two error popping up while running this code below:
the errors were:
and
How can I fix it? `[global] [blas] [gcc] [dnn] [lib] What can I do? |
@lucas-figueiredo-m Once again, this is not a forum and your problem is unrelated to this issue. Either your python installation is bad or the compile flags you set interfere with the compilation. Remove your .theanorc and try again. If that doesn't fix it, try reinstalling anaconda from scratch. |
This installation has been a nightmare! So I took copious notes and attempted to minimize the installs needed. I am a complete newbie python programmer and so I figured if I could get this to work maybe it'll help other people and maybe they'll include this in the documentation.
References:
Theano Documentation
Zero to Lasagne
Neural-Style-Transfer Git
Conda Managing Environments
GPU-accelerated Deep Learning on Windows 10 native Git
Tutorial: Theano install on Windows 7, 8, 10
Making Theano Faster with CuDNN and CNMeM on Windows 10
Download and install Anaconda. DO NOT install in the default directory for now. The .theanorc file WILL NOT WORK IF THERE ARE SPACES in the directory names pointing to the library. Future versions (after theano 0.8.2) may support spaces. Using python 3.4 allows use of nolearn (lasagne) and pydot-ng (keras). I downloaded the 64-bit installer.:
https://www.continuum.io/downloads
Optional - (I did not try this) Downgrade to 3.4:
conda install python==3.4.4
conda update --all
Note: It is important to stay consistent with your 32 vs 64 bit install throughout the process or it may lead to errors.
conda install theano
Download check_blas.py:
https://raw.githubusercontent.com/Theano/Theano/master/theano/misc/check_blas.py
Run it by changing to the directory you saved it into within the command prompt for your new environment and then typing:
python check_blas.py
It will say:
Total execution time: 25.74s on CPU (without direct Theano binding to blas but with numpy/scipy binding to blas).
This will bind theano to the mklBLAS library that was included in Anaconda that copied into your environment in step 2. mkl_rt.dll is the library and the directory is the bin directory. Other install instructions often mention OpenBLAS but the mklBLAS is already included and is about 30% faster. Without binding to BLAS you can't use certain layers e.g. convolutional layers and you can get errors like:
AssertionError: AbstractConv2d Theano optimization failed: there is no implementation available supporting the requested options. Did you exclude both "conv_dnn" and "conv_gemm" from the optimizer? If on GPU, is cuDNN available and does the GPU support it? If on CPU, do you have a BLAS library installed Theano can link against?
It will also limit calcs to float32 which is most of the improvement in the speed. Now when you run check_blas you should have things faster and it should say that theano is binding to blas:
Total execution time: 11.77s on CPU (with direct Theano binding to blas).
Now we need to install the Microsoft C++ compiler. You can get it here:
https://www.visualstudio.com/free-developer-offers/
Download the visual studio community edition. You only need to install:
I did not install anything from the other sections including the Windows and Web Development section
Now you can install CUDA from NVidia here:
https://developer.nvidia.com/cuda-downloads
I only installed the CUDA/development, visual studio integration and runtime files sections.
if you do not include the cl-version then you get the error:
nvcc fatal : nvcc cannot find a supported version of Microsoft Visual Studio. Only the versions 2010, 2012, and 2013 are supported
the D_FORCE_INLINES part is for an Ubuntu bug although I'm not sure it's necessary anymore. It can help prevent this error:
WARNING (theano.sandbox.cuda): CUDA is installed, but device gpu0 is not available (error: cuda unavailable)
Note: This error seems to also show if the g++ version is too new for the CUDA version.
Some installs suggest having a --use-local-env flag as well but with this setup it leads to the error:
It does help with older versions of c++ that are not the Visual Studio Windows Kits version.
LINK : fatal error LNK1104: cannot open file stdio.h
and see the library files and not get the errors:
LINK : fatal error LNK1104: cannot open file libucrt.lib
and
LINK : fatal error LNK1104: cannot open file uuid.lib
Add a new environmental variable called INCLUDE by going to:
My Computer/Right-click/Properties/Advanced System Settings/Environmental Variables/System Variables - New
Fill in the Variable Value:
C:\Program Files (x86)\Windows Kits\10\Include\10.0.14393.0\um;C:\Program Files (x86)\Windows Kits\10\Include\10.0.14393.0\ucrt
Add a second new environmental variable called LIB and fill in the Variable Value:
C:\Program Files (x86)\Windows Kits\10\Lib\10.0.14393.0\ucrt\x64;C:\Program Files (x86)\Windows Kits\10\Lib\10.0.14393.0\um\x64
Make sure you are using the 10.0.HighestNumber
C:\Anaconda3\envs\corrnet35\Library\mingw-w64\bin
Or you will get a g++ error or a mingw error e.g.:
WARNING (theano.configdefaults): g++ not detected ! Theano will be unable to execute optimized C-implementations (for both CPU and GPU) and will default to Python implementations. Performance will be severely degraded. To remove this warning, set Theano flags cxx to an empty string.
And make sure all of the following are also present in the path:
Without the first line you can get the error:
nvcc : fatal error : Cannot find compiler 'cl.exe' in PATH
Total execution time: 9.83s on CPU (with direct Theano binding to blas).
CPU ONLY 13) Update .theanorc to force use of the cpu if you do not have a gpu. I.e. change it to read:
Without the force_device you can get this error:
ERROR (theano.sandbox.cuda): Failed to compile cuda_ndarray.cu: ('nvcc return status...
GPU ONLY 14) Update .theanorc to change global setting for the device to gpu instead of cpu if you have a gpu. I.e. change it to read:
Now when you run it you'll see another big increase in speed:
Total execution time: 0.75s on GPU.
GPU ONLY 15) I signed up as an NVidia developer and downloaded cuDNN, which is a set of files to increase convent speeds, and copied the files into the appropriate directories in the C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0 bin/include/lib directories. In order for cuDNN to work you need to enable it or you get the following error:
Using gpu device 0: GeForce GTX 1060 (CNMeM is disabled, CuDNN not available)
You can enable it with the following in .theanorc:
and now it should say:
Using gpu device 0: GeForce GTX 1060 (CNMeM is disabled, cuDNN 5105)
I enabled CNMeM, with 70% GPU usage by adding this to .theanorc:
And now it says:
Using gpu device 0: GeForce GTX 1060 (CNMeM is enabled with initial size: 70.0% of memory, cuDNN 5105)
Note: You don't need the dnn section if you enable CNMeM since it will automatically use DNN if both are available. The 3 options for CNMeM are:
If the percent is set too high you may get the below error. Either close open scripts or reduce the percent:
ERROR (theano.sandbox.cuda): ERROR: Not using GPU. Initialisation of device gpu failed: initCnmem: cnmemInit call failed! Reason=CNMEM_STATUS_OUT_OF_MEMORY. numdev=1
And if that is happening too often still you can just use the dnn option and skip CNMeM.
GPU ONLY 16) I also tried the cuBLAS library by copying the file cublas64_80.dll from C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\bin into the C:\Anaconda3\Library\bin directory and using that instead of mklBLAS (mkl_rt.dll) and changing the blas line in .theanorc to:
Note: Do not use this if only using cpu and not the gpu. Stick with mklBLAS (mkl_rt).
Steps 15-6 are supposed to increase performance but I got no improvement in speed at all. Intel says cuBLAS should be 6-17x faster so I'm just leaving it that way in case it makes a difference for bigger networks.
Install PyCharm
https://www.jetbrains.com/pycharm/download/?gclid=CIaQwY2q8tACFZ26wAod-ScPwA&gclsrc=aw.ds.ds&dclid=CJC7042q8tACFRO6TwodaKAAAA#section=windows
Open PyCharm, on the bottom right press Configure/Settings and then go to Project Interpreter. Press the down arrow next to Project Interpreter and press Show All. Press the + sign and Add Local. Navigate to and select python.exe in your virtual environment:
C:\Anaconda3\envs\env_name35\python.exe
If you're like me and like undo/redo buttons in a toolbar then you can check View/Toolbar and View/Tool Buttons. You can then customize the toolbar with a right-click on it and select Customize Menus and Toolbars.
conda upgrade --all
Good luck!!
For reference the final GPU .theanorc looks like this:
The final CPU .theanorc looks like this:
And the list of installation files I used is:
The text was updated successfully, but these errors were encountered: