Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CUDA error. #1444

Closed
ghost opened this issue Oct 10, 2020 · 9 comments
Closed

CUDA error. #1444

ghost opened this issue Oct 10, 2020 · 9 comments

Comments

@ghost
Copy link

ghost commented Oct 10, 2020

| _ | |
|_ |_ |_| v0.26.3-rc2 built Oct 3 2020
Detected 4 core(s) and 8 thread(s) in 1 group(s).
Group 0 has 4 core(s) and 8 thread(s).
go nodes 100
Found pb network file: C:\Program Files\ChessBase\Engines.x64\Lc0/703810.pb.gz
Creating backend [cuda-auto]...
Switching to [cuda]...
CUDA Runtime version: 11.1.0
Latest version of CUDA supported by the driver: 11.0.0
WARNING: code was compiled with unsupported CUDA version.
GPU: GeForce GTX 780 Ti
GPU memory: 3 Gb
GPU clock frequency: 1084.5 MHz
GPU compute capability: 3.5
error CUDA error: no kernel image is available for execution on the device (c:\projects\lc0\src\neural\cuda\winograd_helper.inc:563)

Using Windows 10 Pro updated, video cardo GeForce 780 Ti OC.

@mooskagh
Copy link
Member

It seems that you don't have the latest nvidia drivers, updating them will probably help.

@ghost
Copy link
Author

ghost commented Oct 15, 2020

It seems that you don't have the latest nvidia drivers, updating them will probably help.

Drivers updated, still getting error:

   _

| _ | |
|_ |_ |_| v0.26.3 built Oct 10 2020
Detected 4 core(s) and 8 thread(s) in 1 group(s).
Group 0 has 4 core(s) and 8 thread(s).
go nodes 100
Found pb network file: C:\Program Files\ChessBase\Engines.x64\Lc0/703810.pb.gz
Creating backend [cuda-auto]...
Switching to [cuda]...
CUDA Runtime version: 11.1.0
Latest version of CUDA supported by the driver: 11.1.0
GPU: GeForce GTX 780 Ti
GPU memory: 3 Gb
GPU clock frequency: 1084.5 MHz
GPU compute capability: 3.5
error CUDA error: no kernel image is available for execution on the device (c:\projects\lc0\src\neural\cuda\winograd_helper.inc:563)

@malcolmlewis
Copy link

Hi
I see the same error here on openSUSE Tumbleweed, I package an rpm on the openSUSE Build Service, then rebuild locally for Nvidia support
(https://build.opensuse.org/package/show/home:malcolmlewis:TESTING/lc0)

Nvidia driver: 455.28
Cuda: 11.1.0
Cudnn: 8.0.4
Kernel: 5.8.14-1-default
Lc0 version: 26.3+git20201010.bd81641
lc0 benchmark --weights=/data/configuration/weights
       _
|   _ | |
|_ |_ |_| v0.27.0-dev+git.unknown built Oct 16 2020
Loading weights file from: /data/configuration/weights
Creating backend [cudnn-auto]...
Switching to [cudnn]...
CUDA Runtime version: 11.1.0
Cudnn version: 8.0.4
Latest version of CUDA supported by the driver: 11.1.0
GPU: GeForce GT 710
GPU memory: 0.958313 GiB
GPU clock frequency: 954 MHz
GPU compute capability: 3.5
CUDA error: no kernel image is available for execution on the device (../src/neural/cuda/winograd_helper.inc:563) 

@malcolmlewis
Copy link

Hi
OK, figured out the cards I have (GT710) are no longer supported with CUDA 11 and lc0.

@massimilianogoi your card is in the same boat as mine....

@borg323
Copy link
Member

borg323 commented Jan 12, 2021

The cudnn windows package is made with cuda 10, so should work fine with older gpus. Linux users will either have to build with cuda 10 or modify meson.build to pass -arch=compute_35 -code=sm_35 and remove , extra_args: nvcc_extra_args from line 474. The reason is that while CC 3.5 is still supported by cuda 11, the default for code generation is CC 5.2.

@hans-ekbrand
Copy link
Contributor

It seems that you don't have the latest nvidia drivers, updating them will probably help.

Drivers updated, still getting error:

   _

| _ | |
|_ |_ |_| v0.26.3 built Oct 10 2020
Detected 4 core(s) and 8 thread(s) in 1 group(s).
Group 0 has 4 core(s) and 8 thread(s).
go nodes 100
Found pb network file: C:\Program Files\ChessBase\Engines.x64\Lc0/703810.pb.gz
Creating backend [cuda-auto]...
Switching to [cuda]...
CUDA Runtime version: 11.1.0
Latest version of CUDA supported by the driver: 11.1.0
GPU: GeForce GTX 780 Ti
GPU memory: 3 Gb
GPU clock frequency: 1084.5 MHz
GPU compute capability: 3.5
error CUDA error: no kernel image is available for execution on the device (c:\projects\lc0\src\neural\cuda\winograd_helper.inc:563)

Your card GTX 780 Ti is supported by the latest lc0, but you have choose the right download. From the output I can determine that you used the CUDA version, but you should have picked the CUDNN version https://github.com/LeelaChessZero/lc0/releases/download/v0.26.3/lc0-v0.26.3-windows-gpu-nvidia-cudnn.zip

Please retry with this version and report back so we can close this issue.

@hans-ekbrand
Copy link
Contributor

Hi
OK, figured out the cards I have (GT710) are no longer supported with CUDA 11 and lc0.

Not true, your card (and the GTX 780 ti) both support compute capability 3.5 which is enough to run lc0. Please retry the build with the arguments given by borg323 above and report back.

@malcolmlewis
Copy link

Hi
OK, figured out the cards I have (GT710) are no longer supported with CUDA 11 and lc0.

Not true, your card (and the GTX 780 ti) both support compute capability 3.5 which is enough to run lc0. Please retry the build with the arguments given by borg323 above and report back.

Hi All

@hans-ekbrand since I'm following upstream, I just procured a newer Nvidia card for testing ;) Not sure if other users are rebuilding the rpm locally for Nvidia support, but once I get some spare time will look at a build with the older card.

@borg323 thanks for the tip 👍

@ghost
Copy link
Author

ghost commented Jan 16, 2021

So, the CUDA does not work with my configuration (CPU Intel i7-4771, graphic card Nvidia GeForce GTC 780 Ti):

C:\Users\Massimiliano Goi\Desktop\lc0>lc0.exe
_
| _ | |
|_ |_ |_| v0.26.3 built Oct 10 2020
Detected 4 core(s) and 8 thread(s) in 1 group(s).
Group 0 has 4 core(s) and 8 thread(s).
go nodes 100
Found pb network file: C:\Users\Massimiliano Goi\Desktop\lc0/703810.pb.gz
Creating backend [cuda-auto]...
Switching to [cuda]...
CUDA Runtime version: 11.1.0
Latest version of CUDA supported by the driver: 11.2.0
GPU: GeForce GTX 780 Ti
GPU memory: 3 Gb
GPU clock frequency: 1084.5 MHz
GPU compute capability: 3.5
error CUDA error: no kernel image is available for execution on the device (c:\projects\lc0\src\neural\cuda\winograd_helper.inc:563)

While the CUDNN does:

C:\Users\Massimiliano Goi\Desktop\lc0>lc0.exe
_
| _ | |
|_ |_ |_| v0.26.3 built Oct 10 2020
Detected 4 core(s) and 8 thread(s) in 1 group(s).
Group 0 has 4 core(s) and 8 thread(s).
go nodes 100
Found pb network file: C:\Users\Massimiliano Goi\Desktop\lc0/703810.pb.gz
Creating backend [cudnn-auto]...
Switching to [cudnn]...
CUDA Runtime version: 10.0.0
Cudnn version: 7.4.2
Latest version of CUDA supported by the driver: 11.2.0
GPU: GeForce GTX 780 Ti
GPU memory: 3 Gb
GPU clock frequency: 1084.5 MHz
GPU compute capability: 3.5
info depth 1 seldepth 2 time 772 nodes 4 score cp 12 nps 363 tbhits 0 pv e2e4 c7c5
info depth 2 seldepth 3 time 778 nodes 13 score cp 13 nps 764 tbhits 0 pv e2e4 c7c5 g1f3
info depth 3 seldepth 4 time 784 nodes 29 score cp 13 nps 1260 tbhits 0 pv e2e4 c7c5 g1f3 e7e6
info depth 3 seldepth 5 time 796 nodes 85 score cp 13 nps 2428 tbhits 0 pv e2e4 c7c5 b1c3 b8c6 g1e2
info depth 4 seldepth 6 time 805 nodes 148 score cp 14 nps 3441 tbhits 0 pv e2e4 c7c5 b1c3 b8c6 g1f3 e7e5
bestmove e2e4 ponder c7c5

I have read the assignment by type of videocards in the download page so now I can consider solved this problem and I'm closing this issue. Sorry for the late answer but I had time issue

@ghost ghost closed this as completed Jan 16, 2021
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants