v4.5.0 is not compatible with `torch>=2.*.*+cu121`

[MahmoudAshraf97]

Hello
the last release works great with versions of pytorch that use python cuda packages, but when torch that is precompiled with cuda binaries is installed, this error appears:

Unable to load any of {libcudnn_cnn.so.9.1.0, libcudnn_cnn.so.9.1, libcudnn_cnn.so.9, libcudnn_cnn.so}
Invalid handle. Cannot load symbol cudnnCreateConvolutionDescriptor

the only solution so far is to downgrade to 4.4.0 although CuDNN v9.1 is installed both using pip and bundled with pytorch
jhj0517/Whisper-WebUI#348

Update:
as per @BBC-Esq research, ctranslate2>=4.5.0 uses CuDNN v9 which requires CUDA >= 12.3.
Since most issues occur from a conflicting torch and ctranslate2 installations these are tested working combinations:

Torch Version	CT2 Version
2.*.*+cu121	<=4.4.0
2.*.*+cu124	>=4.5.0
>=2.4.0	>=4.5.0
<2.4.0	<4.5.0

For google colab users, the quick solution is to downgrade to 4.4.0 as of 24/10/2024 as colab uses torch==2.5.0+cu12.1

[MahmoudAshraf97]

Update: it's compatible with torch==2.*+cu124 so it's only incompatible with 12.1, I'll open a PR to solve this

[BBC-Esq]

Thanks to me and my analysis here...I want full citation and credit please: ;-)

SYSTRAN/faster-whisper#1082 (comment)

@MahmoudAshraf97 let me know if you need to know how I prefer citations to my contributions made please. Thanks.

[MarkusGehrig]

@BBC-Esq

Thank you for the list of compatible versions. I have the following versions installed in a Docker container, but I still get the cuDNN error. Am I missing something?

OS:
Debian GNU/Linux 12 (bookworm)

nvidia-smi:
NVIDIA RTX 4500 Ada Gene...
NVIDIA-SMI 560.35.03
Driver Version: 560.35.03
CUDA Version: 12.6

Name: torch
Version: 2.5.0

Name: ctranslate2
Version: 4.5.0

Name: nvidia-cudnn-cu12
Version: 9.1.0.70

Name: nvidia-cuda-cupti-cu12
Version: 12.4.127

Name: nvidia-cublas-cu12
Version: 12.4.5.8

Name: nvidia-cuda-runtime-cu12
Version: 12.4.127

[MahmoudAshraf97]

which cudnn error are you getting exactly?

[MarkusGehrig]

@MahmoudAshraf97
Exactly the same as in #1806 (comment)

[minhthuc2502]

@MarkusGehrig Can you try ldconfig -p | grep cudnn_cnn to verify the location of cudnn_cnn lib?

If you found the lib, you can check if this lib is in the runtime search path. If not, try to add it into LD_LIBRARY_PATH and see what happens.

[MarkusGehrig]

Thanks for the quick response. But I still get the error.

Unable to load any of {libcudnn_cnn.so.9.1.0, libcudnn_cnn.so.9.1, libcudnn_cnn.so.9, libcudnn_cnn.so}
Invalid handle. Cannot load symbol cudnnCreateConvolutionDescriptor
Aborted (core dumped)

ldconfig -p | grep cudnn_cnn
libcudnn_cnn.so.9 (libc6,x86-64) => /lib/x86_64-linux-gnu/libcudnn_cnn.so.9
libcudnn_cnn.so (libc6,x86-64) => /lib/x86_64-linux-gnu/libcudnn_cnn.so

printenv LD_LIBRARY_PATH
/lib/x86_64-linux-gnu/libcudnn_cnn.so.9

I also have tried to install it with pip, with no change.

[minhthuc2502]

Try to set only LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/lib/x86_64-linux-gnu/.

And what is the result of this: nm -gD /lib/x86_64-linux-gnu/libcudnn_cnn.so | grep cudnnCreateConvolutionDescriptor?

[MarkusGehrig]

@minhthuc2502

It works like a charm now. It was a path that needed to be set to LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/lib/x86_64-linux-gnu/

Thanks!

[BBC-Esq]

For everybody's future reference, to investigate possible compatibility issues with other CUDA libraries or platforms in general, you can go here. In the lower left you can select the cuDNN version and it will bring up a page that explains all the nuances:

https://docs.nvidia.com/deeplearning/cudnn/v9.1.1/reference/support-matrix.html

This might help troubleshooting based on platform, etc.

Also, if you need to correlate what version of cuDNN you have you can go to either of these places:

https://developer.download.nvidia.com/compute/cudnn/redist/cudnn/
https://pypi.org/project/nvidia-cudnn-cu12/#history