Unable to use WhisperX with Blackwell GPU (sm_120)

[ireadvns]

I have not been able to successfully run WhisperX with a 5090. Stable Pytorch versions do not support CUDA compute capability sm_120, but if I install the nightly via

pip3 install --pre torch torchvision --index-url https://download.pytorch.org/whl/nightly/cu129

Then I get a Could not locate cudnn_ops_infer64_8.dll. Please make sure it is in your library path! error at the beginning of the transcription.

Has anyone successfully made it work? Can you please mention what versions of CUDA, CUDNN, and PyTorch you used?

[ypwcharles]

Same on my machine. Maybe you can have a look at #1087
I'm trying.

[ireadvns]

I tried following the example that worked there. Uninstalled all CUDA and cuDNN, created a new venv. This is my requirements.txt file

# Core dependencies
whisperx==3.3.1
ffmpeg-python>=0.2.0
click>=8.0.0
PyYAML>=6.0
rich>=13.0.0

# PyTorch with CUDA 12.1 support
torch==2.5.1
torchvision==0.20.1
torchaudio==2.5.1

# Whisper dependencies
ctranslate2==4.4.0
faster-whisper==1.1.0

# Additional utilities
tqdm>=4.64.0
pathlib>=1.0.1
psutil>=5.9.0

# Development dependencies (optional)
pytest>=7.0.0
pytest-cov>=4.0.0
black>=22.0.0
flake8>=5.0.0

I then downloaded the standalone .dll files as linked, both the cuBLAS.and.cuDNN_CUDA12_win_v3.7z and
cuBLAS.and.cuDNN_CUDA11_win_v4.7z, and copied the .dll files to the specified venv torch lib directory.

This is the error I encounter when I activate the venv and try run the small model on a 10 second clip

Application error: Insufficient resources to load model 'small': CUDA/cuDNN error during model loading: CUDA error: no kernel image is available for execution on the device                                 
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

Suggestion: Ensure CUDA 12.8 and compatible cuDNN are properly installed. Check that cudnn_ops_infer64_8.dll is in your PATH
Suggestion: Try using a smaller model (e.g., 'medium' or 'small')

[jim60105]

#1182 is expected to function on Blackwell (RTX 50XX), but I have no hardware to test it.
If you could share the results (or any error messages) I would be grateful.

[ireadvns]

#1182 is expected to function on Blackwell (RTX 50XX), but I have no hardware to test it. If you could share the results (or any error messages) I would be grateful.

This actually resolved my issue. CUDA 12.8, Pytorch cu128 and the PR worked without any issues and errors.