Application does not work with RTX 5090

[UberMentch]

A more detailed log can be found here - https://pastebin.com/rcNEJRz2

The relevant section is found below - I'm running this using WSL2, on an Ubuntu-24.04 image, with Docker Desktop installed. I've also made sure that the Container Toolkit is installed. I've read elsewhere that using the PyTorch nightly version might be a solution. If there are changes that I could make to the dockerfiles to test for a fix, I'd be happy to help test them out - I am still a bit new to container building, so I just need a bit of help figuring out what exactly needs done.

2025-04-11 15:04:59 Transcription error: Lightning automatically upgraded your loaded checkpoint from v1.5.4 to v2.5.1. To apply the upgrade to your files permanently, run python -m pytorch_lightning.utilities.upgrade_checkpoint ../scriberr/.venv/lib/python3.12/site-packages/whisperx/assets/pytorch_model.bin
2025-04-11 15:04:59
2025-04-11 15:04:59 Transcription error: /scriberr/.venv/lib/python3.12/site-packages/torch/cuda/init.py:235: UserWarning:
2025-04-11 15:04:59 NVIDIA GeForce RTX 5090 with CUDA capability sm_120 is not compatible with the current PyTorch installation.
2025-04-11 15:04:59 The current PyTorch install supports CUDA capabilities sm_50 sm_60 sm_70 sm_75 sm_80 sm_86 sm_90.
2025-04-11 15:04:59 If you want to use the NVIDIA GeForce RTX 5090 GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/
2025-04-11 15:04:59
2025-04-11 15:04:59 warnings.warn(
2025-04-11 15:04:59
2025-04-11 15:04:59 Transcription progress: Model was trained with pyannote.audio 0.0.1, yours is 3.3.2. Bad things might happen unless you revert pyannote.audio to 0.x.
2025-04-11 15:04:59 Model was trained with torch 1.10.0+cu102, yours is 2.6.0+cu126. Bad things might happen unless you revert torch to 1.x.
2025-04-11 15:04:59
2025-04-11 15:04:59 Transcription error: Traceback (most recent call last):
2025-04-11 15:04:59 File "/app/transcribe.py", line 180, in
2025-04-11 15:04:59 main()
2025-04-11 15:04:59 File "/app/transcribe.py", line 94, in main
2025-04-11 15:04:59 model = whisperx.load_model(
2025-04-11 15:04:59 ^^^^^^^^^^^^^^^^^^^^
2025-04-11 15:04:59 File "/scriberr/.venv/lib/python3.12/site-packages/whisperx/asr.py", line 387, in load_model
2025-04-11 15:04:59 vad_model = load_vad_model(torch.device(device), use_auth_token=None, **default_vad_options)
2025-04-11 15:04:59 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-04-11 15:04:59
2025-04-11 15:04:59 Transcription error: File "/scriberr/.venv/lib/python3.12/site-packages/whisperx/vad.py", line 52, in load_vad_model
2025-04-11 15:04:59 vad_pipeline = VoiceActivitySegmentation(segmentation=vad_model, device=torch.device(device))
2025-04-11 15:04:59 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-04-11 15:04:59
2025-04-11 15:04:59 Transcription error: File "/scriberr/.venv/lib/python3.12/site-packages/whisperx/vad.py", line 203, in init
2025-04-11 15:04:59 super().init(segmentation=segmentation, fscore=fscore, use_auth_token=use_auth_token, **inference_kwargs)
2025-04-11 15:04:59
2025-04-11 15:04:59 Transcription error: File "/scriberr/.venv/lib/python3.12/site-packages/pyannote/audio/pipelines/voice_activity_detection.py", line 128, in init
2025-04-11 15:04:59
2025-04-11 15:04:59 Transcription error: self._segmentation = Inference(model, **inference_kwargs)
2025-04-11 15:04:59
2025-04-11 15:04:59 Transcription error: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-04-11 15:04:59 File "/scriberr/.venv/lib/python3.12/site-packages/pyannote/audio/core/inference.py", line 114, in init
2025-04-11 15:04:59
2025-04-11 15:04:59 Transcription error: self.model.to(self.device)
2025-04-11 15:04:59
2025-04-11 15:04:59 Transcription error: File "/scriberr/.venv/lib/python3.12/site-packages/lightning_fabric/utilities/device_dtype_mixin.py", line 55, in to
2025-04-11 15:04:59
2025-04-11 15:04:59 Transcription error: return super().to(*args, **kwargs)
2025-04-11 15:04:59
2025-04-11 15:04:59 Transcription error: ^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-04-11 15:04:59 File "/scriberr/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1343, in to
2025-04-11 15:04:59
2025-04-11 15:04:59 Transcription error: return self._apply(convert)
2025-04-11 15:04:59
2025-04-11 15:04:59 Transcription error: ^^^^^^^^^^^^^^^^^^^^
2025-04-11 15:04:59 File "/scriberr/.venv/lib/python3.12/site-packages/torch/nn/modules/module.py", line 903, in _apply
2025-04-11 15:04:59
2025-04-11 15:04:59 Transcription error: module._apply(fn)
2025-04-11 15:04:59
2025-04-11 15:04:59 Transcription error: File "/scriberr/.venv/lib/python3.12/site-packages/torch/nn/modules/rnn.py", line 290, in _apply
2025-04-11 15:04:59
2025-04-11 15:04:59 Transcription error: self._init_flat_weights()
2025-04-11 15:04:59 File "/scriberr/.venv/lib/python3.12/site-packages/torch/nn/modules/rnn.py", line 215, in _init_flat_weights
2025-04-11 15:04:59
2025-04-11 15:04:59 Transcription error: self.flatten_parameters()
2025-04-11 15:04:59
2025-04-11 15:04:59 Transcription error: File "/scriberr/.venv/lib/python3.12/site-packages/torch/nn/modules/rnn.py", line 271, in flatten_parameters
2025-04-11 15:04:59
2025-04-11 15:04:59 Transcription error: torch._cudnn_rnn_flatten_weight(
2025-04-11 15:04:59 RuntimeError: CUDA error: no kernel image is available for execution on the device
2025-04-11 15:04:59 CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
2025-04-11 15:04:59 For debugging consider passing CUDA_LAUNCH_BLOCKING=1
2025-04-11 15:04:59 Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

[BarbellDwarf]

So. I found a few issues related to this that have cropped up in the last few weeks. It appears that Pytorch does not support 5000 series GPUs on their stable version of Pytorch. It is only supported in their nightly version of Pytorch. We will need to review what the nightly branch of pytorch has different as it may impact the supported platforms we currently have. I will look at it over this weekend and will get back to you on this. Sorry about that!

[UberMentch]

No need for apology, I appreciate your work. If there's anything I can do to help you out, let me know!

[rishikanthc]

@UberMentch hey would you be able to try the application with pytorch nightly and see if it works ? You will need to modify a couple places but it's not too bad and we can point you to it.

[UberMentch]

Sure, I'll do a fresh clone of the repo, will only change the HARDWARE_ACCEL in .env to gpu for now

[BarbellDwarf]

@UberMentch. So. you may be able to modify the Torch install that occurs in the docker-entrypoint.sh from:
uv pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu126
to
uv pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128

That should install the nightly version which should allow you to run it. make sure to delete any of the Docker volumes that are there so it doesn't have conflicting dependencies.

Let me know if that works!

And @rishikanthc, if that does work. We may need to put in some logic to identify which GPU series is present and then download the appropriate version of Pytorch. But we will see if this works first.

[UberMentch]

Awesome, that seems to have worked, thank you! I'm able to record audio / upload audio and have it transcribe, which I tested with several of the models. I'll list the changes I made to files below to get things working, beyond changing the HARDWARE_ACCEL in .env. Below the changes, I'll paste the log output; there were still a few minor errors that I noticed, but nothing that seems to have prevented the main functionality from working

docker-compose.gpu.yml

• Line 12 - image: scriberr:latest-gpu

docker-entrypoint.sh

• Line 80 - uv pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128

Renamed dockerfile-cuda128 to Dockerfile-gpu, since the docker-compose.gpu.yml was looking for "Dockerfile-gpu"

2025-04-12 11:17:38 [Hooks] Request origin: http://127.0.0.1:3000
2025-04-12 11:17:38 Diarization setting: false
2025-04-12 11:17:38 SAVE ORIG --->
2025-04-12 11:17:38 Saved original file for playback: /scriberr/uploads/1744474658950-original.wav
2025-04-12 11:17:39 Transcription options being used: {
2025-04-12 11:17:39 modelSize: 'large-v3',
2025-04-12 11:17:39 language: 'en',
2025-04-12 11:17:39 threads: 8,
2025-04-12 11:17:39 processors: 4,
2025-04-12 11:17:39 diarization: false
2025-04-12 11:17:39 }
2025-04-12 11:17:39 Diarization setting: false
2025-04-12 11:17:39 Queue settings: { DATABASE_URL: true, USE_WORKER: true }
2025-04-12 11:17:39 Queueing job for audioId: 2
2025-04-12 11:17:39 Job queued successfully: null
2025-04-12 11:17:39 Queued job: {
2025-04-12 11:17:39 audioFile: {
2025-04-12 11:17:39 id: 2,
2025-04-12 11:17:39 fileName: '1744474658950.wav',
2025-04-12 11:17:39 originalFileName: '1744474658950-original.wav',
2025-04-12 11:17:39 originalFileType: 'wav',
2025-04-12 11:17:39 duration: null,
2025-04-12 11:17:39 transcript: null,
2025-04-12 11:17:39 transcriptionStatus: 'pending',
2025-04-12 11:17:39 summary: null,
2025-04-12 11:17:39 summaryPrompt: null,
2025-04-12 11:17:39 summaryStatus: null,
2025-04-12 11:17:39 language: 'en',
2025-04-12 11:17:39 lastError: null,
2025-04-12 11:17:39 peaks: [
2025-04-12 11:17:39 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2025-04-12 11:17:39 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2025-04-12 11:17:39 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2025-04-12 11:17:39 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2025-04-12 11:17:39 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2025-04-12 11:17:39 0, 0, 0, 2, -476, 574, -930, 715, -828, 629,
2025-04-12 11:17:39 -650, 939, -1576, 3440, -3984, 7306, -4991, 7241, -4732, 6362,
2025-04-12 11:17:39 -4317, 6433, -5188, 5980, -4571, 6602, -4472, 6429, -3986, 5781,
2025-04-12 11:17:39 -2577, 5101, -2437, 5003, -2161, 4690, -1772, 3588, -2052, 3509,
2025-04-12 11:17:39 -934, 951, -411, 430, -649, 626, -367, 450, -463, 477,
2025-04-12 11:17:39 ... 604 more items
2025-04-12 11:17:39 ],
2025-04-12 11:17:39 modelSize: 'large-v3',
2025-04-12 11:17:39 threads: 8,
2025-04-12 11:17:39 title: 'recording-1744474659292',
2025-04-12 11:17:39 processors: 4,
2025-04-12 11:17:39 diarization: false,
2025-04-12 11:17:39 transcriptionProgress: 0,
2025-04-12 11:17:39 uploadedAt: 2025-04-12T16:17:39.068Z,
2025-04-12 11:17:39 transcribedAt: null,
2025-04-12 11:17:39 summarizedAt: null,
2025-04-12 11:17:39 updatedAt: null
2025-04-12 11:17:39 }
2025-04-12 11:17:39 }
2025-04-12 11:17:39 Starting transcription for file ID: 2
2025-04-12 11:17:39 Using audio file path: /scriberr/uploads/1744474658950.wav
2025-04-12 11:17:39 Retrieved fileInfo for ID 2: {
2025-04-12 11:17:39 modelSize: 'large-v3',
2025-04-12 11:17:39 language: 'en',
2025-04-12 11:17:39 threads: 8,
2025-04-12 11:17:39 processors: 4,
2025-04-12 11:17:39 diarization: false
2025-04-12 11:17:39 }
2025-04-12 11:17:39 Diarization setting from database: false
2025-04-12 11:17:39 Transcribing audio file: /scriberr/uploads/1744474658950.wav
2025-04-12 11:17:39 Checking if diarization should be enabled: false
2025-04-12 11:17:39 Diarization is disabled.
2025-04-12 11:17:39 Using device: cuda (HARDWARE_ACCEL=cuda)
2025-04-12 11:17:39 Using default compute type: float16
2025-04-12 11:17:39 Executing Python with args: [
2025-04-12 11:17:39 'transcribe.py',
2025-04-12 11:17:39 '--audio-file',
2025-04-12 11:17:39 '/scriberr/uploads/1744474658950.wav',
2025-04-12 11:17:39 '--model-size',
2025-04-12 11:17:39 'large-v3',
2025-04-12 11:17:39 '--output-file',
2025-04-12 11:17:39 '/scriberr/uploads/transcript-2.json',
2025-04-12 11:17:39 '--threads',
2025-04-12 11:17:39 '8',
2025-04-12 11:17:39 '--language',
2025-04-12 11:17:39 'en',
2025-04-12 11:17:39 '--device',
2025-04-12 11:17:39 'cuda',
2025-04-12 11:17:39 '--compute-type',
2025-04-12 11:17:39 'float16'
2025-04-12 11:17:39 ]
2025-04-12 11:17:46 Transcription error: Lightning automatically upgraded your loaded checkpoint from v1.5.4 to v2.5.1. To apply the upgrade to your files permanently, run python -m pytorch_lightning.utilities.upgrade_checkpoint ../scriberr/.venv/lib/python3.12/site-packages/whisperx/assets/pytorch_model.bin
2025-04-12 11:17:46
2025-04-12 11:17:46 Transcription error: /scriberr/.venv/lib/python3.12/site-packages/pyannote/audio/utils/reproducibility.py:74: ReproducibilityWarning: TensorFloat-32 (TF32) has been disabled as it might lead to reproducibility issues and lower accuracy.
2025-04-12 11:17:46 It can be re-enabled by calling
2025-04-12 11:17:46 >>> import torch
2025-04-12 11:17:46 >>> torch.backends.cuda.matmul.allow_tf32 = True
2025-04-12 11:17:46 >>> torch.backends.cudnn.allow_tf32 = True
2025-04-12 11:17:46 See pyannote/pyannote-audio#1370 for more details.
2025-04-12 11:17:46
2025-04-12 11:17:46 warnings.warn(
2025-04-12 11:17:46
2025-04-12 11:17:47 Transcription progress: Model was trained with pyannote.audio 0.0.1, yours is 3.3.2. Bad things might happen unless you revert pyannote.audio to 0.x.
2025-04-12 11:17:47 Model was trained with torch 1.10.0+cu102, yours is 2.8.0. Bad things might happen unless you revert torch to 1.x.
2025-04-12 11:17:47 Progress: 100.00%...
2025-04-12 11:17:47 Transcript saved to /scriberr/uploads/transcript-2.json
2025-04-12 11:17:47
2025-04-12 11:17:48 Python process exited with code 0
2025-04-12 11:17:48 Reading transcript from /scriberr/uploads/transcript-2.json
2025-04-12 11:17:48 Transcription completed with 1 segments
2025-04-12 11:17:48 Sample segment: {"start":0.031,"end":4.857,"text":"","speaker":""}
2025-04-12 11:17:48 Updating transcription status to completed with transcript
2025-04-12 11:17:48 Transcription process finished successfully
2025-04-12 11:17:48 Direct transcription completed for file ID: 2

[rishikanthc]

@UberMentch Thank you so much. This helps !!
@SpirusNox I feel identifying GPUs might be complicated and we don't know what all GPUs need the nightly version. If nightly works, can't we just by default use pytorch nightly for all GPUs ? I don't expect this to break anything.. of course we would need more testers to try it out..

[UberMentch]

No problem, thank you guys for helping me out. If you ever want help testing other things out, let me know! I look forward to seeing this project develop more, I hope you guys have are having fun working on it!

[BarbellDwarf]

I have done some testing with the nighty. And it seems to work fine. But I need to test it on an older GPU to see how it does on that. I will be testing it on a 1070 later today. Thank you for the patience on this!

[BarbellDwarf]

So. It looks like if we go to the nightly version of Pytorch, Nvidia 10 series gpus and older are no longer working. I cannot run it on my 1070 that is in my AI Box. So that is problematic. Since not everyone has the ability to obtain a 50 series GPU.

So @rishikanthc, we could put a check in to see if its anything prior to a 30 series gpu and then install the original drivers (we could make a new bash script that can do this.)

I think that may be the best course of action for backwards compatibility. I bet a large portion of people will be on 20 series or older.

[UberMentch]

As a quick update, I believe I am reading that pytorch stable 2.7.0 now supports Blackwell (50 series) cards. Not sure if it affects 20-series and below cards, but I figured I'd mention it!

[unixbird]

Attempted to use my RTX 3090 and I actually got a stack trace error. Not quite sure if that's on scriberr or on my end however