Add Support for Nvidia Blackwell Architecture (RTX 5000 cards)

[hunterczech]

Hello,
Even with updated pytorch to 2.7.0 which features sm_120 support it still doesn't work meaning the app itself probably has to updated.

[InKu0721]

I also encountered this situation, using 5070 for model training, pytorch installed 2.8.0.dev20250525+cu128, using cuda128, but still unable to train

[dirtmonster1337]

okada figured it out, can we get an update here?

[wxd11011]

Really looking forward to the upcoming update!

[BlankFX1]

While also still hoping for 5000 support getting implemented, I assume this project to be dead by now.
Which is really sad.

[yuanshangyujingshierlou]

虽然我仍然希望能够实现 5000 个支持，但我认为这个项目现在已经夭折了。 这真是令人难过。

我5070ti更新了pytorch后就可以使用了

[BlankFX1]

我5070ti更新了pytorch后就可以使用了
(translated:) My 5070ti works after updating pytorch!

Are you able to give us specific steps on how to do that correctly? A lot of us would be thankful.
翻译
您能告诉我们如何正确操作的具体步骤吗？我们很多人都会感激不尽。

// Edit:

Just re-running
pip install torch torchvision torchaudio
doesn't do the trick.

B:\RVC-beta>pip install torch torchvision torchaudio
Requirement already satisfied: torch in t:\tools\python\python3.10.6\lib\site-packages (2.7.0+cu128)
Requirement already satisfied: torchvision in t:\tools\python\python3.10.6\lib\site-packages (0.22.0+cu128)
Requirement already satisfied: torchaudio in t:\tools\python\python3.10.6\lib\site-packages (2.7.0+cu128)
Requirement already satisfied: filelock in t:\tools\python\python3.10.6\lib\site-packages (from torch) (3.13.1)
Requirement already satisfied: typing-extensions>=4.10.0 in t:\tools\python\python3.10.6\lib\site-packages (from torch) (4.12.2)
Requirement already satisfied: sympy>=1.13.3 in t:\tools\python\python3.10.6\lib\site-packages (from torch) (1.13.3)
Requirement already satisfied: networkx in t:\tools\python\python3.10.6\lib\site-packages (from torch) (3.3)
Requirement already satisfied: jinja2 in t:\tools\python\python3.10.6\lib\site-packages (from torch) (3.1.4)
Requirement already satisfied: fsspec in t:\tools\python\python3.10.6\lib\site-packages (from torch) (2024.6.1)
Requirement already satisfied: numpy in t:\tools\python\python3.10.6\lib\site-packages (from torchvision) (1.26.4)
Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in t:\tools\python\python3.10.6\lib\site-packages (from torchvision) (10.0.1)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in t:\tools\python\python3.10.6\lib\site-packages (from sympy>=1.13.3->torch) (1.3.0)
Requirement already satisfied: MarkupSafe>=2.0 in t:\tools\python\python3.10.6\lib\site-packages (from jinja2->torch) (2.1.5)

B:\RVC-beta\runtime\lib\site-packages\torch\cuda\__init__.py:173: UserWarning:
NVIDIA GeForce RTX 5090 with CUDA capability sm_120 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_61 sm_70 sm_75 sm_80 sm_86 sm_90 compute_37.
If you want to use the NVIDIA GeForce RTX 5090 GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/

//Edit 2
After a few hours of manual debugging and a lot of outdated functions in this project, which are no longer supported by torch/torchvision itself, I had to give up.
Newer versions of torch are utilizing security functions and other modern approaches this project currently lacks of.
It is too outdated for me to migrate them all with my current level of knowledge.

I think it is safe to assume this project will never support RTX 5000.

(Which is sad as I managed to make it train two models with 200 Steps each in under 3 minutes on a 5090, but I wasn't able to fix the function to properly convert the trained weights to a proper and usable model - if that is the correct term for that process.)

[ct-jyjntc]

我5070ti更新了pytorch后就可以使用了
(translated:) My 5070ti works after updating pytorch!

Are you able to give us specific steps on how to do that correctly? A lot of us would be thankful. 翻译 您能告诉我们如何正确操作的具体步骤吗？我们很多人都会感激不尽。

// Edit:

Just re-running pip install torch torchvision torchaudio doesn't do the trick.

B:\RVC-beta>pip install torch torchvision torchaudio
Requirement already satisfied: torch in t:\tools\python\python3.10.6\lib\site-packages (2.7.0+cu128)
Requirement already satisfied: torchvision in t:\tools\python\python3.10.6\lib\site-packages (0.22.0+cu128)
Requirement already satisfied: torchaudio in t:\tools\python\python3.10.6\lib\site-packages (2.7.0+cu128)
Requirement already satisfied: filelock in t:\tools\python\python3.10.6\lib\site-packages (from torch) (3.13.1)
Requirement already satisfied: typing-extensions>=4.10.0 in t:\tools\python\python3.10.6\lib\site-packages (from torch) (4.12.2)
Requirement already satisfied: sympy>=1.13.3 in t:\tools\python\python3.10.6\lib\site-packages (from torch) (1.13.3)
Requirement already satisfied: networkx in t:\tools\python\python3.10.6\lib\site-packages (from torch) (3.3)
Requirement already satisfied: jinja2 in t:\tools\python\python3.10.6\lib\site-packages (from torch) (3.1.4)
Requirement already satisfied: fsspec in t:\tools\python\python3.10.6\lib\site-packages (from torch) (2024.6.1)
Requirement already satisfied: numpy in t:\tools\python\python3.10.6\lib\site-packages (from torchvision) (1.26.4)
Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in t:\tools\python\python3.10.6\lib\site-packages (from torchvision) (10.0.1)
Requirement already satisfied: mpmath<1.4,>=1.1.0 in t:\tools\python\python3.10.6\lib\site-packages (from sympy>=1.13.3->torch) (1.3.0)
Requirement already satisfied: MarkupSafe>=2.0 in t:\tools\python\python3.10.6\lib\site-packages (from jinja2->torch) (2.1.5)

B:\RVC-beta\runtime\lib\site-packages\torch\cuda\__init__.py:173: UserWarning:
NVIDIA GeForce RTX 5090 with CUDA capability sm_120 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_61 sm_70 sm_75 sm_80 sm_86 sm_90 compute_37.
If you want to use the NVIDIA GeForce RTX 5090 GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/

//Edit 2 After a few hours of manual debugging and a lot of outdated functions in this project, which are no longer supported by torch/torchvision itself, I had to give up. Newer versions of torch are utilizing security functions and other modern approaches this project currently lacks of. It is too outdated for me to migrate them all with my current level of knowledge.

I think it is safe to assume this project will never support RTX 5000.

(Which is sad as I managed to make it train two models with 200 Steps each in under 3 minutes on a 5090, but I wasn't able to fix the function to properly convert the trained weights to a proper and usable model - if that is the correct term for that process.)

From what I know is that the nightly build of pytorch on CUDA 12.8 works with 50 series card as I've tested on flowframes, but for this project I tried installing it in a conda environment, which succeeded but wont work, as my card cant be detected in the infer web page, so I reckon they indeed won't be compatible for 50 series due to the deprecated functions I guess in the newest version of pytorch, unless a specific version of this repo is made just for the 50 series.