Support PyTorch with CUDA 12.8 in order to support new NVIDIA Blackwell architecture

[fantauzzi]

Search before asking

• I have searched the Ultralytics issues and found no similar feature requests.

Description

I am trying to train and validate a yolo11s model with the coco dataset on an NVIDIA RTX 4070 ti. The PyTorch version installed by ultralytics, however, doesn't support my GPU, nor I believe the Blackwell architecture sm_120 in general.

Use case

Support the new NVIDIA Blackwell architecture for training and inference.

Additional

When I run

yolo detect train data=coco.yaml model=yolo11s.pt epochs=1 batch=38 workers=16 seed=42 deterministic=True imgsz=640

I get this warning

NVIDIA GeForce RTX 5070 Ti with CUDA capability sm_120 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_50 sm_60 sm_70 sm_75 sm_80 sm_86 sm_90.

and soon thereafter an error

RuntimeError: CUDA error: no kernel image is available for execution on the device

and training doesn't start.

As a workaround, I have manually installed a version of PyTorch that supports CUDA 12.8 and therefore Blackwell with pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128.

Running the same command as before

yolo detect train data=coco.yaml model=yolo11s.pt epochs=1 batch=38 workers=16 seed=42 deterministic=True imgsz=640

training starts and completes, but validation stops with an error. Here attached the complete output

logs.txt

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

[UltralyticsAssistant]

👋 Hello @fantauzzi, thank you for your interest in Ultralytics 🚀! We recommend a visit to the Docs for new users where you can find many Python and CLI usage examples and where many of the most common questions may already be answered.

If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.

Join the Ultralytics community where it suits you best. For real-time chat, head to Discord 🎧. Prefer in-depth discussions? Check out Discourse. Or dive into threads on our Subreddit to share knowledge with the community.

Upgrade

Upgrade to the latest ultralytics package including all requirements in a Python>=3.8 environment with PyTorch>=1.8 to verify your issue is not already resolved in the latest version:

pip install -U ultralytics

Environments

YOLO may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all Ultralytics CI tests are currently passing. CI tests verify correct operation of all YOLO Modes and Tasks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

This is an automated response. An Ultralytics engineer will review your request and assist you soon! Thank you for helping us improve YOLO 🦾

[Y-T-G]

Can you try pip3 install torch torchvision torchaudio --force-reinstall --index-url https://download.pytorch.org/whl/cu128?

[Y-T-G]

Can you also provide the output after running this command in terminal: yolo checks?

[fantauzzi]

This after installing ultralytics regularly:

⮕ yolo -check
/home/fanta/.pyenv/versions/3.12.5/envs/ultralytics/lib/python3.12/site-packages/torch/cuda/__init__.py:287: UserWarning: 
NVIDIA GeForce RTX 5070 Ti with CUDA capability sm_120 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_50 sm_60 sm_70 sm_75 sm_80 sm_86 sm_90.
If you want to use the NVIDIA GeForce RTX 5070 Ti GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/

  warnings.warn(
Ultralytics 8.3.158 🚀 Python-3.12.5 torch-2.7.1+cu126 CUDA:0 (NVIDIA GeForce RTX 5070 Ti, 15828MiB)
Setup complete ✅ (32 CPUs, 30.4 GB RAM, 249.5/1831.7 GB disk)

OS                  Linux-6.11.0-26-generic-x86_64-with-glibc2.39
Environment         Linux
Python              3.12.5
Install             git
Path                /home/fanta/.pyenv/versions/3.12.5/envs/ultralytics/lib/python3.12/site-packages/ultralytics
RAM                 30.45 GB
Disk                249.5/1831.7 GB
CPU                 AMD Ryzen 9 9950X 16-Core Processor
CPU count           32
GPU                 NVIDIA GeForce RTX 5070 Ti, 15828MiB
GPU count           1
CUDA                12.6

numpy               ✅ 2.3.1>=1.23.0
matplotlib          ✅ 3.10.3>=3.3.0
opencv-python       ✅ 4.11.0.86>=4.6.0
pillow              ✅ 11.2.1>=7.1.2
pyyaml              ✅ 6.0.2>=5.3.1
requests            ✅ 2.32.4>=2.23.0
scipy               ✅ 1.16.0>=1.4.1
torch               ✅ 2.7.1>=1.8.0
torch               ✅ 2.7.1!=2.4.0,>=1.8.0; sys_platform == "win32"
torchvision         ✅ 0.22.1>=0.9.0
tqdm                ✅ 4.67.1>=4.64.0
psutil              ✅ 7.0.0
py-cpuinfo          ✅ 9.0.0
pandas              ✅ 2.3.0>=1.1.4
ultralytics-thop    ✅ 2.0.14>=2.0.0

This after uninstalling torch and torchvision, and reinstalling them for CUDA 12.8 with pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu128

⮕ yolo -check
Ultralytics 8.3.158 🚀 Python-3.12.5 torch-2.7.1+cu128 CUDA:0 (NVIDIA GeForce RTX 5070 Ti, 15828MiB)
Setup complete ✅ (32 CPUs, 30.4 GB RAM, 250.7/1831.7 GB disk)

OS                  Linux-6.11.0-26-generic-x86_64-with-glibc2.39
Environment         Linux
Python              3.12.5
Install             git
Path                /home/fanta/.pyenv/versions/3.12.5/envs/ultralytics/lib/python3.12/site-packages/ultralytics
RAM                 30.45 GB
Disk                250.7/1831.7 GB
CPU                 AMD Ryzen 9 9950X 16-Core Processor
CPU count           32
GPU                 NVIDIA GeForce RTX 5070 Ti, 15828MiB
GPU count           1
CUDA                12.8

numpy               ✅ 2.3.1>=1.23.0
matplotlib          ✅ 3.10.3>=3.3.0
opencv-python       ✅ 4.11.0.86>=4.6.0
pillow              ✅ 11.2.1>=7.1.2
pyyaml              ✅ 6.0.2>=5.3.1
requests            ✅ 2.32.4>=2.23.0
scipy               ✅ 1.16.0>=1.4.1
torch               ✅ 2.7.1+cu128>=1.8.0
torch               ✅ 2.7.1+cu128!=2.4.0,>=1.8.0; sys_platform == "win32"
torchvision         ✅ 0.22.1+cu128>=0.9.0
tqdm                ✅ 4.67.1>=4.64.0
psutil              ✅ 7.0.0
py-cpuinfo          ✅ 9.0.0
pandas              ✅ 2.3.0>=1.1.4
ultralytics-thop    ✅ 2.0.14>=2.0.0

[fantauzzi]

Can you try pip3 install torch torchvision torchaudio --force-reinstall --index-url https://download.pytorch.org/whl/cu128?

After doing that, training and validation completed successfully! Output attached.

log2.txt

Output of yolo -check

Ultralytics 8.3.158 🚀 Python-3.12.5 torch-2.7.1+cu128 CUDA:0 (NVIDIA GeForce RTX 5070 Ti, 15828MiB)
Setup complete ✅ (32 CPUs, 30.4 GB RAM, 251.2/1831.7 GB disk)

OS                  Linux-6.11.0-26-generic-x86_64-with-glibc2.39
Environment         Linux
Python              3.12.5
Install             git
Path                /home/fanta/.pyenv/versions/3.12.5/envs/ultralytics/lib/python3.12/site-packages/ultralytics
RAM                 30.45 GB
Disk                251.2/1831.7 GB
CPU                 AMD Ryzen 9 9950X 16-Core Processor
CPU count           32
GPU                 NVIDIA GeForce RTX 5070 Ti, 15828MiB
GPU count           1
CUDA                12.8

numpy               ✅ 2.1.2>=1.23.0
matplotlib          ✅ 3.10.3>=3.3.0
opencv-python       ✅ 4.11.0.86>=4.6.0
pillow              ✅ 11.0.0>=7.1.2
pyyaml              ✅ 6.0.2>=5.3.1
requests            ✅ 2.32.4>=2.23.0
scipy               ✅ 1.16.0>=1.4.1
torch               ✅ 2.7.1+cu128>=1.8.0
torch               ✅ 2.7.1+cu128!=2.4.0,>=1.8.0; sys_platform == "win32"
torchvision         ✅ 0.22.1+cu128>=0.9.0
tqdm                ✅ 4.67.1>=4.64.0
psutil              ✅ 7.0.0
py-cpuinfo          ✅ 9.0.0
pandas              ✅ 2.3.0>=1.1.4
ultralytics-thop    ✅ 2.0.14>=2.0.0

[glenn-jocher]

Great to hear that the --force-reinstall flag resolved the issue! This is actually expected behavior - Ultralytics automatically installs PyTorch dependencies, but newer GPU architectures like Blackwell (sm_120) require the latest PyTorch builds with CUDA 12.8 support.

The solution you used is the correct approach for Blackwell GPUs. We'll update our default PyTorch dependencies as new stable versions become widely available, but users with cutting-edge hardware like the RTX 5070 Ti may need to manually install the appropriate PyTorch version as you've done.

Your workaround using pip install torch torchvision torchaudio --force-reinstall --index-url https://download.pytorch.org/whl/cu128 should be the standard approach for anyone with Blackwell architecture GPUs until the CUDA 12.8 PyTorch builds become the default.

[suttin]

Great to hear that the --force-reinstall flag resolved the issue! This is actually expected behavior - Ultralytics automatically installs PyTorch dependencies, but newer GPU architectures like Blackwell (sm_120) require the latest PyTorch builds with CUDA 12.8 support.

The solution you used is the correct approach for Blackwell GPUs. We'll update our default PyTorch dependencies as new stable versions become widely available, but users with cutting-edge hardware like the RTX 5070 Ti may need to manually install the appropriate PyTorch version as you've done.

Your workaround using pip install torch torchvision torchaudio --force-reinstall --index-url https://download.pytorch.org/whl/cu128 should be the standard approach for anyone with Blackwell architecture GPUs until the CUDA 12.8 PyTorch builds become the default.

Just wanted to add a comment that this also fixed the same issue I was having as OP. And for the search engine, I could only find it with part of the error message, but here's the full error I encountered when setting up my project:

torch.AcceleratorError: CUDA error: no kernel image is available for execution on the device

[glenn-jocher]

Thanks for confirming the solution worked for you too @suttin! Adding the full torch.AcceleratorError: CUDA error: no kernel image is available for execution on the device error message is really helpful for others who might encounter this issue with Blackwell GPUs.