CUDA Error on NVIDIA GeForce RTX 5070 Ti (Blackwell Architecture) - Request for sm_90 Support

[kaworukevin]

Dear FramePack Team,

I am reaching out to report an issue I’m facing while running FramePack on my NVIDIA GeForce RTX 5070 Ti (Blackwell architecture) in GPU mode, and to seek your assistance in resolving it. Despite following the suggestions from an engineer on GitHub and making several modifications, I am still encountering a "CUDA error: no kernel image is available for execution on the device." I believe this may be related to FramePack’s kernel compilation not supporting the Blackwell architecture (sm_90).

System Details:

Operating System: Windows 11

GPU: NVIDIA GeForce RTX 5070 Ti (Blackwell architecture, expected Compute Capability sm_90)

NVIDIA Driver Version: 576.02 (confirmed to be the latest as of April 26, 2025)

PyTorch Version: 2.7.0+cu128

CUDA Toolkit Version: 12.8 (driver supports CUDA 12.9)

FramePack Version: framepack_cu126_torch26

Python Version: 3.10

Issue Description:
When running FramePack in GPU mode, I encounter the following error:

CUDA error: no kernel image is available for execution on the device

This error occurs during the encode_prompt_conds function in hunyuan.py, specifically at the line:

llama_attention_length = int((llama_attention_mask.sum()))

Additionally, PyTorch misidentifies my GPU as sm_120 (Compute Capability 12.0) instead of sm_90, which may also contribute to the issue. I have reported this to the PyTorch team separately.

Steps Taken:

Based on an engineer’s suggestion on GitHub, I modified the code to use bf16 data types to avoid fp32-related issues. Specifically, I set history_latents and the sample_hunyuan function to use torch.bfloat16.

Initially, I ran FramePack in CPU mode, which worked but was very slow (30-60 minutes for a 5-second video).

After switching back to GPU mode and ensuring all tensors and models are on cuda:0, the CUDA error persists.

I updated PyTorch to 2.7.0+cu128 and my NVIDIA driver to the latest version (576.02), but the Compute Capability issue remains unresolved.

Request for Assistance:
Could you please confirm if the current FramePack version (framepack_cu126_torch26) supports the Blackwell architecture (sm_90)? If not, would it be possible to provide a kernel compilation that supports sm_90, or suggest a workaround to resolve the CUDA error? I would also appreciate any guidance on optimizing FramePack for Blackwell GPUs, especially regarding data types or memory management.

You can reach me directly at kaworukevin@gmail.com if you need any additional information or would like to follow up. I’ve attached my modified demo_gradio.py and hunyuan.py files for your reference. Thank you for your time and support!

Best regards,
kaworukevin

[uselessgithub]

You're looking for cu128 / torch 2.7
This issue is already discussed here #267
You have to install it yourself over the cu126_torch26 package.

[Clivey1234]

I've got a 5070ti on Windows 11 with Cuda 12.8 and this worked for me

All I did was install the zip file then in the framepack_cu126_torch26\system\python folder run the following commands:
.\python.exe -m pip install torch torchvision torchaudio --force-reinstall --index-url https://download.pytorch.org/whl/cu128
.\python.exe -m pip install --force-reinstall numpy==1.26.2
.\python.exe -m pip install flatbuffers

then run the run.bat. I know the update.bat works, but cant remember if I did that after the commands or before it.

I've yet to find a solution to get Flash/Xformers/Sage working, but its still pretty quick without them

[uselessgithub]

@Clivey1234
For sageattn you can use precompiled whl from here https://github.com/woct0rdho/SageAttention/releases
like this in your venv
pip3 install "sageattention-2.1.1+cu128torch2.7.0-cp310-cp310-win_amd64.whl"

I've yet to find a solution to get Flash/Xformers/Sage working, but its still pretty quick without them

[goncalorijo]

I've got a 5070ti on Windows 11 with Cuda 12.8 and this worked for me

All I did was install the zip file then in the framepack_cu126_torch26\system\python folder run the following commands: .\python.exe -m pip install torch torchvision torchaudio --force-reinstall --index-url https://download.pytorch.org/whl/cu128 .\python.exe -m pip install --force-reinstall numpy==1.26.2 .\python.exe -m pip install flatbuffers

then run the run.bat. I know the update.bat works, but cant remember if I did that after the commands or before it.

I've yet to find a solution to get Flash/Xformers/Sage working, but its still pretty quick without them

This worked. Thank you.

To be clear:
Download the FramePack 7zip file.
Unzip it.
Run the 3 commands, one after the other on the specified folder.
Run the run.bat.

[Clivey1234]

@Clivey1234 For sageattn you can use precompiled whl from here https://github.com/woct0rdho/SageAttention/releases like this in your venv pip3 install "sageattention-2.1.1+cu128torch2.7.0-cp310-cp310-win_amd64.whl"

I've yet to find a solution to get Flash/Xformers/Sage working, but its still pretty quick without them

I tried that before and again last night. After doing command, FramePack still doesn't detect Sage, it just says "not installed". I know its not my PC as I can get Sage working on Comfyui, just cant seem to get it going on FramePack.

[kaworukevin]

hello.everyone
I'm trying to install PyTorch 2.7.0 and torchvision 0.17.0 with CUDA support on Windows 11, but I'm facing several issues. I need to align with the versions used by my team’s engineers and ultimately run a project called FramePack. I’d really appreciate your help!

My Environment

Operating System: Windows 11 Pro x64

Python Version: 3.10.6

GPU: NVIDIA GeForce RTX 5070 Ti

NVIDIA Driver Version: 576.02

CUDA Version: 12.9 (as shown by nvidia-smi)

Virtual Environment: C:\Users\JXin\Documents\framepack_cu126_torch26\venv

My Goal

Install PyTorch 2.7.0 and torchvision 0.17.0 with CUDA 12.8 support (compatible with CUDA 12.9).

Ensure PyTorch enables CUDA (currently getting "Torch not compiled with CUDA enabled").

Install Triton 3.3.0 and SageAttention 2.1.1 (need Windows-compatible wheel files).

Resolve the NumPy initialization warning: UserWarning: Failed to initialize NumPy: _ARRAY_API not found.

What I've Done

Environment Cleanup:

Removed all WSL2 and Ubuntu residuals (used wsl --unregister and deleted ext4.vhdx).

Cleaned the system-level Python environment (C:\Users\JXin\AppData\Local\Programs\Python\Python310), removing all torch and nvidia-* related packages.

Created a virtual environment: python -m venv venv.

Attempted PyTorch Installation:

Used the command:

pip install torch==2.7.0 torchaudio==2.7.0 --index-url https://download.pytorch.org/whl/cu128 --no-cache-dir
pip install torchvision==0.17.0 --no-cache-dir

Result: Installed CPU versions (2.7.0+cpu, 0.17.0+cpu), not the expected 2.7.0+cu128.

Manual Wheel File Download:

Downloaded:

torch-2.7.0+cu128-cp310-cp310-win_amd64.whl

torchaudio-2.7.0+cu128-cp310-cp310-win_amd64.whl

Installed:

pip install C:\Users\JXin\Downloads\torch-2.7.0+cu128-cp310-cp310-win_amd64.whl
pip install C:\Users\JXin\Downloads\torchaudio-2.7.0+cu128-cp310-cp310-win_amd64.whl

Verified:

python -c "import torch; print(torch.version); print(torch.cuda.is_available())"

Output:

2.7.0+cu128
True

Confirmed PyTorch now supports CUDA.

Issue: Could Not Find torchvision Wheel File:

Could not find torchvision-0.17.0+cu128-cp310-cp310-win_amd64.whl at https://download.pytorch.org/whl/cu128.

Tried using pip to download automatically:

pip install torchvision==0.17.0 --index-url https://download.pytorch.org/whl/cu128 --no-cache-dir

Result: Still installed CPU version (0.17.0+cpu).

NumPy Warning:

During verification, I get:

UserWarning: Failed to initialize NumPy: _ARRAY_API not found

Current NumPy version might not be fully compatible with PyTorch 2.7.0.

My Questions

Cannot Find torchvision-0.17.0+cu128-cp310-cp310-win_amd64.whl:

Does anyone know where I can download this wheel file, or if there’s an alternative version (e.g., 0.18.0 or 0.16.0) that’s compatible with PyTorch 2.7.0?

If using pip to download automatically, do I need to adjust the --index-url?

NumPy Warning:

How can I resolve UserWarning: Failed to initialize NumPy: _ARRAY_API not found? Should I upgrade NumPy to a specific version (e.g., 2.0.0)?

SageAttention Wheel File:

I need sageattention-2.1.1+cu128torch2.7.0+cp310+cp310-win_amd64.whl (Windows version).

I only found sageattention-2.1.1+cu126torch2.6.0-cp310-cp310-win_amd64.whl (for PyTorch 2.6.0 and CUDA 12.6). Is this compatible? If not, where can I find the correct version?

What I've Tried

Verified CUDA environment:

nvidia-smi shows CUDA 12.9, driver is working.

Environment variable CUDA_PATH is set to C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.9.

Tried alternative torchvision versions (e.g., 0.18.0 and 0.16.0), but I'm concerned about compatibility with PyTorch 2.7.0.

Upgraded NumPy:

pip install --force-reinstall numpy==2.0.0

But the warning persists.

Request for Help

How can I resolve the issue of not finding torchvision-0.17.0+cu128-cp310-cp310-win_amd64.whl?

Is there another way to install PyTorch 2.7.0 and torchvision 0.17.0 with CUDA 12.8/12.9 on Windows 11?

How can I fix the NumPy warning?

Where can I find the correct SageAttention 2.1.1 wheel file?

You can also reach me directly at kaworukevin@gmail.com if you have any suggestions or files to share.

Thanks for your help!

[Clivey1234]

@kaworukevin

SageAttention 2.1.1 wheel file

Direct Download: https://github.com/woct0rdho/SageAttention/releases/download/v2.1.1-windows/sageattention-2.1.1+cu128torch2.7.0-cp310-cp310-win_amd64.whl

Links to the other versions: https://github.com/woct0rdho/SageAttention/releases

[pocketpoetry]

Can see if this walk-through works for you

https://youtu.be/dkqFtu0v_eY?si=D8TQsgkjfYmUXepj