Support request for latest CUDA + Python + Torch

[manchuwook]

CUDA 13.0 Compilation Compatibility Issue on Windows

Subject

[Windows] NVCC 13.0 Compilation Fails with "A single input file is required for a non-link phase when an outputfile is specified"

Environment

• OS: Windows 11
• GPU: NVIDIA GeForce RTX 5090 (Blackwell, sm_120)
• CUDA: 13.0
• Python: 3.13
• PyTorch: 2.9.0.dev20250909+cu130 (nightly)
• Visual Studio: 2022 Community (14.44.35207)

Issue Description

SageAttention compilation fails on Windows with CUDA 13.0 due to NVCC's stricter command-line parsing introduced in CUDA 13.0. The error occurs during ninja build execution:

nvcc fatal : A single input file is required for a non-link phase when an outputfile is specified

The problematic command generated by PyTorch's build system includes flags that NVCC 13.0 cannot properly parse:

nvcc --generate-dependencies-with-compile --dependency-output [path].obj.d [other flags] -c file.cu -o output.obj

Root Cause Analysis

NVIDIA made changes to NVCC's command-line parser in CUDA 13.0 that conflict with how PyTorch's ninja build system constructs compilation commands. Specifically:

1. Dependency Generation Flags: The combination of --generate-dependencies-with-compile and --dependency-output with other flags causes parsing conflicts
2. Command Structure: NVCC 13.0 is more strict about flag ordering and combinations
3. Windows-Specific: This issue appears to be Windows-specific due to path handling and flag escaping differences

Impact

This affects users with:

• RTX 50xx series GPUs (requiring CUDA 12.8+ for sm_120 support)
• CUDA 13.0 installations
• Windows development environments
• Cutting-edge PyTorch nightly builds

Many RTX 5090 users require CUDA 13.0 for optimal performance and compatibility.

Attempted Solutions

We've tried several workarounds:

1. Environment variables (USE_NINJA=0, DISTUTILS_USE_SDK=1)
2. PyTorch cpp_extension patching
3. Real-time ninja file modification
4. Direct setup.py compilation approaches

None successfully resolve the core NVCC command-line parsing issue.

Proposed Solutions

Short-term

1. Pre-built Wheels: Provide CUDA 13.0 Windows wheels for major Python versions (3.11, 3.12, 3.13)
2. Build Instructions: Document CUDA 12.8 as the recommended version for Windows compilation
3. Alternative Toolchain: Investigate MSBuild instead of ninja for Windows CUDA 13.0

Long-term

1. NVCC Compatibility: Work with NVIDIA and PyTorch teams to resolve command-line generation
2. Build System Updates: Modify build scripts to generate CUDA 13.0-compatible commands
3. Conditional Compilation: Detect CUDA version and adjust build flags accordingly

Workaround for Users

Current recommended approach for RTX 5090 users:

1. Use CUDA 12.8 instead of 13.0 (performance impact is minimal)
2. Install available pre-built wheels when possible
3. Use SageAttention 1.0 (Triton-only) as fallback: pip install sageattention==1.0.6

Additional Context

This issue affects the broader PyTorch ecosystem, not just SageAttention. Similar reports exist for other CUDA extensions on Windows with CUDA 13.0. The issue appears to be a compatibility gap that will likely require coordination between multiple teams (NVIDIA, PyTorch, extension maintainers).

We appreciate the work on SageAttention and understand this is a broader ecosystem challenge. Any guidance or timeline for CUDA 13.0 Windows support would be helpful for the community.

System Information

PyTorch: 2.9.0.dev20250909+cu130
CUDA available: True
CUDA version: 13.0
Device name: NVIDIA GeForce RTX 5090
Compute Capability: (12, 0)

[manchuwook]

I used to see these things with #PRAGMAS before in the past - I don't know if there is an equivalent.

[Arcitec]

Thanks for documenting the issue.

Since this is an issue in Ninja, did you file a ticket there?

[igor-iki]

so anyone have wheel for cuda13? cant build sageattention with pytorch 2.10.0.dev20250926+cu130
error: command 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v13.0\\bin\\nvcc.EXE' failed with exit code 4294967295 [end of output]

[manchuwook]

Thanks for documenting the issue.

Since this is an issue in Ninja, did you file a ticket there?

I had not, but I can look into it.

[alessandro893]

I managed to build SageAttention for Pytorch 2.9.0 + cuda 13

SageAttention2:

sageattention-2.2.0-cp312-cp312-win_amd64.zip

SageAttention3:

sageattn3-1.0.0-cp312-cp312-win_amd64.zip

[rugabunda]

@manchuwook here is a patch I made that allowed me to build it:

#228 (comment)

I made a script that automates this process that was tested with the latest bleeding cuda 13.0; pytorch 2.9; Python 3.12.11. Probably earlier versions that were also broken.

Activate your environment

git clone https://github.com/thu-ml/SageAttention.git

Drop the code below into a file in its root and run:

python build_sage.py

# Sageattention build patcher
import os
import subprocess
import sys

# Apply patches
print("Applying patches...")
files = ['csrc/fused/fused.cu', 'csrc/qattn/qk_int_sv_f16_cuda_sm80.cu']

for f in files:
    if os.path.exists(f):
        with open(f, 'r') as file:
            content = file.read()
        if '#include <vector>' not in content:
            content = '#include <vector>\n#include <typeindex>\n' + content
            with open(f, 'w') as file:
                file.write(content)
            print(f"✓ Patched {f}")

# Patch setup.py
with open('setup.py', 'r') as f:
    setup_content = f.read()

setup_content = setup_content.replace(
    'CXX_FLAGS = ["/O2", "/openmp"]',
    'CXX_FLAGS = ["/O2", "/openmp", "/std:c++20", "/DENABLE_BF16"]'
)
setup_content = setup_content.replace('"-std=c++17"', '"-std=c++20"')

with open('setup.py', 'w') as f:
    f.write(setup_content)

print("✓ Patches applied successfully")

# Set environment variables
os.environ['EXT_PARALLEL'] = '4'
os.environ['NVCC_APPEND_FLAGS'] = '--threads 8'
os.environ['MAX_JOBS'] = '32'

# Uninstall old version first
subprocess.run([sys.executable, '-m', 'pip', 'uninstall', 'sageattention', '-y'])

# Build and install with pip
print("\nBuilding with pip...")
subprocess.run([sys.executable, '-m', 'pip', 'install', '.', '--no-build-isolation'], check=True)
print("\n✓ Build complete!")

[aaronjolson]

Wow, thanks @rugabunda ! I did not think this would work for me since I am running in a conda env, but it totally did, this is great!

[JasonHoku]

@manchuwook here is a patch I made that allowed me to build it:

#228 (comment)

I made a script that automates this process that was tested with the latest bleeding cuda 13.0; pytorch 2.9; Python 3.12.11. Probably earlier versions that were also broken.

Activate your environment

git clone https://github.com/thu-ml/SageAttention.git

Drop the code below into a file in its root and run:

python build_sage.py

# Sageattention build patcher
import os
import subprocess
import sys

# Apply patches
print("Applying patches...")
files = ['csrc/fused/fused.cu', 'csrc/qattn/qk_int_sv_f16_cuda_sm80.cu']

for f in files:
    if os.path.exists(f):
        with open(f, 'r') as file:
            content = file.read()
        if '#include <vector>' not in content:
            content = '#include <vector>\n#include <typeindex>\n' + content
            with open(f, 'w') as file:
                file.write(content)
            print(f"✓ Patched {f}")

# Patch setup.py
with open('setup.py', 'r') as f:
    setup_content = f.read()

setup_content = setup_content.replace(
    'CXX_FLAGS = ["/O2", "/openmp"]',
    'CXX_FLAGS = ["/O2", "/openmp", "/std:c++20", "/DENABLE_BF16"]'
)
setup_content = setup_content.replace('"-std=c++17"', '"-std=c++20"')

with open('setup.py', 'w') as f:
    f.write(setup_content)

print("✓ Patches applied successfully")

# Set environment variables
os.environ['EXT_PARALLEL'] = '4'
os.environ['NVCC_APPEND_FLAGS'] = '--threads 8'
os.environ['MAX_JOBS'] = '32'

# Uninstall old version first
subprocess.run([sys.executable, '-m', 'pip', 'uninstall', 'sageattention', '-y'])

# Build and install with pip
print("\nBuilding with pip...")
subprocess.run([sys.executable, '-m', 'pip', 'install', '.', '--no-build-isolation'], check=True)
print("\n✓ Build complete!")

YOU LEGEND!!

After an hour of struggling to build sageattention on the latest UI version, I'm happy to report:
This worked perfectly and easily on a newer version, ComfyUI 0.12.3 with Python version: 3.13.11 and pytorch version: 2.10.0+cu130

[KLL535]

Does anyone have a whl sageattn3 for:
CUDA: 13.1
Python: 3.13
PyTorch: 2.11.0.dev20260211+cu130
Does not compile (error C2872: 'std': ambiguous symbol)

[sideraldx]

@manchuwook Aquí hay un parche que hice que me permitió construirlo:
#228 (comentario)
Hice un script que automatiza este proceso que fue probado con la última versión de Bleeding Cuda 13.0; PyTorch 2.9; Python 3.12.11. Probablemente versiones anteriores que también estaban rotas.
Activa tu entorno
git clone https://github.com/thu-ml/SageAttention.git
Coloque el siguiente código en un archivo en su raíz y ejecute:
python build_sage.py

# Sageattention build patcher
import os
import subprocess
import sys

# Apply patches
print("Applying patches...")
files = ['csrc/fused/fused.cu', 'csrc/qattn/qk_int_sv_f16_cuda_sm80.cu']

for f in files:
    if os.path.exists(f):
        with open(f, 'r') as file:
            content = file.read()
        if '#include <vector>' not in content:
            content = '#include <vector>\n#include <typeindex>\n' + content
            with open(f, 'w') as file:
                file.write(content)
            print(f"✓ Patched {f}")

# Patch setup.py
with open('setup.py', 'r') as f:
    setup_content = f.read()

setup_content = setup_content.replace(
    'CXX_FLAGS = ["/O2", "/openmp"]',
    'CXX_FLAGS = ["/O2", "/openmp", "/std:c++20", "/DENABLE_BF16"]'
)
setup_content = setup_content.replace('"-std=c++17"', '"-std=c++20"')

with open('setup.py', 'w') as f:
    f.write(setup_content)

print("✓ Patches applied successfully")

# Set environment variables
os.environ['EXT_PARALLEL'] = '4'
os.environ['NVCC_APPEND_FLAGS'] = '--threads 8'
os.environ['MAX_JOBS'] = '32'

# Uninstall old version first
subprocess.run([sys.executable, '-m', 'pip', 'uninstall', 'sageattention', '-y'])

# Build and install with pip
print("\nBuilding with pip...")
subprocess.run([sys.executable, '-m', 'pip', 'install', '.', '--no-build-isolation'], check=True)
print("\n✓ Build complete!")

¡¡ERES LEYENDA!!

Después de una hora de luchar para crear sageattention en la última versión de la interfaz de usuario, me complace informar: Esto funcionó perfecta y fácilmente en una versión más nueva, ComfyUI 0.12.3 con la versión de Python: 3.13.11 y la versión de PyTorch: 2.10.0+cu130

brother could you make a tutorial, I'm in the same problem, but I really can't solve it