[Windows][Python 3.12] Torch 2.6.0+cu124 fails with Triton ImportError: AttrsDescriptor missing

[SkybuckFlying]

Describe the bug

🐛 Describe the bug

[Windows][Python 3.12] Torch 2.6.0+cu124 fails with Triton ImportError: AttrsDescriptor missing

Description

On Windows 11 + Python 3.12, installing the CUDA 12.4 builds of PyTorch and running Unsloth (or Hugging Face/TorchAO quantization) fails due to a Triton import error. The bundled Triton runtime in PyTorch appears incompatible with the expected API.

Environment

• OS: Windows 11
• Python: 3.12
• Torch: 2.6.0+cu124
• Torchvision: 0.21.0+cu124
• xformers: 0.0.29.post3
• GPU: NVIDIA GeForce RTX 4070 Ti (CUDA 12.4)
• Package: Unsloth (latest pip release)

Steps to Reproduce

1. Create a Python 3.12 venv on Windows:

   python -m venv unsloth_env
   .\unsloth_env\Scripts\Activate.ps1
   

2. Install pinned packages:

pip install torch==2.6.0+cu124 torchvision==0.21.0+cu124 xformers==0.0.29.post3 --index-url https://download.pytorch.org/whl/cu124
pip install unsloth

3. Run:
   python -m unsloth run unsloth/Nemotron-3-Nano-30B-A3B-GGUF --file Nemotron-3-Nano-30B-A3B-F16.gguf

Actual Behavior
Crash occurs during Unsloth startup patches:

ImportError: cannot import name 'AttrsDescriptor' from 'triton.compiler.compiler'
(G:\tools\Unsloth\unsloth_env\Lib\site-packages\triton\compiler\compiler.py)

Full stack trace shows attempting to import from Triton, which is missing.
Expected Behavior
PyTorch should ship a Triton runtime compatible with TorchAO/Hugging Face quantization modules on Windows + Python 3.12. Model inference should proceed without Triton import errors.
Notes
• On Linux or Python 3.10/3.11, Triton wheels are available and this error does not occur.
• On Windows + Python 3.12, pip cannot install Triton (), so users are stuck with the bundled runtime.
• This blocks Unsloth and Hugging Face quantization workflows on Windows with Python 3.12.
Suggested Fix
• Publish Triton wheels for Windows + Python 3.12.
• Or bundle a compatible Triton runtime inside PyTorch 2.6.0+cu124 for Windows.
• Alternatively, add guards in TorchAO/Hugging Face to skip Triton‑dependent patches when symbols are missing.

Environment Report (first attempt failed: wget not available on windows 11)

Versions

G:\Junk>wget https://raw.githubusercontent.com/pytorch/pytorch/main/torch/utils/collect_env.py
'wget' is not recognized as an internal or external command,
operable program or batch file.

G:\Junk>

Environment Report (second attempt better: see below)

(unsloth_env) PS G:\tools\Unsloth> python collect_env.py
Collecting environment information...
PyTorch version: 2.6.0+cu124
Is debug build: False
CUDA used to build PyTorch: 12.4
ROCM used to build PyTorch: N/A

OS: Microsoft Windows 11 Pro (10.0.22631 64-bit)
GCC version: Could not collect
Clang version: 21.1.3
CMake version: Could not collect
Libc version: N/A

Python version: 3.12.2 (tags/v3.12.2:6abddd9, Feb 6 2024, 21:26:36) [MSC v.1937 64 bit (AMD64)] (64-bit runtime)
Python platform: Windows-11-10.0.22631-SP0
Is CUDA available: True
CUDA runtime version: 12.6.77
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 4070 Ti
Nvidia driver version: 580.88
cuDNN version: Could not collect
Is XPU available: False
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True
Caching allocator config: N/A

CPU:
Name: AMD Ryzen 9 7950X3D 16-Core Processor
Manufacturer: AuthenticAMD
Family: 107
Architecture: 9
ProcessorType: 3
DeviceID: CPU0
CurrentClockSpeed: 4201
MaxClockSpeed: 4201
L2CacheSize: 16384
L2CacheSpeed: None
Revision: 24834

Versions of relevant libraries:
[pip3] numpy==2.3.5
[pip3] torch==2.6.0+cu124
[pip3] torchao==0.14.1
[pip3] torchvision==0.21.0+cu124
[pip3] triton-windows==3.5.1.post22
[conda] Could not collect
(unsloth_env) PS G:\tools\Unsloth>

FULL ERROR TRACE:

(unsloth_env) PS G:\tools\Unsloth> python -m unsloth run unsloth/Nemotron-3-Nano-30B-A3B-GGUF --file Nemotron-3-Nano-30B-A3B-F16.gguf
🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.
Unsloth: Failed editing tqdm to replace Inductor Compilation:
🦥 Unsloth: Will patch your computer to enable 2x faster free finetuning.
Traceback (most recent call last):
  File "G:\tools\Unsloth\unsloth_env\Lib\site-packages\torch\_inductor\runtime\hints.py", line 46, in <module>
    from triton.backends.compiler import AttrsDescriptor
ImportError: cannot import name 'AttrsDescriptor' from 'triton.backends.compiler' (G:\tools\Unsloth\unsloth_env\Lib\site-packages\triton\backends\compiler.py)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "G:\tools\Unsloth\unsloth_env\Lib\site-packages\unsloth_zoo\temporary_patches\utils.py", line 107, in <module>
    from transformers.processing_utils import Unpack
  File "G:\tools\Unsloth\unsloth_env\Lib\site-packages\transformers\processing_utils.py", line 75, in <module>
    from .modeling_utils import PreTrainedAudioTokenizerBase
  File "G:\tools\Unsloth\unsloth_env\Lib\site-packages\transformers\modeling_utils.py", line 73, in <module>
    from .quantizers import HfQuantizer
  File "G:\tools\Unsloth\unsloth_env\Lib\site-packages\transformers\quantizers\__init__.py", line 14, in <module>
    from .auto import AutoHfQuantizer, AutoQuantizationConfig, register_quantization_config, register_quantizer
  File "G:\tools\Unsloth\unsloth_env\Lib\site-packages\transformers\quantizers\auto.py", line 62, in <module>
    from .quantizer_torchao import TorchAoHfQuantizer
  File "G:\tools\Unsloth\unsloth_env\Lib\site-packages\transformers\quantizers\quantizer_torchao.py", line 39, in <module>
    import torchao
  File "G:\tools\Unsloth\unsloth_env\Lib\site-packages\torchao\__init__.py", line 110, in <module>
    from torchao.quantization import (
  File "G:\tools\Unsloth\unsloth_env\Lib\site-packages\torchao\quantization\__init__.py", line 6, in <module>
    from .autoquant import (
  File "G:\tools\Unsloth\unsloth_env\Lib\site-packages\torchao\quantization\autoquant.py", line 11, in <module>
    from torchao.dtypes import (
  File "G:\tools\Unsloth\unsloth_env\Lib\site-packages\torchao\dtypes\__init__.py", line 1, in <module>
    from . import affine_quantized_tensor_ops
  File "G:\tools\Unsloth\unsloth_env\Lib\site-packages\torchao\dtypes\affine_quantized_tensor_ops.py", line 11, in <module>
    from torchao.dtypes.affine_quantized_tensor import (
  File "G:\tools\Unsloth\unsloth_env\Lib\site-packages\torchao\dtypes\affine_quantized_tensor.py", line 17, in <module>
    from torchao.quantization.quant_primitives import (
  File "G:\tools\Unsloth\unsloth_env\Lib\site-packages\torchao\quantization\quant_primitives.py", line 206, in <module>
    register_custom_op = _register_custom_op(quant_lib)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\tools\Unsloth\unsloth_env\Lib\site-packages\torchao\utils.py", line 210, in _register_custom_op
    from torch._inductor.decomposition import register_decomposition
  File "G:\tools\Unsloth\unsloth_env\Lib\site-packages\torch\_inductor\decomposition.py", line 27, in <module>
    from torch._inductor.utils import pad_listlike
  File "G:\tools\Unsloth\unsloth_env\Lib\site-packages\torch\_inductor\utils.py", line 50, in <module>
    from torch._inductor.runtime.hints import DeviceProperties
  File "G:\tools\Unsloth\unsloth_env\Lib\site-packages\torch\_inductor\runtime\hints.py", line 67, in <module>
    from triton.compiler.compiler import AttrsDescriptor
ImportError: cannot import name 'AttrsDescriptor' from 'triton.compiler.compiler' (G:\tools\Unsloth\unsloth_env\Lib\site-packages\triton\compiler\compiler.py)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<frozen runpy>", line 189, in _run_module_as_main
  File "<frozen runpy>", line 148, in _get_module_details
  File "<frozen runpy>", line 112, in _get_module_details
  File "G:\tools\Unsloth\unsloth_env\Lib\site-packages\unsloth\__init__.py", line 89, in <module>
    import unsloth_zoo
  File "G:\tools\Unsloth\unsloth_env\Lib\site-packages\unsloth_zoo\__init__.py", line 194, in <module>
    from .temporary_patches import (
  File "G:\tools\Unsloth\unsloth_env\Lib\site-packages\unsloth_zoo\temporary_patches\__init__.py", line 19, in <module>
    from .gemma import *
  File "G:\tools\Unsloth\unsloth_env\Lib\site-packages\unsloth_zoo\temporary_patches\gemma.py", line 22, in <module>
    from .utils import (
  File "G:\tools\Unsloth\unsloth_env\Lib\site-packages\unsloth_zoo\temporary_patches\utils.py", line 122, in <module>
    raise Exception(e)
Exception: cannot import name 'AttrsDescriptor' from 'triton.compiler.compiler' (G:\tools\Unsloth\unsloth_env\Lib\site-packages\triton\compiler\compiler.py)
(unsloth_env) PS G:\tools\Unsloth>

Diagnostic Note

Checking Triton installation inside the venv:

(unsloth_env) PS G:\tools\Unsloth> pip show triton
WARNING: Package(s) not found: triton
(unsloth_env) PS G:\tools\Unsloth> pip list | findstr triton
triton-windows     3.5.1.post22
(unsloth_env) PS G:\tools\Unsloth>

This confirms that the official triton package is not available on Windows + Python 3.12.
Instead, PyTorch bundles triton-windows, which lacks the AttrsDescriptor symbol required by TorchAO/Hugging Face quantization.

Explanation of problem:

The short answer is: the triton-windows package that ships with PyTorch on Windows + Python 3.12 is incomplete compared to the official Triton builds on Linux/macOS.

Here’s why it fails even though a Triton package is present:

---

🔎 What’s Happening

• Linux/macOS builds: PyTorch depends on the official triton wheels published to PyPI. These contain the full compiler/runtime API, including AttrsDescriptor.
• Windows + Python 3.12: There are no official triton wheels. Instead, PyTorch bundles a fallback package called triton-windows (you saw 3.5.1.post22 in your venv).
• Problem: triton-windows does not expose the same symbols as the official Triton package. In particular, AttrsDescriptor is missing from both triton.backends.compiler and triton.compiler.compiler.
• Result: When TorchAO/Hugging Face quantization code tries to import AttrsDescriptor, the import fails, causing the crash.

---

⚠️ Why It Matters

• PyTorch’s bundled Triton runtime is not feature‑complete on Windows. It’s enough for some inductor kernels, but not for higher‑level quantization modules that expect the full Triton API.
• That’s why this environment shows triton-windows installed, but the error trace still bombs out — the symbol simply isn’t there.

---

✅ What Needs Fixing

• Option 1: Publish official Triton wheels for Windows + Python 3.12, so users can pip install triton and get the full API.
• Option 2: Update triton-windows to include missing symbols like AttrsDescriptor.
• Option 3: Add guards in TorchAO/Hugging Face to skip Triton‑dependent imports when running on Windows with triton-windows.

---

So the reason it’s “not working” is that PyTorch is shipping a reduced Triton runtime (triton-windows) that doesn’t match the API TorchAO/Hugging Face expect. That mismatch is exactly what this error trace demonstrates.

Environment details

Also filed here:

pytorch/pytorch#170508

[woct0rdho]

Each PyTorch minor version is only guaranteed to work with a specific Triton minor version, see https://github.com/woct0rdho/triton-windows?tab=readme-ov-file#3-pytorch . Honestly it's hard to implement in package managers.

As you're using torch 2.6.0, you should install triton-windows<3.3.