Hand-written prologue: faster short calls, no compiler, RX 9060 XT opt-in (v0.4.0)

The quantization step in front of the attention kernel is hand-written HIP now. That brings short calls level with PR #368's instead of up to 1.6× slower, takes the Triton compiler off the default path, and removes the sequence-length ceiling.

python -m pip install --no-deps sageattention-2.2.0+amd.gfx12.4-cp312-cp312-win_amd64.whl

Requirements (unchanged): Windows 10/11, Python 3.12, PyTorch 2.13.0+rocm10.0.0, an RX 9070 or RX 9070 XT. Linux runs from source; Linux testing of this release is in #2.

What changed

• Short calls are faster. A call of a few hundred tokens took up to 1.6× PR #368's time; now it's about even (0.85–1.13, timed one call at a time). Image and video sizes are unchanged.
• Nothing to compile on first use. In ComfyUI with an empty Triton cache, the first call takes about 0.5 s instead of about 2 s.
• No length ceiling. v0.3.0 stopped serving around 100 000 tokens at H3's layout; the new prologue uses 64-bit offsets.
• Same output. Bit-identical to v0.3.0 with smooth_k off. With smooth_k on, the key mean is now summed in fp64, so a small share of outputs differ by rounding, with the same accuracy against exact attention.
• RX 9060 XT (gfx1200), opt in. Set SAGEATTN_SK1_GFX1200=1. The machine code is byte-identical to the RX 9070 builds, but nobody has run it on a 9060 XT yet. If you have one, docs/GFX1200-TESTING.md takes about 15 minutes.
• SAGEATTN_SK1_PROLOGUE=triton brings back v0.3.0's prologue if you ever need it.

Worth knowing

• head_dim 64 still uses PR #368's kernel. A second attempt, on the new prologue, was still slower, so that one needs its own design.
• PyTorch 2.14 / ROCm 10.1 isn't supported yet: on the current WHQL driver PyTorch 2.14's own kernels don't run. An experimental build is in #3.

Details in CHANGELOG.md and docs/FINDINGS.md (F162–F166).

sha256 0f2055e46a217f8490ab963ed4f8bfc82b669a621f048a7e72df0ff728416504

Re-packaged on 2026-10-19 with the corrected README and credits. The code and binaries are byte-identical to the first upload (sha256 1801413a…5762), so there is no need to reinstall.

PyTorch 2.14 test build of v0.4.0 (untested)

v0.4.0 built against PyTorch 2.14.0+rocm10.1.0, for testing in #3. It is not supported yet: it compiled and imports, but it hasn't run on a GPU. For PyTorch 2.13, use v0.4.0.

python -m pip install --no-deps sageattention-2.2.0+amd.gfx12.4-cp312-cp312-win_amd64.whl

Same code as v0.4.0. It has the same file name as the v0.4.0 wheel, so keep them apart; installed next to the wrong PyTorch, the import stops and names the version it needs.

sha256 7680a28b8ce868288adce9de7fda5251baa1334c24cf099ee2813e0427aea3c2

Fused q/k/v views on the hand-written kernels (v0.3.0)

Fused q/k/v views on the hand-written kernels, and Linux support when built from source.

python -m pip install --no-deps sageattention-2.2.0+amd.gfx12.3-cp312-cp312-win_amd64.whl

Requirements (unchanged): Windows 10/11, Python 3.12, PyTorch 2.13.0+rocm10.0.0, an RX 9070 or RX 9070 XT. Linux is supported from source: the loader carries @boxwrench's Linux branch (tested on a Radeon AI PRO R9700, Ubuntu 24.04, torch 2.9.1+rocm7.2.1), but this wheel is Windows-only and there is no prebuilt Linux wheel yet.

What changed

• MiniMax-H3's q/k/v are now served. H3 fuses q, k and v into one projection, so they arrive as views into a single tensor and k's row stride is 3*H*D instead of its own width. Earlier releases refused that layout, which meant the call quietly ran PR #368's path even with SAGEATTN_SK1_BF16=1. The acceptance rule is now the raw launch's own requirement, stated once in sk1_loader.strided_view_reason and used by both the pre-launch envelope and the loader's own check, so the two cannot disagree; a refusal carries its reason instead of a catch-all.
• Against PR #368 at six production sizes (N = 27 280 to 99 792), the full call is 6–13 % faster (0.87–0.94× its time). In a real H3 w4a8 int4 render at 864x480 and 124 frames, the sampling phase is 5 % faster at 4 steps and 15 % faster at 8 (0.950× and 0.847× its time), same checkpoint and seed.
• The version strings agree again: setup.py, sageattention/__init__.py and sageattention/_version.py all say 2.2.0+amd.gfx12.3.

Worth knowing

• Those margins are ratios, not milliseconds. This GPU's clock cannot be pinned, so every timing block is ratio-only.
• The sweep stops at N = 99 792 because that is where the launch's 32-bit addressing runs out, not because the margin does.
• head_dim 64 still uses PR #368's kernel. It has its own entry on the list.
• The native extension is byte-identical to v0.2.0's, deliberately: no C++ or HIP source changed in this release, so nothing was rebuilt.

There's a note from me at the end of the README about where this is going.

See the README for the numbers and docs/FINDINGS.md (F160–F161) for how they were measured.

sha256 f23517243970fc57d014661e8d568e21decf327106bf7de01ac36c65e04a7a96

v0.2.0

int8 Q·K for fp16, and bf16 models on the hand-written kernels.

python -m pip install --no-deps sageattention-2.2.0+amd.gfx12.2-cp312-cp312-win_amd64.whl

Requirements (unchanged): Windows 10/11, Python 3.12, PyTorch 2.13.0+rocm10.0.0, an RX 9070 or RX 9070 XT.

What changed

• fp16 now computes Q·K in int8, with one scale per token. Inside real Krea2 renders its per-call error is 0.82× the fp8 kernel's, at the same step time. SAGEATTN_SK1_INT8=0 brings back v0.1.0's fp8 kernel, bit for bit.
• bf16 models now run on the hand-written kernels (on by default). Against PR #368's bf16 path: 0.76 of its time non-causal, 0.44 causal. In a bf16 Krea2 render: 6.9 % faster per step than PyTorch SDPA at 1 MP and 14.2 % at 2 MP. SAGEATTN_SK1_BF16=0 sends bf16 back to PR #368's path.
• Fallback order for fp16 is int8 kernel, fp8 kernel, PR #368. Nothing raises unless you set a switch to 1.

Worth knowing

• On long sequences (around 47k tokens, non-causal, i.e. 720p-video lengths) int8 is up to 1.7 % slower per attention call than the fp8 kernel. If you'd rather have the speed there, set SAGEATTN_SK1_INT8=0. bf16 models never use int8, so they aren't affected.
• head_dim 64 still uses PR #368's kernel. A head_dim 64 kernel was built and tested, and it was slower.

See the README for the numbers and docs/FINDINGS.md (F155–F159) for how they were measured.

sha256 b6b76ad32b4ccddae728341d0c3f345f57c7e459fa4fa62152dc711516b566db

v0.1.0

First release.

A drop-in sageattention build for AMD Radeon RX 9070 / RX 9070 XT (gfx1201) on Windows. fp16 attention with head_dim 128 runs on a hand-written fp8 HIP kernel; everything else uses the gfx12 kernels from SageAttention PR #368.

Requirements: Windows 10/11, Python 3.12, PyTorch 2.13.0+rocm10.0.0, an RX 9070 or RX 9070 XT.

python -m pip install --no-deps sageattention-2.2.0+amd.gfx12.1-cp312-cp312-win_amd64.whl

Speed against PR #368's kernel (full call, quantization included; ratios below 1.0 are faster):

• causal: 0.51 – 0.65
• non-causal: 0.77 – 0.94

In ComfyUI (Krea2, fp16, 8 steps, stock --use-sage-attention call): 5.4 % faster per step than PyTorch SDPA at 1 MP and 13.7 % faster at 2 MP.

Notes

• smooth_k is on by default; it improved accuracy on every real capture measured.
• Set SAGEATTN_SK1_BACKEND=0 to disable the hand-written kernel, or =1 to require it.
• The compiled extensions match this exact PyTorch version; build from source for any other.

See the README for results and accuracy, and docs/ for the full write-up.

sha256 64c7e235ee683c549e68e9af307b38cc60710efd375586841cd3ad188cd3e25a