v0.6.2 — SM86 / RTX 30 Support

v0.6.2 — SM86 / RTX 30 Support

• Added MiniMax H3 Sol Attention support for SM86 Ampere GPUs, including RTX 30-series cards.
• SM86 uses the existing pointer-kernel family shared with SM89 and SM120; SM90/SM100/SM121 remain on their existing TMA paths.
• Community validated on an RTX 3090 Ti in strict mode at 25,323 tokens, with both BF16 and residual-int8 QK/PV configurations completing successfully.
• The six repository tests applicable to SM86 plus a contributor runtime-stack check passed; the SM120-only comparison skipped as expected.
• After merge, all seven repository tests passed on SM120, including pointer-vs-TMA parity, INT8 paths, H3 modulation fusion, and KJNodes handoff coverage.
• No changes to attention math, model weights, sparsity settings, output quality, or existing architecture paths.
• SM86 performance has not yet been formally benchmarked.

Thanks to @craftingmod for the SM86 implementation and RTX 3090 Ti validation.

v0.6.1 — KJNodes Low-VRAM compatibility

Fixed

• Added compatibility with KJNodes' MiniMax H3 Low VRAM Attention activation-list handoff.
• Fixed 'list' object has no attribute 'shape' when using KJ Low VRAM Attention before either this repository's MiniMax H3 Memory Efficient Sol Attention or Scheduled Sol Attention node.
• Sol now inspects the handed-off tensor without consuming it during eligibility checks, preserves the list for dense fallback, and consumes/releases it when the Sol path runs. KJ's early activation release remains intact.

Compatibility

• Supported: KJ MiniMax H3 Low VRAM Attention → MiniMax H3 Memory Efficient Sol Attention
• Supported: KJ MiniMax H3 Low VRAM Attention → MiniMax H3 Scheduled Sol Attention
• Still unsupported: combining KJ Low VRAM Attention with MiniMax H3 Fused Modulation, because both patch the complete H3 block forward.

Validation

• All 7 regression tests pass.
• Real KJ low-VRAM block-forward GPU integration passes.
• No attention math, model weights, numerical accuracy, or architecture dispatch changed.
• SM89/SM120 remain on the pointer path; SM90/SM100/SM121 remain on TMA.
• No benchmark rerun was required; the v0.6.0 benchmark matrix remains current.

v0.6.0 - SM120 pointer path and exact H3 fusion

Highlights

• Faster SM120 forward dispatch: RTX 5090 now uses the pointer forward kernels. SM89 remains on pointer kernels; SM90, SM100, and SM121 remain on their existing TMA paths.
• Inline residual-INT8 Q preparation: SM89/SM120 diagonal-threshold pointer kernels derive Q quantization and routing thresholds from the BF16 Q tile already loaded by the forward kernel. At 32K tokens this reduced peak working allocation by 189.1 MiB versus the v0.5.9 materialized-Q path, with bit-identical output.
• Exact MiniMax H3 modulation fusion: the new MiniMax H3 Fused Modulation node fuses segmented AdaLN scale/shift and gated residual updates while explicitly preserving eager BF16 rounding. A real ComfyUI DiTBlock test remained bit-identical.
• Patch composition preserved: global KJ Sage -> KJ MiniMax H3 memory-efficient Sage -> local H3 Sol continues to compose. The fusion resolves attention and MLP implementations at call time, and attention calls outside local Sol remain on their selected backend.
• No cache work is included in this release.

RTX 5090 release benchmarks

MiniMax H3 shape (B=1, H=56, D=128, BF16 inputs, tau=1.0), median of 20 calls after three warmups; second process run after autotune caching:

Method	8K	16K	32K	65K
PyTorch SDPA	23.97 ms	94.85 ms	378.41 ms	1,516.16 ms
SageAttention	3.93 ms	14.68 ms	58.09 ms	229.63 ms
Sol BF16	2.84 ms	9.57 ms	35.72 ms	139.28 ms
Sol residual int8_qk	2.27 ms	8.10 ms	30.06 ms	116.64 ms
Sol int8_qk+pv	1.98 ms	7.05 ms	25.91 ms	98.46 ms

Across 8K-65K, BF16 reaches 1.38-1.65x Sage throughput, residual INT8 reaches 1.73-1.97x, and opt-in INT8+PV reaches 1.98-2.33x.

Accuracy and validation

• Residual int8_qk relative L2 versus Sol BF16: 0.00802.
• Opt-in int8_qk+pv relative L2 versus Sol BF16: 0.01396; it remains default-off.
• All-exact Sol versus PyTorch SDPA relative L2: 0.000973.
• SM120 pointer versus TMA output: bit-identical for BF16 and residual INT8.
• Inline-Q versus the former materialized-Q path: bit-identical across aligned/ragged tails, exact sinks, dense query rows, and INT8+PV.
• H3 fused modulation/gating versus eager BF16: bit-identical.
• Five GPU regression tests, Python compilation, package import, and node registration all passed.

Full measurements and historical third-party comparisons: BENCHMARKS.md.

v0.5.9

What's changed

• Fused K's 64-token block-mean reduction and residual INT8 quantization into one Triton kernel, eliminating the second full read of K.
• Kept the residual-key INT8 formulation and FP32 reduction accumulation. Validation found no routing flips.
• Applied the optimization in shared preprocessing only. SM89 keeps its pointer forward kernels; SM90, SM100, SM120, and SM121 keep their TMA forward kernels.
• Updated the English and Chinese READMEs with the exact KJNodes composition order.

Performance and accuracy validation

RTX 5090 (SM120), torch 2.10.0+cu130, Triton 3.6.0:

• Isolated K/V-summary plus K-quant preprocessing: 26–36% faster at 8K, 16K, and 65K tokens. The 32K result was faster but noisier.
• All-exact bf16 versus PyTorch SDPA: 0.000983 relative L2.
• Residual INT8 versus bf16: 0.00982 relative L2.
• No routing flips in the tested 8K and 32K comparisons.

KJNodes MiniMax H3 order

1. Patch Sage Attention (KJNodes)
2. MiniMax H3 Memory Efficient Sage Attention Patch (KJNodes)
3. MiniMax H3 Scheduled or Memory Efficient Sol Attention Patch (this repository)

Sol-eligible tokens use Sol-Attn. Short, gated, unsupported, or failed non-strict calls use the captured MiniMax memory-efficient Sage forward. Attention outside that H3 object patch continues to use the global Sage override.

Full changelog: v0.5.8...v0.5.9

v0.5.8 — default tau 1.3, int8_pv optimizations

v0.5.8

Changes

Default tau raised to 1.3

The fixed-tau nodes (generic Sol-Attn patch, MiniMax H3 Memory Efficient patch) now default to tau=1.3 instead of the paper's 1.0, and the scheduled node's tau_start moved from 1.25 to 1.3. Tuned for MiniMax H3 video workflows — higher sparsity on the default path with no visible quality loss in local testing. Lower it back to 1.0 if you want the paper's exact recipe.

int8_pv kernel optimizations

• V absmax fused into _reduce_kv — V's per-channel quantization scale is now computed inside the same kernel pass that already reads V for the block-mean summary (WRITE_AMAX constexpr on _reduce_vc_kernel). Previously quantize_v_per_channel did a separate v.abs().amax() — a full extra read of V. Saves that read whenever int8_pv is enabled.
• Block-max P exponentiation — the int8 P quantization now exponentiates against the per-block max instead of the running softmax max. P lands directly in [0, 1], so the softmax rescale and the quantization scale fold into one constant; one less division per routed block and lower register pressure.

Docs honesty pass on int8_pv

The int8_pv widget tooltips and README parameter rows previously claimed a speedup over int8_qk. Measured across reruns it is within autotune noise of int8_qk alone on RTX 5090 — the docs now say so. It remains available as an opt-in (accuracy 0.014 rel L2 vs bf16, still 2.1x more accurate than full-key int8 designs at 0.029).

Verification (RTX 5090, SM120)

• int8_qk vs bf16: rel L2 0.0080 (unchanged)
• int8_qk+pv vs bf16: rel L2 0.0139 (unchanged)
• all-exact mode vs SDPA: rel L2 0.00097 (unchanged)
• strided-view output bit-identical to contiguous input, including ragged lengths
• tau=1.3 default path + conditioning sinks: finite, correct

Since v0.5.5

• v0.5.5 added: int8_pv (opt-in), SM121/DGX Spark support, fresh cross-repo benchmarks (our int8_qk fastest at 8K), platform-agnostic rounding, autotune compat probe.
• The DGX Spark community benchmark (1.48-1.92x vs SageAttention) is in BENCHMARKS.md.

v0.5.5 — int8_pv, SM121/DGX Spark, fresh cross-repo benchmarks

v0.5.1 to v0.5.5 Changelog

v0.5.5

New features

• int8_pv — opt-in quantization of the P-V dot product (per-token P scale, per-channel V scale), building on the existing int8_qk residual path. Available on all three Sol-Attn nodes. Accuracy: 0.014 rel L2 vs bf16 (from 0.008 with int8_qk alone), still 2.1x more accurate than full-key int8 designs (kijai: 0.029). Default off; enable for long sequences where every millisecond matters.
• SM121 (DGX Spark) support added to all architecture gates, README, and README_ZH. SM121 shares the same TMA kernel path as SM120. (Community PR #3.)
• Platform-agnostic rounding (_round_to_int8) replaces libdevice.rint — fixes compatibility with Triton builds that lack the libdevice extra.
• Autotune compatibility probe for older Triton versions (cache_results / do_bench guarded behind a runtime check).

Benchmarks (2026-08-07)

Full cross-repo rerun — 8 methods, same inputs, one session:

Method	8,192	16,384	32,768	65,536
PyTorch SDPA	22.49	96.86	349.03	1,405.18
SageAttention	3.80	15.15	54.69	234.51
KingGore flex	4.57	10.56	32.23	134.77
kijai bf16	2.90	9.05	33.16	145.33
kijai int8	2.60	7.78	27.85	113.88
ours bf16	3.20	10.37	37.91	151.63
ours int8_qk	2.52	8.17	28.75	114.81
ours int8_qk+pv	3.04	8.91	29.99	118.11

Highlights:

• Our int8_qk is fastest at 8K (2.52 ms vs kijai 2.60). kijai leads 16K-65K by 3-5%.
• kijai's latest commit (0a92202) diverged from our bf16 path (0.0018 rel L2); the installed 0e334dc was bit-identical.
• Our int8 accuracy remains 3.6x better than kijai's full-key int8 (0.008 vs 0.029 rel L2).

Accuracy

Method	vs SDPA (all-exact), 8K	vs ours bf16 (tau=1), 8K
ours bf16	0.00097	0
ours int8_qk	0.00980	0.00803
ours int8_qk+pv	0.01741	0.01398
kijai int8	0.00987	0.02855

---

v0.5.2

• int8_qk kernel improvements — the int8 exact path now uses a leaner tile layout and corrected scaling. Reduces numerical overhead in the residual dot.
• Autotune fixes — _lean_do_bench integration hardened for edge-case Triton versions.
• Updated BENCHMARKS.md with the 2026-08-05 cross-repo rerun numbers.

---

v0.5.1

• Cross-repo benchmark suite — first full comparison run against kijai's Triton Sol-Attn, KingGore's flex_attention fork, SageAttention, and PyTorch SDPA at H3 width.
• int8_qk accuracy validated — confirmed 0.008 rel L2 vs the bf16 path (~3.7x more accurate than full-key int8 at the time).
• Updated README/README_ZH with measured speedup numbers for the strided zero-copy path vs the copy-based hook path.
• Updated peak VRAM tables (attention + FFN chunking).

0.5.2

ComfyUI-sol-attn v0.5.1 – v0.5.2

v0.5.2 — SM89 node-layer fix

• Fixed SM89 (RTX 40-series) falling back on every call. The kernel's validator accepted (8, 9) since v0.4.0, but the node-layer architecture gates (SUPPORTED_ARCHES in nodes.py, SOL_ARCHES in minimax.py) never included it — so on real SM89 hardware every Sol call raised _Unsupported before reaching the kernel. Added (8, 9) to both sets. Two-line fix. First real-hardware SM89 confirmation: RTX 4080 SUPER, torch 2.13.0+cu130, triton-windows 3.

v0.5.1 — per-arch dispatch flip

• Speed parity achieved. Measured TMA vs pointer kernels head-to-head on SM120: bf16 is faster through pointers (up to 18% at 65K tokens), int8 is faster through TMA. Dispatch now picks per path — bf16 → pointer kernels, int8 → TMA on SM120; SM90/SM100 keep TMA; SM89 uses pointer exclusively. Pure load-path change; outputs bit-identical, accuracy advantage untouched.
• Result: every measured cell within ~2–4% of kijai's Triton node (autotune-pick territory); ours takes bf16 at 8K tokens outright.
• BENCHMARKS.md and "Why this repo" updated to reflect parity standing.

0.5.0

ComfyUI-sol-attn v0.5.0

Highlights

• dense_blocks widget on both MiniMax H3 Sol nodes — keep chosen transformer blocks fully dense with a simple spec like 0-2,-1 (first three and last block; negatives count from the end). First/last blocks are the most approximation-sensitive, so this is the cheap quality lever when pushing higher tau. Blocks are simply left unpatched — zero runtime cost.
• Autotune timings now persist to disk (cache_results) and the sweep uses an L2-sized flush buffer instead of Triton's flat 256 MB one — the per-token-count JIT tax is paid once ever per size, not once per process, and the sweep itself peaks lower.
• New defaults from pipeline testing: min_tokens 4,096 on all Sol nodes, tau_start 1.25 on the scheduled node, and Chunk FeedForward min_tokens 8,192.
• README pipeline guide — where every node goes in the chain (loader → optional KJ Patch Sage Attention → Sol patch → Chunk FeedForward → optional EasyCache → sampler), including how KJNodes' sage patches compose with ours (backend patch in front works and covers gated steps; the mem-eff sage object patch is adopted as fallback when applied before ours, and shadows us if applied after).

Benchmarks

BENCHMARKS.md updated with a 2026-08-05 rerun against kijai's refreshed repo and a pipeline simulation: 22-step run, first 20% of steps dense on SageAttention, the rest Sol with the cosine tau ramp — 1.22–1.26× vs all-Sage and 7.6–8.1× vs stock SDPA on attention math (best with int8_qk). Honest standings: after kijai's pointer-default update his kernels lead ours by ~3–10% on speed; ours keeps the ~3.7× int8 accuracy advantage (0.008 vs 0.030 rel L2) and the feature set above.

Notes

• Full test suite passes on RTX 5090 (torch 2.10.0+cu130, Triton 3.6.0), including dense-block skip counting and schedule/graph checks.
• Contains no kernel behavior changes vs v0.4.8 — benchmark numbers shift only from autotune caching.

Full changelog: v0.4.8...v0.5.0

0.4.8

ComfyUI-sol-attn v0.4.8 (includes v0.4.7)

v0.4.8 — critical fix

• Fixed an int32 addressing overflow that crashed runs past ~99,900 packed tokens. The pointer-based preprocess/quantization kernels computed rows × 21,504 (H3's fused-qkv stride) in int32; at 103K tokens the offsets wrapped and produced illegal memory accesses that surfaced asynchronously in unrelated code (prefetch/offload paths), making the crashes look like loader or VRAM problems. Addressing is now int64 exactly where it can overflow. Verified at 103,237 tokens: all finite, bit-identical to contiguous input, int8 path working.
• Cost: bf16 speed unchanged; int8 pays ~7% at 65K tokens only, smaller sizes unaffected or faster.
• Documented that PyTorch SDPA itself overflows int32 on H3's strided qkv views past ~100K tokens (BENCHMARKS.md).

If you saw "CUDA error: an illegal memory access was encountered" on long MiniMax H3 generations with v0.4.x, upgrade.

v0.4.7 — int8_qk rework + cross-repo benchmarks

• Redesigned int8_qk for accuracy and speed: K is quantized per token after centering by its 64-token block mean, so only the small per-block residual goes through the int8 dot while the mean term stays exact in bf16 (added back from the routing scores the kernel already computes). q is quantized per token, fused with the diag routing threshold (one Q read), and int8 tiles load through pointers on all arches.
• Best measured int8 accuracy: 0.008 relative L2 vs the exact bf16 path — ~3.7× closer than full-key int8 quantization (0.030).
• Fastest measured int8 Sol-Attn at 8K / 16K / 65K tokens (2.56 / 8.64 / 108.95 ms at the time of measurement; 65K is 118.2 ms after v0.4.8) and ~parity at 32K, in a same-input cross-repo comparison.
• New BENCHMARKS.md (dated 2026-08-04): speed, peak-VRAM, and accuracy tables comparing this repo, kijai/ComfyUI-SolAttn_triton, KingGore/ComfyUI_sol-attn_Blackwell, SageAttention, and PyTorch SDPA — with fidelity caveats for hard-mask methods.
• Cross-validation: our bf16 path is bit-identical to kijai's independent implementation (0.000000) and SDPA-parity in all-exact mode.
• New "Why this repo" section in the README (EN + 中文).

Full changelog: v0.4.0...v0.4.8

0.4.0

ComfyUI-sol-attn v0.4.0

Highlights

• SM89 support (RTX 40-series) — pointer kernel twins for both the bf16 and int8_qk paths, dispatched automatically on pre-TMA GPUs. Bit-identical to the TMA kernels in forced-dispatch validation (bf16 and int8, including ragged lengths). Not yet run on physical SM89 hardware — reports welcome.
• Conditioning sink (sink_conditioning) on both MiniMax H3 Sol nodes — H3's packed text/conditioning/reference/audio KV blocks stay on the exact path (~3% cost), protecting prompt adherence and audio sync. exact_kv (default), exact_kv_and_rows (also dense conditioning query rows, ~20% cost), or off. The video span is captured automatically from the model's PackedLayout.
• Simpler, more robust scheduling — the scheduled node now reads transformer_options["sigmas"] directly instead of patching the model's forward; one less patch point, same ramp behavior.
• Fallback adoption — applied after a memory-efficient sage attention patch (e.g. KJNodes'), the H3 Sol nodes adopt the sage forward as their fallback: eligible steps run Sol-Attn, gated steps run mem-efficient sage. Order matters: sage patch first, Sol node after.

Also in this release

• Ragged sequence validation — verified bit-identical to contiguous input and SDPA-parity at 8,191 / 12,345 / 38,247 tokens (real H3 packed size).
• Trimmed autotune — 8→5 configs per kernel; same measured speed, lower JIT tax on new token counts.
• Pointer-based preprocess/quantization kernels (works on all supported arches).
• README updates (EN + 中文): new widgets, SM89 notes, corrected caveats, and ComfyUI-SolAttn_triton added to acknowledgements for the patterns adopted here.

Validation

• Full test suite passes on RTX 5090 (torch 2.10.0+cu130, Triton 3.6.0): strided≡contiguous, SDPA parity (rel L2 0.00097), int8_qk parity, sink-forced-exact ≡ dense, pointer twins ≡ TMA kernels, schedule/gate/injector unit checks, node registration.
• Benchmarks unchanged vs v0.3.0 within noise: 1.14–1.44× vs SageAttention at 8K–65K tokens (H3 width), plus 1.2–1.3× further with int8_qk at 16K+.

Full changelog: v0.3.0...v0.4.0