Skip to content

MagiAttention V1.1.8

Latest

Choose a tag to compare

@Strivin0311 Strivin0311 released this 07 Nov 06:55
· 3 commits to main since this release
4fe8114

MagiAttention V1.1.8

This release adds index-sparse DSA kernels for Blackwell, learnable attention sinks in the CuTeDSL backend, and overlap-aware range scheduling. It also fixes Top-k compatibility and concurrency issues, removes backward range-contract device-to-host reads, and improves sink preprocessing and sparse-attention execution.

New Features

  1. Index-Sparse DSA Kernels (#388)

    • Add native SM100 forward/backward and SM120 forward entry points using per-query-block K-token indices.
    • Add packed-layout Triton DSA attention, a grouped-layout Triton backward alternative, and CuTeDSL radix-select Top-k helpers with deterministic index ordering and tie-breaking.
    • Support LSE-output gradients in native SM100 index-sparse backward and packed Triton autograd.
  2. Learnable Attention Sinks (#367)

    • Support multiple learnable sink logits and sink gradients in the CuTeDSL backend, including dense and range attention.
    • Integrate sink handling into distributed attention and FP32 partial-gradient accumulation.
  3. Communication-Overlap Scheduling (#374)

    • Add sm_margin and dynamic range scheduling on SM100/SM110 to reserve execution resources for concurrent communication.
    • Enable Q/O communication with CuTeDSL and provide guidance for NCCL CTA-cluster configuration.

Enhancements and Performance

  • Bypass redundant logsumexp for a single attention sink while retaining FP32 conversion and contiguous preprocessing; preserve the existing multi-sink reduction (#388).
  • Add asynchronous V gathers and L2-sized K passes to SM100 index-sparse backward, with FP32 dQ accumulation across passes (#388).
  • Treat packed Triton KV length as a runtime argument to avoid specializing solely on SKV (#388).

Bug Fixes

  • Fix CuTeDSL Top-k compilation with DSL 4.4.2 by passing raw LLVM pointers to memory operations and using compatible vector extraction (#388).
  • Isolate Top-k synchronization workspaces by CUDA stream and launch through the current Torch stream, preventing ordinary multi-stream calls from sharing mutable synchronization state (#388).
  • Return zero output and negative-infinity LSE for empty packed-Triton sparse rows, with zero query gradients, instead of dividing by zero (#388).
  • Remove first-backward device-to-host range reads by deriving range-contract flags from CPU metadata. Preserve the actual reordered-range semantics, including Q sortedness, Q union coverage, and K partition checks (#388).
  • Correct persistent-backward tile handoffs and FP32 forward shared-memory reuse on SM100/SM110 (#374).
  • Correct sink handling for empty-key and skipped distributed stages, including asymmetric K/V head dimensions (#367).

Dependency Updates

  • Extend native CuTeDSL support through DSL 4.8.1. Supported pairs remain explicitly validated (#388):

    nvidia-cutlass-dsl quack-kernels
    4.4.2 0.4.1
    4.6.2 0.6.4
    4.7.0–4.8.1 0.6.5

    DSL 4.6.0 and 4.6.1 remain excluded; libs-base must match the DSL version. Independent pip ranges do not imply support for arbitrary DSL/Quack combinations.

  • FA4 retains its separate DSL >=4.4.2,<4.6.0 requirement and tested DSL 4.4.2 / Quack 0.4.1 pair.

CI and Build

  • Pin the lint runner to Ubuntu 24.04 for a stable CI environment (#388).

Compatibility Notes

  • Native index-sparse kernels are low-level entry points with FP16/BF16 and head-dimension-128 constraints; they do not automatically replace public attention dispatch. Native forward requires q_block_size * (num_heads_q / num_heads_kv) == 128. SM120 index-sparse backward is not included.
  • Packed Triton attention returns log2 LSE; native index-sparse attention uses natural-log LSE. Their layouts and index contracts differ and require explicit adaptation when combining backends.
  • sm_margin applies to supported SM100/SM110 range paths; forward head dimension 192 does not honor the reservation. For NCCL communication overlap, configure NCCL_CGA_CLUSTER_SIZE=1 before creating process groups.
  • Removing backward range-contract reads does not guarantee that every metadata-construction or backend path is globally free of device-to-host transfers.

Full Changelog

v1.1.7...v1.1.8