1.29.0.dev66375696

Automated nightly build from NVIDIA-internal CI. Not a supported release.
Only the 14 most recent are kept; older tags are deleted. Do not pin to this tag.

Version 1.29.0.dev66375696
Pipeline 66375696
GitLab 32d955d9c105 (develop)
GitHub a8565253bc23 (develop, the commit this tag points at)

pip install --pre --no-index \
  --find-links https://github.com/NVIDIA/cudnn-frontend/releases/expanded_assets/v1.29.0.dev66375696 \
  nvidia-cudnn-frontend

18 / 18 wheels.
Not built (no runner): win/arm64.

1.29.0.dev66254195

Automated nightly build from NVIDIA-internal CI. Not a supported release.
Only the 14 most recent are kept; older tags are deleted. Do not pin to this tag.

Version 1.29.0.dev66254195
Pipeline 66254195
GitLab 9e7783ea3c98 (develop)
GitHub edc7c2833327 (develop, the commit this tag points at)

pip install --pre --no-index \
  --find-links https://github.com/NVIDIA/cudnn-frontend/releases/expanded_assets/v1.29.0.dev66254195 \
  nvidia-cudnn-frontend

21 / 21 wheels.
Not built (no runner): win/arm64.

v1.28.0 release

cuDNN Frontend v1.28.0 Release Notes

cuDNN Frontend v1.28.0 is the recommended version for cuDNN 9.25.1 and later releases.

New: cudnn.fla — a drop-in accelerator for flash-linear-attention 🚀 🚀

cudnn.fla (#596) monkeypatches the flash-linear-attention ops that cuDNN can serve onto cuDNN's Blackwell (SM100) kernels, with a transparent fallback to FLA everywhere else, so results never change:

import cudnn.fla
cudnn.fla.accelerate_fla()   # before importing FLA layers/models
import fla                   # GatedDeltaNet / KDA now run on cuDNN where supported

• Gated DeltaNet (chunk_gated_delta_rule) — the GDN convention (log-space decay g, post-sigmoid beta, GVA where HV > H) mapped onto cuDNN's native op, reproducing the fused-layer knobs use_gate_in_kernel, use_beta_sigmoid_in_kernel, and use_qk_l2norm_in_kernel.
• KDA (Kimi Delta Attention, chunk_kda) — channel-wise gate plus scalar beta, l2norm forward and backward through cuDNN. BF16 only; FP16 declines and falls back.
• GatedMLP (#686) — an opt-in adapter (accelerate_fla(targets="gated_mlp")) backed by cudnn.gemm.ops.swiglu_mlp. The patch registry is target-selective, incremental, idempotent, and independently restorable via restore_fla(targets=...) / is_accelerated(target).

Configurations cuDNN cannot serve raise cudnnGraphNotSupportedError / NotImplementedError and fall back to FLA — never a wrong answer. Correctness is pinned by test/python/linear_attention/test_fla_compat.py, which requires cuDNN to match FLA within FLA's own BF16 noise on the output and every gradient.

Underneath, the linear-attention stack gained KDA and GDN-2 backward support (#556), a safe beta guard for GDN-2 (#722), packed-QKV views for native GDN (#685), state-layout and convention alignment with FLA/FlashInfer plus a context/IMA fix (#644), and successive CPU-overhead and instruction-cache/numerics passes on the FROST linear-attention kernels (#616, #708, #759). See docs/fe-oss-apis/fla.md.

New: JAX support across the CuTeDSL GEMM APIs 🚀 🚀

The GEMM CuTeDSL APIs are now type-erased (#529): every API under python/cudnn/gemm/cutedsl/ accepts JAX arrays alongside torch tensors, and the modules import and resolve their public symbols without torch installed — torch is imported only when torch tensors or dtypes are passed, and JAX only when JAX arrays are.

On top of that, cudnn.jax.call (#553) wraps CuTeDSL's native JAX integration (cutlass.jax.cutlass_call) and gives every JAX-reachable GEMM API a jax.jit entry point — gemm_amax, gemm_swiglu (including blockscaled MXFP8), gemm_srelu, gemm_dsrelu, gemm_proj_rope_mxfp8 (both BF16 and MXFP8 input paths), and the grouped and discrete-grouped families in their pointer-array modes. APIs without a JAX data path raise a clear error rather than failing obscurely. JAX outputs the kernel already writes are no longer zero-initialized first (#631).

New: First-class cudnn.Handle 🚀 🚀

cudnn.create_handle() now returns a Handle object that owns {backend handle, device, stream} instead of a bare int (#612). The per-handle state that had accreted as module-global side tables and per-engine device queries — the stream cache, and the three parallel device stacks used by the backend handle, pygraph, and FROST — unify behind Handle.stream and Handle.device. This matters because the Python engines (FROST, CuTeDSL, linear attention) need a device and a stream, not a cudnnHandle_t.

Backward compatibility is transparent for normal use: every handle-taking API (execute, set_stream, get_stream, destroy_handle, all graph methods) is Handle-aware, extracting the backend handle explicitly at each named handoff. Design notes and a full call-site inventory are in docs/handle_first_class_design.md.

New: GNN simple aggregation 🚀 🚀

from cudnn.gnn import CscGraph, agg_simple

cudnn.gnn.agg_simple (#647) exposes the cuDNN GNN AggSimple backend as a PyTorch custom operator with autograd, fake-tensor, and torch.compile support, handling graph validation and backend invocation so callers never touch the low-level GNN structures. Requires cuDNN 9.26 or newer and compute capability 8.0+; not supported on Windows. See docs/operations/gnn/agg_simple.md.

New: FROST SDPA on Ampere, Ada/Blackwell-consumer, and Rubin 🚀 🚀

The FROST engine family introduced in v1.27.0 now spans every architecture the frontend targets.

• SM80 (A100) — CuTe-DSL prefill (generic and d=256) and backward (generic and a d=64 fast path, roughly 2× on plain dense MHA) engines plus standalone cudnn.sdpa adapters (#493), later ported to plan-time compilation with TemplateParams kernels and sym_int THD extents (#689). Previously the manifest had no engine below SM100.
• SM120 — a forward per-tensor FP8 (E4M3) engine (#509) grown through several rounds of features and tuning (#531, #562, #595), MLA d_qk=192 / d_v=128 support (#507), and a backward engine (#486) extended to all head sizes ≤256 including d192/d256, non-compact layouts and deterministic dQ (#533), sliding-window attention (#505), GQA/MQA, padding masks, right-band widening and sink-token gradients (#557), deterministic 2-kernel mode and dBias (#707), and native service of declared strided layouts (#666).
• SM107 (Rubin) — per-tensor FP8 prefill via a sibling kernel (#576), with the has_lse specialization and a static SMEM guard (#579), a fused LDTM row-max and row-sum-in-MMA epilogue (#580), and the softmax_precision knob axis lit up with an F16x2 exponent on the d128 sibling (#651).
• MLA backward — sdpa_bwd gains MLA support (#643).

Head-dim envelopes and engine identity. Per-tensor FP8 now serves the dense head-dim ENVELOPE through the same TMA zero-padding path the F16/BF16 flavors use, and the engine table collapses to one engine per architecture × dtype family — head dims became a lowering concern (kernel-flavor selection) rather than an engine identity (#587).

Ragged / THD, with zero host reads. THD execute on SM100 (#606) and SM120 (#608) now performs no device-to-host reads at all — no .tolist() syncs, no host cumsum, no pageable H2D — building its metadata on device against a plan-time envelope grid, which makes the path CUDA-graph capturable (issue #552, with #543 binding host prep to the launch stream and plan-time-only THD compile keys). The FP8/MXFP8 SM100/SM107 engines were moved onto the same envelope design (#648) and the legacy pre-envelope THD leg removed (#622). Supporting work: native THD declared-stride support in the SM100/SM120 F16 forward kernels (#526), the cu_seq_len prefix-sum length form (#522), ragged stats on SM100 (#512) and SM120 (#508), and ragged S_kv tails served on the F16 rows via synthesized padding (#581).

Masking, splitting, and heuristics. Forward heuristics can now recommend the same engine several times under different complete knob assignments, which makes split-KV graph-reachable for the first time (#692); recommend() is a pure, backend-blind entry point that autotuners can call with hand-built graph facts. Split-KV also landed for the SM100/SM120 prefill kernels (#658), the KV split the heuristic chose now runs on the true cluster shape (#720), and pack_gqa is supported (#709). On masking, SM100 gained causal right-band widening with per-sequence THD bottom-right diagonals (#485), bottom-right diagonal plus sliding window (#584) — after which the bottom_right_with_swa notch was retired because every row serves BR + SWA (#623) — and full causal mask support (#498).

Other FROST SDPA work. SM100 MXFP8 for d192/d128 (#661); dense LSE written directly to non-contiguous, dense-compatible layouts (#712); an execute path made async where it can be, no longer re-deriving build-time facts (#570); FP8 scales folded in-kernel with a baked 2⁴ P-cast bias, removing Scale_S from below the graph (#619); the Amax_S output dropped from the FP8 kernels (#602); a has_lse specialization for the FP8/MXFP8 SM100 flavors (#574); and a strict LSE/sink/seq-lens execute contract with no torch.empty in execute (#484).

Removal: the legacy standalone SM100 d=256 forward and backward stacks, along with the cudnn::sdpa_{fwd,bwd}_d256 experimental torch ops; SM80 forward moved onto the same SdpaFwdDsl adapter path SM100/SM120 use, so one lowering function drives every forward cell (#682). d=256 remains available through the graph API on backend engines.

Updates to Graph API 🚀 🚀

SDPA

• max_total_seq_len_q / max_total_seq_len_kv on the forward node (#740). sdpa_backward has accepted these since cuDNN 9.6; the forward node never did, so a ragged graph could not express its packed token total and the FROST forward path had to infer a loose upper bound from the bound buffers' element span. A loose bound is memory-safe but not benign — masked rows are still multiplied, so an over-allocated, unwritten tail poisons whole tiles through 0 * NaN. Every framework already holds this number (q.shape[0] in vLLM, SGLang, TransformerEngine, Megatron-Core, PyTorch, FlashInfer); it can now be declared.
• Stats must be FP32 (#690). The SDPA kernels always compute and store logsumexp as FP32. A graph that declared Stats with a narrower dtype — explicitly, or implicitly by leaving it unset with a non-FP32 io_data_type — built and executed fine, and the kernel then wrote FP32 rows past the end of the caller's buffer, surfacing as silent corruption of adjacent allocations, illegal memory accesses, or driver launch failures. Stats is now set to FLOAT at creation, ...

v1.27.0 release

cuDNN Frontend v1.27.0 Release Notes

cuDNN Frontend v1.27.0 is the recommended version for cuDNN 9.24.0 and later releases.

New: Python-native cudnn.pygraph 🚀 🚀

cudnn.pygraph is now a Python-native graph class (#336). Graph structure — nodes, tensors, and parameters — lives in Python and is fully introspectable, while execution dispatches through pluggable backends: Python DSL engines and the cuDNN C++ backend.

• Graph IR — an engine-agnostic op DAG whose input/output port names match the C++ pybind kwarg names everywhere. 100% of the C++ op surface is covered (54 pointwise ops, 25 structured ops, the SDPA family, and matmul).
• Backend contract — engines.BaseEngine defines a propose_plans → build_plan → execute lifecycle, with each engine owning a stable engine_id in a reserved region.
• Router — create_execution_plans() produces a ranked plan list mixing Python plans with a single delegating entry for the cuDNN backend. Plan indices are two-level and stable, so select_plan() and the classic at-index APIs keep working.
• Classic-API compatibility is preserved: the same errors at validate(), the same conditional-output behavior, torch dtype acceptance, ragged (THD) offsets, and deserialize/build_plans passthrough.

See docs/python_graph_and_execution_backends.md for the full design.

Note: the internal pybind class pygraph was renamed to backend_graph (reachable only as cudnn._pybind_module.backend_graph). Nothing public imports the pybind name; the public cudnn.pygraph is now the Python class.

New: FROST engines 🚀 🚀

Open-source cuDNN engines written with CUTLASS primitives (#476):

• FROST engine implementations, kernel templates, and engine routing.
• Coverage for GEMM, grouped MoE matmul, fused epilogues, linear attention, and SDPA workloads, with FP4/FP8 and MXFP8 formats, variable-length sequences, recurrent states, masking, and quantized outputs.
• Unified plan discovery, ranking, selection, fallback, workspace handling, and execution reporting.
• Standalone GEMM benchmarks under benchmark/gemm/frost/ and Python tests. See python/cudnn/frost/README.md.

These engines are registered as Python engines and are selected through the new cudnn.pygraph router, so they are reachable from the graph API rather than only as standalone kernels — the linear-attention operations below are the first consumers of that path.

New: Linear attention — GDN 🚀 🚀

A new cudnn.linear_attention package (#476) provides gated linear-attention operations through the Graph API, as well as PyTorch custom operators, exported from cudnn.linear_attention.ops.

• Graph-API native. Each op is a thin adapter that executes cached single-node pygraphs (GDN/GDN_BWD).
• Two backends per operation — a FROST engine (default on SM100/SM103) with a cuTile engine as the fallback elsewhere.
• Registered through torch.library.custom_op, so they compose with autograd, torch.compile, and DDP.
• THD token-packed layout — [total_tokens, heads, dim] tensors plus cu_seqlens boundaries — with grouped-value attention (GVA/GQA) and per-sequence recurrent state ports (initial state in, final state out).
• Benchmarks live in benchmark/linear_attention/ (including a Dockerfile and benchmark_single_linear_attention.py), with tests under test/python/linear_attention/.

Updates to Graph API 🚀 🚀

SDPA

• Deterministic backward with linear workspace on Hopper (#398). On cuDNN 9.25+, use_deterministic_algorithm on sm90 can now route to an ordered-dQ engine whose workspace is linear — rather than quadratic — in sequence length. The faster dP-workspace path is still used when it fits the existing 256 MB limit (CUDNN_FRONTEND_ATTN_DP_WORKSPACE_LIMIT is honored). Long-sequence and THD/ragged deterministic training that previously failed to allocate now runs; results remain bitwise reproducible. No behavior change for cuDNN < 9.25 or other architectures.
• Mixed-form sequence lengths in SDPA forward (#430). The Q and KV sides may now independently choose per-batch (seq_len_*) or cumulative (cu_seq_len_*) representation. Paged-attention integrations that hold a cumulative Q prefix sum alongside per-batch KV lengths no longer need to materialize a KV-side prefix sum. Requires cuDNN 9.25.0 or later on the unified surface.
• cu_seq_len_q / cu_seq_len_kv are now exposed on the sdpa_fp8 Python binding (#366).
• FP32 I/O is now rejected up front in the unified SDPA node, with the allowlist dtype check applied consistently on both SDPA paths (#449, #454).

Data types and operations

• Added DOUBLE (F64) compute data type support for convolution and pointwise scaling attributes (#423).
• Causal conv1d Python bindings now precheck kernel sizes and raise range-specific ValueErrors for NCW (2–256), NWH (2–128), B2B projection (2–32), and B2B mixer (2–256) (#465, #472). This prevents an unsupported width-specialized NWH launch that could fault the CUDA context on SM90.
• The 9.22 reshape-mode attribute is now guarded at runtime rather than only at compile time, on both the graph API and the legacy OperationBuilder_v8 path (#466, #467).

Serialization and plan management

• Graph JSON v2 — repro extraction now emits UID-based graph JSON 2.0 (#280). Integer UIDs are the tensor-table identity used by node references, ragged-offset references, and tensor dumps, so anonymous tensors are no longer dropped and duplicate names no longer collapse. Missing UIDs are assigned before validation while preserving user-supplied UIDs; malformed versions, duplicate identities, and dangling references are rejected with typed errors.
• Ragged-offset tensor descriptors are now serialized in full — dimensions, strides, and data type — with nested offsets followed recursively and shared descriptors deduplicated (#412).
• Graph::serialize accepts a serialize_structure flag (default true), making it symmetric with the handle-based deserialize path and enabling plan-only round trips after deserialize(handle, ...) (#371).
• Documented the behavior of serializing a selected plan (#364).

Build and integration

• No-RTTI builds are now supported: dynamic_cast has been removed from the public headers (#477).
• Added a PEP 735 dev dependency group, so pip install --group dev works as a prerequisite to deprecating requirements.txt (#359).

Open-Source Kernels 🚀 🚀

SDPA

• Native SM100 d192/d128 SDPA forward kernel for DSv3 MLA (#488). Instead of selecting a padded d=256 flavor, the D_QK = 192 / D_V = 128 logical shape now has a dedicated Blackwell prefill kernel (BF16 and FP16, dense CGA2 classic pipeline). For top-left causal S=8192 it reaches 87% compute SOL / 820 useful TFLOPS — roughly 1.5× a padded d=256 proxy. The existing d128, d256, and d512 kernels are unchanged.
• The OSS SDPA prefill SM100 engine is now restricted to sm_100, with samples skipped elsewhere (#473).

GEMM fusions

• GEMM + RoPE + MXFP8 fusion (#367) — an experimental SM100+ kernel fusing the projection GEMM, per-head RoPE, and dual-direction MXFP8 quantization, intended for DeepSeek-V3 forward training. Returns row-wise and column-wise FP8 outputs with E8M0 scale tensors. A second variant specialized for MXFP8 inputs was added, and the original kernel renamed to the _bf16in flavor, so recipes that project in either BF16 or MXFP8 are covered (#438).
• BF16 grouped GEMM MoE kernels (#415) — BF16 grouped GEMM, GLU, dGLU, and WGrad APIs for both dense and discrete MoE layouts, routed through the shared public APIs alongside the block-scaled implementations.
• Rubin (SM107) support — new block-scaled grouped GEMM GLU/dGLU/quant kernels with architecture-aware dispatch and cache keys (#447), a Rubin WGRAD kernel (#456), FP8 K-tiling and output-staging tuning (#457), and compatibility across Blackwell CUTLASS wheels (#464). Options the Rubin kernels do not support raise NotImplementedError rather than silently producing invalid results.
• Grouped GEMM quantization no longer requires a probability tensor (#458), and single-group grouped GEMM metadata handling was optimized.
• The GEMM fusion sources were reorganized for clarity (#459).
• MXFP8 performance improvements (#481).

DSA (DeepSeek Sparse Attention)

• FP8/MXFP8 and compressed Top-K indexer paths (#370). Adds an SM90 FP8 indexer path (E4M3 Q/K with per-token/head FP32 descales) and SM100 MXFP8 indexer and dense score-recompute paths (E4M3 Q/K with packed E8M0 block scales), for BSHD and THD inputs including compact padded MXFP8 scale layouts. The new SM100-only indexer_forward_top_k_wrapper produces Top-K indices, selected logits, optional fused softmax, and optional LSE without materializing the dense score tensor; deterministic=True resolves K-th-boundary ties toward the smallest local KV indices. Existing BF16 paths are preserved.
• Added SM90 DSA qh16 indexer forward support (#388).
• indexer_forward_wrapper accepts an optional pre-allocated out tensor, avoiding repeated internal allocation in iterative calls (#470).

CSA (Compressor)

• Fused Compressor forward and backward CuTe-DSL kernels ported from Megatron-LM (#427). The gated-softmax pooling region between the two projection GEMMs and the RMSNorm — gather-index build, gather, + APE, overlap-window transform, fp32 windowed softmax, gated weighted sum — collapses from roughly 39 forward and 51 backward kernel launches per call into one forward and one backward kernel. Two follow-on optimizations (32-bit vectorized forward access; kernel-side zero-writes in the backwar...

v1.26.0 release

cuDNN Frontend v1.26.0 Release Notes

cuDNN Frontend v1.26.0 is the recommended version for cuDNN 9.24.0 and later releases.

Updates to Graph API 🚀 🚀

SDPA

• Added unified-engine FP8 and MXFP8 forward SDPA support (#301). Requires cuDNN 9.25.0 or later (#339).
• SDPA backward with head dimension d=256 now uses the cuDNN-native path on cuDNN 9.23+, bypassing the OSS kernel path (#335).

Data types

• Added a new BYTE_BOOLEAN frontend data type (#302). Boolean tensors automatically map to BYTE_BOOLEAN when running against cuDNN 9.25 and later (#339).

Serialization and plan management

• Graph::deserialize now accepts an enforce_precompiled option to require precompiled engine plans during deserialization (#323).
• Added a run_warmup opt-out and a reuse-parsed-json overload to Graph::deserialize, reducing repeated parsing overhead (#329).

Open-Source Kernels (CuTe DSL) 🚀 🚀

Block-sparse attention - Video Sparse Attention

• Added block-sparse attention CuTe DSL kernels for Hopper and Blackwell (#333).

DSA Deepseek Sparse Attention

• Added q causal offsets and SM100F support (#316).
• Optimized the DSA backward SM100 kernel (#318).
• Aligned DSA indexer kernels and fixed dense score-gradient clipping (#297).

Grouped GEMM

• grouped_gemm_quant_wrapper_sm100 now accepts an optional caller-provided output tensor (#338).
• Added SReLU support in the grouped GEMM Hadamard fusion (#315).
• CuTe DSL 4.5 compatibility: migrated cute.core.ThrMma and cute.make_fragment usage (#321) and switched the dGLU dbias reduction to a constexpr loop to fix a DSL 4.5 regression (#322).

Benchmarks, Samples and Documentation ✨✨

• SDPA benchmark now computes SOL% using the sampled SM clock and per-architecture MMA throughput (#314), with refreshed benchmarking artifacts for cuDNN 9.24.0.27 (#306).
• Added a CuTe DSL fusion-kernel benchmark suite with initial B200 and B300 results (#303).
• Test and sample robustness improvements from fuzzer mining across cuDNN 9.18–9.24, including block-scale and SDPA fixes (#330).
• Updated the conv get-plan sample heuristic configuration count (#278).

Bug Fixes 🐛

• Fixed a grid-dimension overflow in the DSA backward convert kernel on SM100 (#331).
• Fixed an illegal memory access in indexer_topk_wrapper (#312).
• Fixed SM100 sparse score recompute compact top-k code generation (#317).
• Fixed the reduce_dKV validity guard incorrectly comparing the top-k column position (#298).
• Fixed the scale-factor sort order in block_scale_quantize.h (#319).
• Fixed a synchronization issue in MXFP8 tests (#325).
• Fixed SDPA attribute handling in the flash attention node (#326).
• Fixed an unused ragged-offset version-error variable (#299).

Acknowledgements 🙏

Thanks to everyone who contributed to this release:

@dimitar-asenov, @HollowMan6, @Hyaloid, @jiayus-nvidia, @Jie-Fang, @jiemingz, @NVIDIA-JerryChen, @phu0ngng, @shraiysh, @sraman-rgb, @szluyu99, @take-cheeze, @vincejhan, @Vinnie6167, and @zianglih.

v1.25.0 release

cuDNN Frontend v1.25.0 Release Notes

cuDNN has moved completely to github for development. Please direct your PRs to develop and file issues in github.

cuDNN Frontend v1.25.0 is the recommended version for cuDNN 9.23.0 and later releases.

Updates to Graph API 🚀 🚀

SDPA

• cu_seqlens in unified SDPA — the unified SDPA path now accepts cumulative sequence-length tensors, enabling variable-length (packed) batches without padding.
• Ragged offset multiplier — added frontend support for the per-tensor ragged offset multiplier (CUDNN_ATTR_TENSOR_RAGGED_OFFSET_MULTIPLIER), letting ragged offsets be stored in coarser units and scaled back to element offsets by the engine. Exposed through Tensor_attributes (getters/setters, validation, serialization) and the Python tensor() bindings. Requires cuDNN 9.24.0.

Structured plan pinning

• Added get_engine_and_knobs_at_index, which returns the structured (engine_id, {KnobType_t: value}) for a plan instead of a stringified tag, so a tuned plan can be persisted and replayed exactly via create_execution_plan(engine_id, knobs) even as plan enumeration drifts across versions. Available in C++ (Graph, Execution_plan_list) and Python.
• Extended KnobType_t with SWAP_AB, INPUT_TMA_ENABLE, and OUTPUT_TMA_ENABLE.

Reduction

• Added optional group_offset support to the reduction node (Reduction_attributes::set_group_offset), so cuDNN FE can express per-expert reductions for MoE grouped GEMM workloads. Wires CUDNN_ATTR_OPERATION_REDUCTION_GROUP_OFFSET_DESC with runtime version checks (cuDNN ≥ 9.24.0), and exposes the optional argument through the Python reduction binding.

Open-Source Kernels 🚀 🚀

• Row-scale grouped GEMM quantization — added row-scale support to the grouped GEMM quant path.
• DSA — fixed CuTe-DSL guards and added the SM90 indexer-forward kernel.
• dgeglu — config values are now compile-time constants instead of runtime values.

General Improvements ✨✨

• Static linking of libcudnn is now supported.
• libcudart loading — the selected libcudart can be overridden via CUDNN_FRONTEND_CUDART_LIB_NAME, and the shim now warns instead of throwing when multiple libcudart libraries are found, improving robustness in containerized environments.
• Windows / MSVC — consolidated getenv access and fixed C4996/C4005 compiler warnings on MSVC.

Bug Fixes 🐛

• Fixed variant-pack-template lifecycle bugs and added defensive null checks.
• Deserialize-owned containers are now cleared on re-deserialize to prevent stale state.
• Use a static signature for sfd_col_d_srelu_tensor.

Samples

• Skip TensorIR MemBound / compile-time-const samples on consumer Blackwell (SM12x).
• Skip the flexible-graph SDPA backward sample on SM120 and above.

Benchmarking 📊

• Added an autoregressive video DiT SDPA configuration with GB200 / GB300 results.
• Updated the SDPA benchmarking artifacts and removed stale H200 artifacts.

Acknowledgements

External contributors

• Thanks @take-cheeze for adding support for static linking of libcudnn.
• Thanks Ziang Li for adding row-scale support to the grouped GEMM quant path.
• Thanks Jiayu Sun — DSA CuTe-DSL guard fixes and the SM90 indexer-forward kernel.

v1.24.1 release

cuDNN Frontend v1.24.1 Release Notes

cuDNN Frontend v1.24.1 is the recommended version for cuDNN 9.23.0 and later releases.

General Improvements 🚀 🚀

Updates to Graph API

• Rotary Position Embedding (RoPE) is now available as a cudnn operation, usable both standalone and as a preprocessing stage for the SDPA engine. See the sample for usage. RoPE fusion with SDPA requires cuDNN 9.24.0.
• SDPA backward now supports hidden dimension d=256. Requires cuDNN 9.23.0 or later.

Open-Source Kernels 🚀 🚀

• Introduced a DSA module featuring the following DSA/CSA kernels for DsV4:
  • Indexer Forward: CuTe-DSL score kernel (Q @ Kᵗ, ReLU, head reduce, ratio causal mask). Non-fused; pair with Indexer Top-K for the top-K stage.
  • Indexer Top-K: SM100 CuTe-DSL radix top-K kernel with per-row seq_lens.
  • Sparse Attention Backward: DSA backward (FlashMLA-shape, SM90/SM100).
  • Sparse Indexer / Attention Score Recompute: Sparse (top-K) recomputation of indexer and attention scores for training loss.
  • Dense Indexer / Attention Score Recompute: Dense (full-KV) analogues of the above.
  • Indexer Backward: Three-stage pipeline (score-grad, three GEMMs, dtype cast) for sparse top-K score tensors.
  • Dense Indexer Backward: Full-KV counterpart of Indexer Backward.
• Grouped GEMM GLU forward kernel with fused Hadamard transform.

Skills

• Added a new Claude skill for converting cuteDSL kernels into experimental cuDNN APIs.

Enhancements

• Noisy logging messages are now emitted only once per process.
• Convolution problems are now rejected when total filter size exceeds INT32_MAX.
• Support for ragged input order has been added for grouped GEMM weight gradients.

Bug Fixes

• Fixed an issue in the reshape operator when called with 1D tensors.
• Fixed missing square_alpha scaling in dgeglu and dswiglu.
• Fixed a race condition in lazy variant-pack-template preparation observed in some single-threaded scenarios.

New Samples

• Added new samples for memory-bound fusions.

Acknowledgements

The Native Sparse Attention forward-prop kernels, supporting head dim = 128 and optimized for the Blackwell architecture, were implemented in CuteDSL.

These kernels were a collaborative effort, jointly developed by: Jie Feng, Akash Mehra, Vincent Zhang, Dominik Ernst, Xinbo Zhao, Aditya Vavre, Vedaanta Agarwalla, Mingyang Wang, Anerudhan Gopal, Paul Springer, Yang Xu, and Nima Tajbakhsh.

v1.24.0 release

cuDNN Frontend v1.24.0 Release Notes

cuDNN Frontend v1.24.0 is the recommended version for cuDNN 9.22.0 and later releases.

General Improvements 🚀 🚀

Updates to Graph API

• Rotary Position Embedding (RoPE) is now available as a cudnn operation, usable both standalone and as a preprocessing stage for the SDPA engine. See the sample for usage. RoPE fusion with SDPA requires cuDNN 9.24.0.
• SDPA backward now supports hidden dimension d=256. Requires cuDNN 9.23.0 or later.

Open-Source Kernels 🚀 🚀

• Introduced a DSA module featuring the following DSA/CSA kernels for DsV4:
  • Indexer Forward: CuTe-DSL score kernel (Q @ Kᵗ, ReLU, head reduce, ratio causal mask). Non-fused; pair with Indexer Top-K for the top-K stage.
  • Indexer Top-K: SM100 CuTe-DSL radix top-K kernel with per-row seq_lens.
  • Sparse Attention Backward: DSA backward (FlashMLA-shape, SM90/SM100).
  • Sparse Indexer / Attention Score Recompute: Sparse (top-K) recomputation of indexer and attention scores for training loss.
  • Dense Indexer / Attention Score Recompute: Dense (full-KV) analogues of the above.
  • Indexer Backward: Three-stage pipeline (score-grad, three GEMMs, dtype cast) for sparse top-K score tensors.
  • Dense Indexer Backward: Full-KV counterpart of Indexer Backward.
• Grouped GEMM GLU forward kernel with fused Hadamard transform.

Skills

• Added a new Claude skill for converting cuteDSL kernels into experimental cuDNN APIs.

Enhancements

• Noisy logging messages are now emitted only once per process.
• Convolution problems are now rejected when total filter size exceeds INT32_MAX.
• Support for ragged input order has been added for grouped GEMM weight gradients.

Bug Fixes

• Fixed an issue in the reshape operator when called with 1D tensors.
• Fixed missing square_alpha scaling in dgeglu and dswiglu.
• Fixed a race condition in lazy variant-pack-template preparation observed in some single-threaded scenarios.

New Samples

• Added new samples for memory-bound fusions.

Acknowledgements

The Native Sparse Attention forward-prop kernels, supporting head dim = 128 and optimized for the Blackwell architecture, were implemented in CuteDSL.

These kernels were a collaborative effort, jointly developed by: Jie Feng, Akash Mehra, Vincent Zhang, Dominik Ernst, Xinbo Zhao, Aditya Vavre, Vedaanta Agarwalla, Mingyang Wang, Anerudhan Gopal, Paul Springer, Yang Xu, and Nima Tajbakhsh.

v1.23.0-release

cuDNN Frontend v1.23.0 is the recommended version for cuDNN 9.21.0 and later releases.

cudnn-frontend now has pip wheels for python 3.14t.

New APIs 🚀 🚀

Causal Conv1d

• Depthwise causal 1-D convolution with optional fused silu activation (requires cuDNN 9.22.0): y = activation(conv1d_causal(x, w) + b) Supports forward and backward passes with torch.autograd and torch.compile. (Not supported on Windows yet)

Updates to Graph API

Transpose (requires cuDNN 9.22.0)

• Added new Graph::transpose with Transpose_attributes(permutation, optional compute dtype, name)

Slice (requires cuDNN 9.22.0)

• Extend Slice_attributes with set_strides for per-axis slice steps; strided slices update inferred output shape and strides accordingly.
• Python: pygraph.slice now honors each dimension's slice.step

Concatenate (requires cuDNN 9.22.0)

• Extend Concatenate_attributes with set_in_place_index (optional). When unset, concatenate runs out-of-place per backend rules.

Reshape (requires cuDNN 9.22.0)

• Introduce ReshapeMode_t(VIEW_ONLY,LOGICAL) and Reshape_attributes::set_reshape_mode so reshapes can select view-style vs lexicographic logical reshape.

Compile-time constants (requires cuDNN 9.22.0)

• Added cudnn.scalar_type(RUNTIME_PARAM,COMPILE_TIME_CONST) and Graph::tensor(scalar, ScalarType) overloads, so scalars can be execution-time variant-pack inputs or constants embedded in the plan.
• Tensor_attributes can be marked as a compile-time constant or a normal runtimepass-by-value scalar;

Open source kernels 🚀 🚀

• GEMM + sReLU: High-performance implementation of squared-ReLU fused with GEMM.
• GEMM + dsReLU: High-performance implementation of dsquared-ReLU fused with GEMM.
• Grouped GEMM + GLU + Hadamard: Dense grouped GEMM GLU forward fusion with a fused Hadamard transform and per-expert AMAX reduction.
• Grouped GEMM + sReLU: Contiguous grouped squared-ReLU GEMM for MoE workloads.
• Grouped GEMM + dsReLU: Contiguous and discrete grouped dsquared-ReLU GEMM for MoE workloads.
• RMSNorm + RHT + amax: A fused CUTE DSL kernel for NVIDIA Blackwell GPUs (SM100+) that applies RMS normalization, a block-diagonal Hadamard transform with fixed block size 16, and a per-CTA amax reduction.

Fix block-scale quantize
The scale tensor uses a 128x4 reordered layout (TensorReordering_t::F8_128x4). When the reordering type is set on the scale tensor, the frontend will automatically pad the inferred scale dimensions to align with the 128x4 block structure (non-batch, non-axis dimensions are padded to multiples of 128, and the quantize axis dimension is padded to multiples of 4).

• GEMM + sReLU: High-performance implementation of squared-ReLU fused with GEMM.
• GEMM + dsReLU: High-performance implementation of dsquared-ReLU fused with GEMM.
• Grouped GEMM + GLU + Hadamard: Dense grouped GEMM GLU forward fusion with a fused Hadamard transform and per-expert AMAX reduction.
• Grouped GEMM + sReLU: Contiguous grouped squared-ReLU GEMM for MoE workloads.
• Grouped GEMM + dsReLU: Contiguous and discrete grouped dsquared-ReLU GEMM for MoE workloads.
• RMSNorm + RHT + amax: A fused CUTE DSL kernel for NVIDIA Blackwell GPUs (SM100+) that applies RMS normalization, a block-diagonal Hadamard transform with fixed block size 16, and a per-CTA amax reduction.

General Improvements ✨✨

• Grouped GEMM APIs now default to dynamic MNKL compilation across GLU, dGLU, SwiGLU, dSwiGLU, SReLU, dSReLU, and quant wrappers. Set CUDNN_FE_GROUPED_GEMM_DYNAMIC_MNKL=0 to restore the previous M-only dynamic behavior.

• Grouped GEMM wgrad wrapper APIs now support caller-provided output buffers (wgrad_tensor for dense, wgrad_ptrs for discrete)

• Unused internal c_tensor removed from Grouped GEMM quant path

Bug fix 🐛

• Grouped GEMM GLU bias compilation issue for 64B-aligned inputs with dynamic MNKL

• Fix an issue with dropout in Blackwell when cudnn frontend 1.21 version is used with cudnn backend 9.21 and 9.22.

Benchmarking 📊

• Updated the benchmark results for the SDPA improvements. Added Kimi-K2.6, LTX-2, Qwen 2.5 , Wan2.2 to the benchmark results page.

Acknowledgements:

• Thanks @haowen-han for fixing a bug in the block-scale matmul sample.

v1.22.1-release

cuDNN Frontend v1.22.1 is the recommended version for cuDNN 9.20.0 and later releases.

General Improvements 🚀 🚀

• Introducing PyTorch custom operator wrapping cuDNN's MoE Grouped Gemm operation.

    ```python
        def moe_grouped_matmul(
            token: torch.Tensor,
            weight: torch.Tensor,
            first_token_offset: torch.Tensor,
            token_index: Optional[torch.Tensor] = None,
            token_ks: Optional[torch.Tensor] = None,
            mode: str = "none",
            top_k: int = 1,
        ) -> torch.Tensor
    ```
  

  See test/python/test_moe_grouped_matmul_op.py for usage.

• 🕒 We will be rolling out new native custom torch ops in upcoming releases – stay tuned! 😃

Open-Source Kernels 🚀 🚀

• Blackwell sdpa fprop kernel supporting head dim = 256, written in cuteDSL. Support added through the torch-op above or callable as a standalone API. See samples for the API usage. Requires nvidia-cutlass-dsl[cu13]==4.4.1

Updates:

• GroupedGemmWgradSm100 and grouped_gemm_wgrad_wrapper_sm100 expose the grouped GEMM weight-gradient kernel. See grouped_gemm_wgrad.html for API reference moe_blockscaled_grouped_gemm_wgrad.py for samples.

Acknowledgements:

Blackwell sdpa fprop kernel supporting head dim = 256, written in cuteDSL kernel was jointly developed by Shengbin Di, Yuxi Chi, and Linfeng Zheng in close collaboration with Alibaba. We would like to extend special thanks to the core contributors from Alibaba: Siyu Wang, Haoyan Huang, Lanbo Li, Yun Zhong, Man Yuan, Minmin Sun, Yong Li, and Wei Lin for their significant contributions to this work.