v1.27.0 release

cuDNN Frontend v1.27.0 Release Notes

cuDNN Frontend v1.27.0 is the recommended version for cuDNN 9.24.0 and later releases.

New: Python-native cudnn.pygraph 🚀 🚀

cudnn.pygraph is now a Python-native graph class (#336). Graph structure — nodes, tensors, and parameters — lives in Python and is fully introspectable, while execution dispatches through pluggable backends: Python DSL engines and the cuDNN C++ backend.

• Graph IR — an engine-agnostic op DAG whose input/output port names match the C++ pybind kwarg names everywhere. 100% of the C++ op surface is covered (54 pointwise ops, 25 structured ops, the SDPA family, and matmul).
• Backend contract — engines.BaseEngine defines a propose_plans → build_plan → execute lifecycle, with each engine owning a stable engine_id in a reserved region.
• Router — create_execution_plans() produces a ranked plan list mixing Python plans with a single delegating entry for the cuDNN backend. Plan indices are two-level and stable, so select_plan() and the classic at-index APIs keep working.
• Classic-API compatibility is preserved: the same errors at validate(), the same conditional-output behavior, torch dtype acceptance, ragged (THD) offsets, and deserialize/build_plans passthrough.

See docs/python_graph_and_execution_backends.md for the full design.

Note: the internal pybind class pygraph was renamed to backend_graph (reachable only as cudnn._pybind_module.backend_graph). Nothing public imports the pybind name; the public cudnn.pygraph is now the Python class.

New: FROST engines 🚀 🚀

Open-source cuDNN engines written with CUTLASS primitives (#476):

• FROST engine implementations, kernel templates, and engine routing.
• Coverage for GEMM, grouped MoE matmul, fused epilogues, linear attention, and SDPA workloads, with FP4/FP8 and MXFP8 formats, variable-length sequences, recurrent states, masking, and quantized outputs.
• Unified plan discovery, ranking, selection, fallback, workspace handling, and execution reporting.
• Standalone GEMM benchmarks under benchmark/gemm/frost/ and Python tests. See python/cudnn/frost/README.md.

These engines are registered as Python engines and are selected through the new cudnn.pygraph router, so they are reachable from the graph API rather than only as standalone kernels — the linear-attention operations below are the first consumers of that path.

New: Linear attention — GDN 🚀 🚀

A new cudnn.linear_attention package (#476) provides gated linear-attention operations through the Graph API, as well as PyTorch custom operators, exported from cudnn.linear_attention.ops.

• Graph-API native. Each op is a thin adapter that executes cached single-node pygraphs (GDN/GDN_BWD).
• Two backends per operation — a FROST engine (default on SM100/SM103) with a cuTile engine as the fallback elsewhere.
• Registered through torch.library.custom_op, so they compose with autograd, torch.compile, and DDP.
• THD token-packed layout — [total_tokens, heads, dim] tensors plus cu_seqlens boundaries — with grouped-value attention (GVA/GQA) and per-sequence recurrent state ports (initial state in, final state out).
• Benchmarks live in benchmark/linear_attention/ (including a Dockerfile and benchmark_single_linear_attention.py), with tests under test/python/linear_attention/.

Updates to Graph API 🚀 🚀

SDPA

• Deterministic backward with linear workspace on Hopper (#398). On cuDNN 9.25+, use_deterministic_algorithm on sm90 can now route to an ordered-dQ engine whose workspace is linear — rather than quadratic — in sequence length. The faster dP-workspace path is still used when it fits the existing 256 MB limit (CUDNN_FRONTEND_ATTN_DP_WORKSPACE_LIMIT is honored). Long-sequence and THD/ragged deterministic training that previously failed to allocate now runs; results remain bitwise reproducible. No behavior change for cuDNN < 9.25 or other architectures.
• Mixed-form sequence lengths in SDPA forward (#430). The Q and KV sides may now independently choose per-batch (seq_len_*) or cumulative (cu_seq_len_*) representation. Paged-attention integrations that hold a cumulative Q prefix sum alongside per-batch KV lengths no longer need to materialize a KV-side prefix sum. Requires cuDNN 9.25.0 or later on the unified surface.
• cu_seq_len_q / cu_seq_len_kv are now exposed on the sdpa_fp8 Python binding (#366).
• FP32 I/O is now rejected up front in the unified SDPA node, with the allowlist dtype check applied consistently on both SDPA paths (#449, #454).

Data types and operations

• Added DOUBLE (F64) compute data type support for convolution and pointwise scaling attributes (#423).
• Causal conv1d Python bindings now precheck kernel sizes and raise range-specific ValueErrors for NCW (2–256), NWH (2–128), B2B projection (2–32), and B2B mixer (2–256) (#465, #472). This prevents an unsupported width-specialized NWH launch that could fault the CUDA context on SM90.
• The 9.22 reshape-mode attribute is now guarded at runtime rather than only at compile time, on both the graph API and the legacy OperationBuilder_v8 path (#466, #467).

Serialization and plan management

• Graph JSON v2 — repro extraction now emits UID-based graph JSON 2.0 (#280). Integer UIDs are the tensor-table identity used by node references, ragged-offset references, and tensor dumps, so anonymous tensors are no longer dropped and duplicate names no longer collapse. Missing UIDs are assigned before validation while preserving user-supplied UIDs; malformed versions, duplicate identities, and dangling references are rejected with typed errors.
• Ragged-offset tensor descriptors are now serialized in full — dimensions, strides, and data type — with nested offsets followed recursively and shared descriptors deduplicated (#412).
• Graph::serialize accepts a serialize_structure flag (default true), making it symmetric with the handle-based deserialize path and enabling plan-only round trips after deserialize(handle, ...) (#371).
• Documented the behavior of serializing a selected plan (#364).

Build and integration

• No-RTTI builds are now supported: dynamic_cast has been removed from the public headers (#477).
• Added a PEP 735 dev dependency group, so pip install --group dev works as a prerequisite to deprecating requirements.txt (#359).

Open-Source Kernels 🚀 🚀

SDPA

• Native SM100 d192/d128 SDPA forward kernel for DSv3 MLA (#488). Instead of selecting a padded d=256 flavor, the D_QK = 192 / D_V = 128 logical shape now has a dedicated Blackwell prefill kernel (BF16 and FP16, dense CGA2 classic pipeline). For top-left causal S=8192 it reaches 87% compute SOL / 820 useful TFLOPS — roughly 1.5× a padded d=256 proxy. The existing d128, d256, and d512 kernels are unchanged.
• The OSS SDPA prefill SM100 engine is now restricted to sm_100, with samples skipped elsewhere (#473).

GEMM fusions

• GEMM + RoPE + MXFP8 fusion (#367) — an experimental SM100+ kernel fusing the projection GEMM, per-head RoPE, and dual-direction MXFP8 quantization, intended for DeepSeek-V3 forward training. Returns row-wise and column-wise FP8 outputs with E8M0 scale tensors. A second variant specialized for MXFP8 inputs was added, and the original kernel renamed to the _bf16in flavor, so recipes that project in either BF16 or MXFP8 are covered (#438).
• BF16 grouped GEMM MoE kernels (#415) — BF16 grouped GEMM, GLU, dGLU, and WGrad APIs for both dense and discrete MoE layouts, routed through the shared public APIs alongside the block-scaled implementations.
• Rubin (SM107) support — new block-scaled grouped GEMM GLU/dGLU/quant kernels with architecture-aware dispatch and cache keys (#447), a Rubin WGRAD kernel (#456), FP8 K-tiling and output-staging tuning (#457), and compatibility across Blackwell CUTLASS wheels (#464). Options the Rubin kernels do not support raise NotImplementedError rather than silently producing invalid results.
• Grouped GEMM quantization no longer requires a probability tensor (#458), and single-group grouped GEMM metadata handling was optimized.
• The GEMM fusion sources were reorganized for clarity (#459).
• MXFP8 performance improvements (#481).

DSA (DeepSeek Sparse Attention)

• FP8/MXFP8 and compressed Top-K indexer paths (#370). Adds an SM90 FP8 indexer path (E4M3 Q/K with per-token/head FP32 descales) and SM100 MXFP8 indexer and dense score-recompute paths (E4M3 Q/K with packed E8M0 block scales), for BSHD and THD inputs including compact padded MXFP8 scale layouts. The new SM100-only indexer_forward_top_k_wrapper produces Top-K indices, selected logits, optional fused softmax, and optional LSE without materializing the dense score tensor; deterministic=True resolves K-th-boundary ties toward the smallest local KV indices. Existing BF16 paths are preserved.
• Added SM90 DSA qh16 indexer forward support (#388).
• indexer_forward_wrapper accepts an optional pre-allocated out tensor, avoiding repeated internal allocation in iterative calls (#470).

CSA (Compressor)

• Fused Compressor forward and backward CuTe-DSL kernels ported from Megatron-LM (#427). The gated-softmax pooling region between the two projection GEMMs and the RMSNorm — gather-index build, gather, + APE, overlap-window transform, fp32 windowed softmax, gated weighted sum — collapses from roughly 39 forward and 51 backward kernel launches per call into one forward and one backward kernel. Two follow-on optimizations (32-bit vectorized forward access; kernel-side zero-writes in the backward) give 1.20–1.36× on the forward kernel and 1.21–1.50× on the backward region while leaving forward, dKV, and dScore bitwise unchanged.
• The compressor now requests ex2.approx.ftz through fastmath= so it builds at the supported cutlass-dsl floor (#463).

Block-sparse attention (BSA)

• Removed dead fragment allocations (#392).
• Fixed a backward hang on cute-dsl 4.6.0 and corrected the bulk-copy elect_one gate for 4.6.2 and 4.7 (#382, #453).

Toolchain

• Moved to nvidia-cutlass-dsl 4.6.0 (#368) and cleared CUTLASS DSL deprecation warnings across the CuTe DSL kernels, including the .ptr migration for cute.struct scalar fields (#365, #376).

Tooling and Developer Experience ✨✨

• python -m cudnn.collect_env (#400) — a new environment-forensics tool for bug reports, wired into the issue template and README. It reports the frontend version with mismatch flags (imported cudnn.__version__ vs. pip metadata vs. torch.backends.cudnn.version()), traces the frontend's libcudnn dlopen search order, and distinguishes loaded from installed GPU libraries by parsing /proc/self/maps — flagging the version-confusion cases that dominate unreproducible issues. It is stdlib-only with individually guarded probes, so it still produces a report when import cudnn or torch is broken, and can be run standalone.
• Agent- and contributor-friendly repository layout (#402, #413) — a root AGENTS.md plus scoped guides under include/cudnn_frontend/, python/cudnn/, test/, and samples/; an llms.txt docs index; skills discovery for coding agents; and expanded CONTRIBUTING.md sections on development environment, testing, and formatting.
• Improved GitHub issue and pull request templates (#375), with PR label guidance documented in the template and AGENTS.md (#489).
• Documentation fixes for FE-OSS links and the DSA architecture code fence (#360), and clearer wording throughout the environment-report docs (#409).

Samples, Benchmarks and Tests 📊

• LayerNorm and RMSNorm samples now accept (B, S, H) tensor shapes directly instead of reshaping to (B*S, H, 1, 1) (#432).
• Added NWH and back-to-back causal conv1d notebooks with refreshed outputs (#246).
• Updated SDPA benchmarking artifacts for cuDNN 9.24.0.43 (#362) and 9.25.0.15 (#417).
• Fixed repo-root path resolution in the bench_moe benchmark (#348).
• Reorganized the FE OSS tests by feature (#372) and the GEMM projection tests (#374).
• Added SDPA edge-case coverage (#328) and fixed architecture-independent SDPA repro failures (#386).
• test_mhas_v2: extended backward random head-dim coverage to d=256 (#425); removed ALiBi, score_max/sum_exp, dropout randomization (#435) and with_rope (#474) from the random forward/backward tests.

Bug Fixes 🐛

C++ frontend

• Fixed Engine_v8::Knob::getMaxValue(), which returned the minimum value (#443).
• Fixed an uncaught ValueError in flatten_pass_by_value on malformed hex input (#343).
• Fixed a handle leak in the BatchNorm sample (#399).

Python / OSS kernels

• Fixed a default-stream race: _get_default_stream(None) now resolves to torch's current stream (#483).
• Fixed a block-scale quantize output alignment issue (#491).

DSA

• Fixed offset alignment, stream handling, and CUDA Graph capture in the indexer-backward paths; grad_loss is now consistently a single-element FP32 CUDA tensor (#354).
• Fixed two latent synchronization bugs in the SM100 DSA backward kernel (#395) and a latent TMEM WAR race in the SM100 dKV drain at head_dim = 576 (#396).
• Fixed a latent SMEM handoff race in the SM100 DSA indexer backward kernel (#426).
• Fixed stream-ordering and input-validation holes in the SM100 DSA backward interface (#429).
• Fixed SM100 DSA sink normalization (#421) and zero top-k lengths in the SM100 DSA backward (#439).
• Fixed the qh32 sparse backward path (#388).
• indexer_top_k now falls back to scalar stores for odd top_k (#407), and out-of-bounds lanes in the variable-length indexer top-k are fixed (#410).

Licensing 📜

• cuDNN Frontend is now Apache-2.0. NVIDIA-authored sources have been relicensed from MIT to the Apache License 2.0 using the standard NVIDIA OSS SPDX header (#408). Files that still carry surviving lines from external contributors whose permission has not yet been established remain under MIT, as do FlashAttention- and QuACK-derived files. The package metadata is now license = "Apache-2.0 AND MIT".
• Every source file now carries exactly one SPDX-License-Identifier tag. The complete per-file mapping — including the commit that introduced each surviving external line — is in LICENSING.md, alongside LICENSE.txt (Apache-2.0), LICENSE-MIT.txt, NOTICE, and THIRD_PARTY_LICENSES.txt.
• This change is comment- and header-only; there is no functional or API impact.

Acknowledgements 🙏

Thanks to everyone who contributed to this release:

@adshen, @Anerudhan, @bmanthos, @brandonfzhang, @chaseblock, @derdrdirk, @egilliam-nv, @fallintoplace, @hwanseoc, @hxbai, @JackRao123, @jhjpark, [@jiefan] @jiayus-nvidia, @kangbintNV, @kunlunl, @liujane-dev, @pmdavies-nv, @rmhaskarnvidia, @saltyminty, @sraman-rgb, @terminator123, @vedaanta, @vincejhan, @WanZzzzzz, @yanqinz2, @yanzhuo607, @YangXu1990uiuc, @yeliu-oss, and @zkyue.

Special thanks for the kernel contributions that came from outside this repository:

• The fused CSA Compressor forward and backward kernels were ported from Megatron-LM, following maintainer guidance on Megatron-LM PR #5984.
• The BF16 grouped GEMM, GLU, dGLU, and WGrad kernels originate from the CuTe DSL kernel library.
• The FROST / cuTile engines were developed with contributions from across the cuDNN team.