CUTLASS 4.6.0

[hwu36]

• Release documentation that explains how to accurately profiling GEMM performance.

CuTe DSL

• New features

  • New fine-grained compilation API: cute.compile_to that gives control over the what stage the compiler outputs. This feature allows customization of the path from compilation to runtime execution. cute.compile_to is considered experimental in 4.6.
  • Experimental Feature: Added the IKET (In-Kernel-Event-Tracing) profiler for instrumentation-based intra-kernel activities tracing. This enables fine-grained profiling and makes it easier to understand persistent, warp-specialized kernels' performance. This is a beta feature provided by CUTLASS Python until a NVIDIA DevTools product is released, there is no guarantee that this interface will remain stable!
  • Distribute compiler binaries to accompany cute.compile_to allowing users to build customized compile-execute pipelines outside of Python. Both static and shared compiler and executor/runtime libraries will be provided. Compiler binaries will be uploaded to GitHub with each release.
  • Supported AoT cross-compilation for aarch64-linux-gnu
  • Support for two launch attributes: launch completion events (cudaLaunchAttributeLaunchCompletionEvent), for recording an event once all thread blocks have begun executing, and launch programmatic events (cudaLaunchAttributeProgrammaticEvent), for PDL event-based synchronization
  • Supported auto calculating per-kernel shared memory carveout preference, or use new launch option preferred_smem_carveout to set manually.
  • Auto-deduced smem size for launching kernels
    • Launch config smem now defaults to None for auto-calculating kernel shared memory usage, which is recommended unless manual control is required.
    • Warnings will be raised when the manually set shared memory size is insufficient or exceeds the GPU maximum.
    • The default shared memory usage calculation aligns with CUDA C++ static shared memory behavior, i.e. summing all allocations additively.
    • An additional launch option smem_merge_branch_allocs is provided to merge shared memory allocations across mutually exclusive code branches, which is recommended for inlined mega-kernels to reduce total footprint.
  • SASS dumping in DSL is now supported in a self-contained manner - no CUDA toolkit installation required to get nvdisasm

• Bug fixing and improvements

  • Add the missing elect_one in cute.copy for bulk copy.
    • The elect_one required for async bulk copy was missing in cute.copy. It's now generated in cute.copy automatically.
    • Nesting elect_one will cause functionality issues. Please remove elect_one around cute.copy with async bulk copy.
    • Elect_one around direct async bulk copy instruction should be kept as it bypasses the cute.copy layer and will not be affected by this fix.
    • Affected copy atoms are CopyBulkG2SOp, CopyBulkG2SMulticastOp, CopyBulkS2GOp, CopyBulkS2GByteMaskOp, and CopyBulkS2SOp.
    • An Example showing changes to avoid nesting elect_one could be found in this PR
  • Improvements on linter support with more type ignores cleaned up
  • Improvements on tvm-ffi CUDA runtime error diagnostics
  • Improvements on dataclass support for TVM-FFI
  • Fixed a regression on compilation time
  • Enhancement on compile time checks to reject mis-aligned smem operand for TMA
  • Long-deprecated API clean-up, including:
    • cute.core.ThrMma, please use cute.ThrMma instead
    • cute.core.ThrCopy, please use cute.ThrCopy instead
    • cute.make_fragment, please use cute.make_rmem_tensor instead
  • Fixed following issues
    • [BUG] pip upgrade/downgrade breaks nvidia-cutlass-dsl #3132
    • [CuTe DSL] libs-base and libs-cu13 4.4.x ship divergent _cutlass_ir.so for the same path; libs-base emits malformed _mma PTX for SM120 mxf4nvf4 mma #3170
    • [BUG] cutlass-dsl 4.5.1: -libs-base and -libs-cu13 wheels write the same path #3259
    • [BUG] cute.gemm rejects valid SM90 WGMMA m64n8k32 in SS mode — degenerate partition layout (1,1):(0,0) #3290
    • [BUG][CuTeDSL] AOT compilation fails with GPUArch("sm_100f") + TVM-FFI on B200 node #3298
    • [BUG] UInt64 constants are compared to i64::max instead of u64::max in consteval #3312
    • [BUG] #3327
    • [BUG] Unhelpful error when cutedsl runtime shared libs aren't found #3329
    • [BUG] CuTeDSL segfaults when using vector<fp8> #3342
    • [BUG] Allow boolean types for sm100_utils.make_smem_layout_epi #3348

CUTLASS Operator API

• CUTLASS Operator API is a new addition to the CUTLASS Python stack, providing easy interfaces
  to discover CUTLASS Python DSL kernels & integrate them in your code.
  • pip install nvidia-cutlass-operators to get started
• Operator API Overview
  • See currently supported kernels
• Basic GEMM tutorial
  • More tutorials here
• GitHub source
• API Reference

CUTLASS C++

• Add example 113 for Hopper GEMM with activation fusion.
  • Supports standard and gated activations (e.g., SiLu) with fp8 and fp16 inputs.
  • Covers both regular GEMM and grouped GEMM variants.
• Improve SM90 grouped/ptr-array GEMM with EVT support.
  • Adds the EVT (Epilogue Visitor Tree) plumbing required to do activation, bias, and auxiliary-tensor fusion inside SM90 grouped and ptr-array GEMM kernels.
• Add ptr-array TMA collective for tensor/token-scaled FP8 grouped GEMM Blackwell SM120/SM121 kernels.
  • Implement CollectiveMma and CollectiveBuilder specializations for MainloopSm120ArrayTmaWarpSpecialized, enabling ptr-array grouped GEMM (MoE expert dispatch) with tensor- and token-level FP8 scaling.
  • Corresponding unit test
• Add tileN = 8,16 for Blackwell SM120 blockscale GEMM kernels.
• Fix DescriptorIterator::operator+ in mma_traits_sm100.hpp to use 32-bit arithmetic on CUDA toolkit version <= 13.3, preserving the high half of the smem descriptor.
• Fix a CUDA structured bindings header issue.
• Various improvements and fixes from the community and CUTLASS team. Thanks to everyone who submitted PRs!
• Optimal code generation with CUDA toolkit versions 13.3.

---

This discussion was created from the release CUTLASS 4.6.0.

[Peter9606]

Hi @hwu36, thanks for the release! It seems that the documentation link for profiling GEMM performance in the description is currently broken (returning a 404 error). Could you please double-check or update it when you have a moment? Thanks!

[hwu36]

it is fixed in the main and release/4.6 branches. thanks for catching.