Skip to content

v13.4.0: [Release] CUDA Tile IR 13.4.0

Latest
Latest

Choose a tag to compare

View all tags
@dcaballe dcaballe released this 10 Sep 03:46
v13.4.0
7e8e2e6

This release is aligned with the CUDA Tile IR specification included in CUDA Toolkit 13.4.

Supported Architectures

  • (Developer Preview) Added support for sm_107 (Rubin) architecture.

New Operations

  • Added op cuda_tile.fpowi for element-wise floating-point exponentiation with a signed-integer exponent.
  • Added op cuda_tile.insert to insert a source subtile into a destination tile at a given subtile index. The source shape must evenly divide the destination shape.
  • Added op cuda_tile.memory_fence_alias_tko, a token-ordered fence that orders operations accessing the same physical memory through different virtual aliases.
  • Added op cuda_tile.gdc_launch_dependents_tko for grid dependency control, signaling that programmatic-dependent-launch (PDL) dependent kernels may begin. It is a no-op below sm_90.
  • Added op cuda_tile.gdc_wait_tko for grid dependency control, waiting for predecessor-kernel completion with acquire semantics. It is a no-op below sm_90 and pairs with cuda_tile.gdc_launch_dependents_tko.

New Types and Attributes

  • (Developer Preview) Added the f8E5M3FNU (8-bit floating-point) type. It is an alternative floating-point type intended for use as a block-scale type in cuda_tile.mmaf_scaled with f4E2M1FN (fp4) elements.

Modified Operations

  • (Developer Preview) Modified op cuda_tile.mmaf_scaled to add support for f4E2M1FN inputs with f8E5M3FNU scale factors accumulating to f32.
  • Modified op cuda_tile.loop to support function return: a cuda_tile.return may appear inside a loop body to return from the enclosing function.
  • Modified ops cuda_tile.load_view_tko and cuda_tile.store_view_tko to add a new inbounds attribute so that programs can convey that accesses along specified dimensions are statically known to be in-bounds, allowing the compiler to skip bounds checking for those dimensions. Dimensions left unmarked default to conservative bounds checking, which matches the behavior of prior versions.
  • Modified op cuda_tile.ftoi to add an optional saturating modifier that clamps out-of-range values and converts NaN to 0.
  • Modified op ftof to support a richer rounding-mode matrix, including nearest_away for f32 to tf32.
  • Renamed op cuda_tile.pow to cuda_tile.fpowf, complementing the new integer-exponent cuda_tile.fpowi.

Compiler Improvements

  • Added opt-in compiler optimization remarks that identify selected load and store instructions and explain TMA instruction-selection failures. In this release they are emitted by cuda-tile-optimize in verbose mode.
  • The Tile IR-level optimization pipelines in cuda-tile-optimize now apply canonicalization, common-subexpression elimination and loop-invariant code motion according to the selected optimization level. See documentation for further details.

Documentation Improvements

  • Documented that the combiner function for cuda_tile.reduce and cuda_tile.scan must be commutative as well as associative. This was already required by the implementation, which may reorder the combine freely.
  • Added memory-alignment guidance for pointer loads, stores, atomic operations, and view accesses, including byte-alignment requirements for sub-byte element types.
  • Clarified cuda_tile.partition_view and cuda_tile.strided_view out-of-bounds semantics, distinguishing in-bounds view indices from partially out-of-bounds tiles and documenting load padding and store masking.
  • Clarified the cuda_tile.tensor_view memory contract for 4-bit elements, including dense packing, byte-alignment requirements, and little-endian nibble order.
  • Expanded cuda_tile.gather_scatter_view documentation with sparse-dimension indexing rules and multidimensional gather, scatter, padding, and out-of-bounds examples.
  • Expanded optimization-hint reference documentation with per-operation and architecture-specific constraints.

Fixed Issues

  • Fixed an sm_120 compiler crash when an f16 constant was converted to an FP8 type with cuda_tile.ftof and the result was passed to cuda_tile.print_tko.

Known Issues

  • A cuda_tile.loop or cuda_tile.for operation may fail compilation or produce incorrect results when a tile produced by a load is carried between iterations. The issue may occur when the next iteration's tile is loaded before the carried tile's final use. As a workaround, place the next load after the carried tile's final use, or restructure the loop so the loaded tile is not carried between iterations.
  • Converting a tf32 tile loaded from global memory to f32 may produce incorrect values if the lower 13 mantissa bits of any loaded value are nonzero. These bits do not affect MMA operations on the loaded tile. The issue becomes observable only after conversion to f32. As a workaround, ensure that the lower 13 mantissa bits of every tf32 value loaded from global memory are zero.

Open Source Release
The following changes are specific to the open-source release.

  • Builds against LLVM 9ebb067a8a2b, which provides the f8E5M3FNU type definitions and the public MLIR_BINDINGS_PYTHON_EXTRA_* arguments for add_mlir_python_modules.
  • Added attribute ptr_attr to classify ptr values.
  • Fixed the print_tko example in the README and added a test for it. Fixes #16.
Assets 2
Loading