Skip to content

v0.4.0

Latest

Choose a tag to compare

@github-actions github-actions released this 17 Nov 23:01

This release contains:

  1. SQTT thread-trace profiler for RDNA3 and RDNA4. The decoder matches AMD's reference decoder on its traces. Capture is one option from Rust (ProfileOptions { sqtt: Some(SqttOptions) }) or SVOD_SQTT. The report ranks why each kernel is slow, maps findings to tk statements through the kernel's own line table, and replays and compares saved captures offline. The fixes its traces found on the RX 9070 XT:

    • attention with a bias: d128 −19 %, d64 −7 %;
    • nearest with the RDNA4 split barrier: d128 k16 −5.6 %.

    A new docs page, Profiling with a Thread Trace, covers it.

  2. MI300X (gfx942) tile kernels:

    • bf16 GEMM 4096³ at 647.6 TFLOP/s (rocBLAS 699.1);
    • attention d128 at 440.8 (CK 418).

    Indexing works past 2³¹ elements, and AMD plans can run longer than 30 s.

  3. GEMM tail split along K for partial last waves, and a raster knob. New cost-model terms come from measurements on MI300X, the RTX 3060 and the RX 9070 XT.

  4. AMD multi-lane hangs fixed. Kernargs use a ring, and the host flushes HDP as ROCr does. Four compute lanes are now the default. Four threads dispatching at once finish 3.1–3.4× sooner on gfx1201 and gfx1151.

  5. tk invariants are types:

    • access kinds;
    • kernel brands;
    • owned accumulators;
    • admitted configs;
    • program phases.

    Misuse that tests used to catch is now a compile error.

  6. YOLO26x f16 on the RX 9070 XT: 3.66 → 3.39 ms. Pinned batches plan static, narrow convs pad their channels, and channel windows remove every CSP chunk/cat copy.

  7. Atomic add and svod_tk::ops::scatter_add on CUDA, AMD and CPU.

  8. Breaking changes:

    • HostWrite<'_, T> replaces the unsound as_host_bytes_mut, as_array_mut, jit::view_mut and Tensor::array_view_mut.
    • The tk builder API is typed:
      • Kernel::record;
      • ParamRef<'k, T, A>;
      • Admitted specs;
      • Program<P: Phase>.
    • The tune store is format 11. Store files are keyed by crate version, so 0.4.0 starts a fresh store, and AMD kernels miss the object cache once.

Crates

What's Changed

  • Bump DeterminateSystems/determinate-nix-action from 3.23.3 to 3.24.1 by @dependabot[bot] in #209
  • tk: MI300X (gfx942) kernels, wide matrix core, and indexing past i32 by @npatsakula in #213
  • tk: parameter kinds, kernel brands and owned accumulators in the type system by @npatsakula in #214
  • tk: admitted configs and program phases as types by @npatsakula in #215
  • tk: a GEMM's raster is a config knob by @npatsakula in #216
  • tk: GEMM tail split and the cost-model terms it measured by @npatsakula in #217
  • amd: multi-lane hangs fixed: kernarg ring, host HDP flush, HostWrite guard by @npatsakula in #218
  • amd: no machine sinking under a clang that sinks matrix products by @npatsakula in #219
  • amd: review fixes (AQL retirement, queue errors, KFD runtime, cache acquire) by @npatsakula in #220
  • tk: review fixes (admitted specs, builder refusals, planner terms) by @npatsakula in #221
  • runtime: skip a kernel whose grid a variable bound to 0 empties by @npatsakula in #224
  • device: a spec whose name is not ASCII is refused instead of panicking or matching an alias by @mexus in #235
  • Bump DeterminateSystems/determinate-nix-action from 3.24.1 to 3.24.2 by @dependabot[bot] in #227
  • [+] tk: atomic add and scatter_add by @npatsakula in #211
  • test: shared fixtures, probes as examples, tk_bench pin by @npatsakula in #222
  • tk: tk_bench times configs as the tuner does by @npatsakula in #223
  • sqtt: an SQTT profiler for RDNA3 and RDNA4 by @npatsakula in #229
  • sqtt: a ranked verdict per kernel, and the attention and nearest fixes it found by @npatsakula in #230
  • tk: a pinned batch plans static, narrow convs pad their channels, tk_rank regret by @npatsakula in #231
  • tk, yolo: channel windows, so CSP blocks stop copying for chunk and cat by @npatsakula in #232
  • sqtt: statements from the traced kernel's own line table, no twin by @npatsakula in #236
  • sqtt: vendored data trimmed, one classification pass, module split and a profiling page by @npatsakula in #237
  • tk review stack: profiler, SQTT verdicts and statements, channel windows, tk_bench timing by @npatsakula in #238
  • [*] docs: figures for the thread-trace profiling page by @npatsakula in #239

Full Changelog: v0.3.0...v0.4.0