Skip to content

v1.6.0

Latest

Choose a tag to compare

@jansel jansel released this 15 Nov 18:22
· 43 commits to main since this release
aaf24f4

What's Changed

  • [bench] Direct baseline adapters allocate their output per call by @jansel in #4214
  • [cutedsl] Zero-fill elision and host full-sum tail: constexpr branches, chained loop ranges, sum / scalar by @jansel in #4215
  • [cutedsl] cute_pdl: the cute_pdl_spread pad, tcgen05 release after each role wait, the flash-backward surface by @jansel in #4216
  • [cutedsl] Packed half atomics in the collective epilogue; fp16 atomic outputs accumulate in fp16 by default by @jansel in #4217
  • [cutedsl] cute_pdl: host zero fills as dependent fill kernels, the wait at the first touch of a filled tensor by @jansel in #4218
  • [cutedsl] mma_dag: admit fixed-size tile loops; the gdn recurrence yields to the dag knob by @jansel in #4219
  • [cutedsl] Gathered MMA: bind the row recipe's per-group metadata once per CTA, in the kernel's own scalar registers by @jansel in #4220
  • [cutedsl] Pointwise: Philox seed as a scalar kernel argument, multi-packet lane loops, streaming store policy by @jansel in #4221
  • [cutedsl] GEMV hl.dot on mma.sync: cute_dot_fragment_mma, and the cp.async ring route cute_dot_ring_mma by @jansel in #4222
  • [cutedsl] Register-MMA row program for gated jagged attention at 16/32-row query tiles by @jansel in #4223
  • [cutedsl] Grouped tcgen05 row-union kernel: per-tile PDL wait, epilogue rounds, trimmed persistent prologue by @jansel in #4224
  • [cutedsl] Split-K workspace: reducer loads in flight behind PDL, and a reducer-free atomic epilogue knob by @jansel in #4225
  • [compiler] Reduce over multiple dims as one merged reduction dim by @coder-2011 in #4030
  • [CI] Bump tritonbench pin for the TPU bridge torch.accelerator fix by @yushangdi in #4231
  • [pallas] keep launcher caches from leaking device buffers by @cota in #4182
  • [runtime] stop kernel_source from keeping kernels alive by @cota in #4183
  • [autotuner] Poison only the cached blocks a fresh allocation can reuse by @yushangdi in #4232
  • [autotuner] Replace the benchmark worker after a device out-of-memory error by @yushangdi in #4233
  • [pallas] do not register the run time of failed TorchTPU executions by @cota in #4184
  • [autotuner] Warn when final selection disqualifies some finalists by @yushangdi in #4234
  • [examples] Compute the attention reference without autograd by @yushangdi in #4235
  • [autotuner] Release the cached device memory once the baseline is computed by @yushangdi in #4246
  • [cutedsl] Allow a lane-loop split over packed-word loads when the body writes nothing by @yushangdi in #4247
  • [pretuned_kernels] Use fast_math for cross_entropy by @yushangdi in #4241
  • [megakernel] gemma4_a4b_moe: drop the stale reduction_loops from the B200 config by @yushangdi in #4248
  • [megakernel] gpt_oss_moe: drop the stale reduction_loops from the standalone routing config by @yushangdi in #4249
  • [megakernel] kda_decode: import rms_norm_gated from vLLM's ops.kda by @yushangdi in #4250
  • [runtime] Cache the Triton driver backend in the scratch allocator by @yushangdi in #4242
  • [megakernel] qwen3_decode_layer: size the persistent-vs-standalone bound to its measured noise by @yushangdi in #4251
  • [runtime] Memoize the fast-dispatch schema per exact argument layout by @yushangdi in #4243
  • [pallas] wait for TorchTPU tensors when tracing device time by @cota in #4185
  • [autotuner] fix misreporting of configs with compile failures by @cota in #4186
  • [runtime] Key omitted-default calls on guard facts published later by @yushangdi in #4244
  • Cut CuTe config normalization and factoring pass costs in slow tests by @jansel in #4236
  • Add --codex-sandbox to autoreview by @jansel in #4237
  • Stop importing examples/ as a package by @jansel in #4238
  • [runtime] Keep storage-offset guard facts on prepared calls by @yushangdi in #4245
  • Keep agent skills in .agents and point .claude at it by @jansel in #4239
  • Skip the native-seeded mamba2 case whose block_m exceeds chunk_size by @jansel in #4240
  • [megakernel] more pretuned megakernels for sm100 by @ethche in #4211
  • [compiler] Order the pointer indexing strategy's loads and stores when placing lane loops by @aditvenk in #4252
  • [compiler] Add backend-neutral seams for block reductions, reduction indices and config defaults by @aditvenk in #4253
  • [metal] Reduce across threads and lanes, and lower the dashboard's elementwise and reduction kernels by @aditvenk in #4254
  • [metal] Lower K-loop matmuls through MPP cooperative tensors by @aditvenk in #4255
  • [benchmark] Re-tune the H100 linear-attention table for current codegen by @yushangdi in #4262
  • [benchmark] Restore the B200 linear-attention table to the kernels it was tuned as by @yushangdi in #4291
  • [metal] Lower online-softmax attention through the MPP planner by @aditvenk in #4256
  • [Pallas] Count VMEM scratch in the launcher's VMEM estimate by @yushangdi in #4292
  • [testing] Split the test suite into backend and CPU lanes by @jansel in #4273
  • [ci] Run CPU lanes and shards in CI and trim the pull-request matrix by @jansel in #4274
  • [compiler] Skip CuTe codegen passes whose trigger construct is absent by @jansel in #4275
  • [testing] Shrink the CuTe grid tests to all-pairs covers and shared references by @jansel in #4276
  • [testing] Compile each CuTe flash and collective kernel once per module by @jansel in #4277
  • [testing] Mark the slow tier and shrink the subprocess and autotune tests by @jansel in #4278
  • Ask agents to run autoreview.py on commits and finished work by @jansel in #4279
  • [metal] Size the threadgroup from num_warps, and cut MPP staging and mask overhead by @aditvenk in #4257
  • [metal] Let the autotuner choose persistent launches by @aditvenk in #4258
  • [metal] Lower matmul chains: shared threadgroup tiles, whole-K hl.dot, dot results as operands by @aditvenk in #4259
  • [Pallas] Count emit_pipeline buffers in the launcher's VMEM estimate by @yushangdi in #4290
  • [Pallas] Pad a pl.ds dim for every loop that tiles it by @yushangdi in #4297
  • [metal] Retire the MPP graph pass; the planner lowers every K-loop matmul by @aditvenk in #4260
  • [autotune] Update autotuner API documentation by @yushangdi in #4125
  • [metal] Add a Metal benchmark of the dashboard's nightly kernels by @aditvenk in #4263
  • [cutedsl] Backend interface: hooks with no-op defaults, the isolation check in lint, and the CuTe code that lived in core moves into the backend by @jansel in #4301
  • [cutedsl] Retire dead and duplicated code: the core cute_* aliases, the leftover layout and dormant tcgen05 pipeline modules, CodegenDict and the memory_ops shim, cluster-helper and heuristic shims by @jansel in #4302
  • [cutedsl] Retire the superseded pre-move copies: selection twins, scan and reduction scaffolds, the predicated-load factory and the role sets take over; shared fill walker; resident-while memo by @jansel in #4303
  • [cutedsl] Flat grouped kernel: named plan placeholders, the operand-pipeline docstring, and the flash render golden, PASS2 equivalence and static-tail tests by @jansel in #4304
  • [cutedsl] Benchmark harness, the mamba2 reference and the example tests: report completeness per provider, NaN-only reference errors, the chunked-reference exponent mask, Blackwell gates by @jansel in #4305
  • [cutedsl] Tests and docs: cpu_render_target in the remaining CPU render files, the kernel catalogue, the cluster FMA pair fold, the README's test pins by @jansel in #4306
  • [metal] Drop duplicate Metal tests by @aditvenk in #4264
  • [cutedsl] Keep five boolean predicates typed as bool under newer PyTorch by @jansel in #4331
  • [ci] Make lint.sh run the pre-commit hooks CI runs by @jansel in #4332
  • [scripts] Add gpu_board.py to share the machine's GPUs between agents by @jansel in #4333
  • [testing] Add run_tests.py: CI's test lanes locally, sharded across GPUs by @jansel in #4334
  • [docs] Rewrite AGENTS.md and the skills for autonomous development by @jansel in #4335
  • [AMD][flydsl] Import ReductionOp from flydsl.expr by @umechand-amd in #4267
  • [AMD][FLYDSL] Bugfix Raise an error on unsupported tensor access by @umechand-amd in #4270
  • deprecate constrain_unify and move legacy symbolic unification into Helion (#4199) by @laithsakka in #4199
  • [CI] Spread ROCm benchmark shards over the host's GPUs; keep a shard's results past a crashed kernel by @yushangdi in #4287
  • [autotuner] Drain the HIP launch queue in do_bench's warm-up and timed loops by @yushangdi in #4293
  • [CI] Benchmark MI350X on torch 2.15 with ROCm 10.0 by @yushangdi in #4326
  • [docs] Describe Helion's backends in the README and docs intro by @yushangdi in #4321
  • [compiler] Name a callable object by its type in host type annotations by @yushangdi in #4294
  • [torch.compile] Emit valid host calls for annotated and callable-arg kernels by @yushangdi in #4295
  • [cute] Keep score_mod_bwd's gradient literals as nodes in the Jacobian region by @yushangdi in #4296
  • [cute] Give the flex hook's cooperative stream its own KV tile binding by @yushangdi in #4298
  • [cute] Plan the flex hooks' shared rings over every region of the plan by @yushangdi in #4319
  • [cute] Re-stage a partial flex tile's first chunks after a full tile by @yushangdi in #4320
  • [metal] Parse MetalOverrides' MSL with a C++ expression parser by @aditvenk in #4352
  • [cutedsl] Enable tcgen05 on grouped_gemm_deepgemm binds with the current API by @yushangdi in #4367
  • [CI] Benchmark H100 and B200 on torch 2.15 to fix the mamba2_chunk_state nightly failures by @yushangdi in #4372
  • [autotuner] Isolate forked autotune workers from tqdm's lock and parent garbage by @yushangdi in #4371
  • [cute] Store the flex forward's running row max beside the LSE by @yushangdi in #4365
  • [cute] Decline a flex LSE store that mixes the softmax's units by @yushangdi in #4366
  • [cutedsl] Call cp_async_shared_global from cutlass.experimental.primitives by @yushangdi in #4369
  • [testing] Retry a racing read in test_hold_claim_renews_and_removes by @AmesingFlank in #4400
  • [cutedsl] Accept CuTe DSL 4.7.1 for the split barrier init by @AmesingFlank in #4397
  • [metal] Lower F.gelu, exact and tanh-approximate by @aditvenk in #4353
  • [autodiff] Alias an unsqueeze of a scaled tangent by @AmesingFlank in #4398
  • [autodiff] Re-enable the mean backward tests on ROCm by @AmesingFlank in #4399
  • [compiler] Avoid tensor wrapper warnings for identity comparisons by @ethche in #4394
  • [pallas] support pltpu.TensorCoreMesh on JAX 0.13+ by @cota in #4401
  • [metal] Don't take a loaded host tensor's dims as blocks around an MPP matmul by @aditvenk in #4354
  • [autotuner] Add AccuracyCheck, an accuracy check the autotune cache keys by @yushangdi in #4380
  • [metal] Give a dim hl.specialize pins under dynamic shapes lanes, as a literal by @aditvenk in #4355
  • [compiler] Support reductions over tunable factory dimensions by @ethche in #4382
  • [metal] Give each jammed lane its own lane array when the array is stored at another lane's index by @aditvenk in #4356
  • [compiler] Move the literal-dim analysis and operand dim matching out of cute/ by @aditvenk in #4357
  • [metal] Refuse dims a thread would read at another element, instead of miscompiling by @aditvenk in #4358
  • torch_tpu: update pin to bfe8373b6e113ca1480119a71cb043449491db0c by @cota in #4402
  • [compiler] Branch a host-registered block's loops into every root that nests them by @aditvenk in #4359
  • [metal] Read MPP view sizes and outer strides at run time, and stage dequantized operands in bfloat by @aditvenk in #4360
  • [testing] Pin debug_dtype_asserts off in the Pallas reduction-strategy tests by @yushangdi in #4429
  • [metal] Load masked elements unconditionally and select the fill afterwards by @aditvenk in #4361
  • [metal] Fuse float multiply-adds into metal::fma by @aditvenk in #4362
  • [metal] Lower Tensor.view(dtype) between dtypes of one width by @aditvenk in #4363
  • [metal] Jam a lane loop into a for loop whose body loads a value no lane depends on by @aditvenk in #4364
  • [testing] Account for the fbcode MTIA tunables in the config key tables by @AmesingFlank in #4425
  • [testing] Look up fbtriton by name in skipIfFbTriton by @AmesingFlank in #4426
  • [testing] Pin the benchmark timeout in the compile deadline tests by @AmesingFlank in #4427
  • [testing] Keep the inherited PYTHONPATH in the runtime_hooks import test by @AmesingFlank in #4428
  • [scripts] Add a multi-GPU runner mode and preserve failure diagnostics by @jansel in #4351
  • [compiler] Preserve typed input and scalar semantics across backends by @jansel in #4407
  • [compiler] Preserve tunable, binding and config identity end to end by @jansel in #4408
  • [compiler] Preserve scalar and reduction semantics through lowering by @jansel in #4409
  • [cutedsl] Preserve typed operands and ownership across ordered MMA phases by @jansel in #4410
  • [compiler] Preserve dynamic host geometry and guarded structural exports by @jansel in #4411
  • [compiler] Preserve host storage provenance across aliases and effects by @jansel in #4412
  • [compiler] Preserve logical reductions, captures and graph effects by @jansel in #4413
  • [cutedsl] Preserve computed MMA ownership and fragment lifetimes by @jansel in #4414
  • [cutedsl] Fuse exact S2F6 BF16 decoding on supported targets by @jansel in #4415
  • [cutedsl] Lower proved paged block-scaled online attention by @jansel in #4416
  • [cutedsl] Schedule proved attention pairs with explicit capture overlap by @jansel in #4417
  • [autotuner] Add rotating input banks and preserve valid finalist measurements by @jansel in #4418
  • [cutedsl] Lower conditional attention and expose configured schedules by @jansel in #4419
  • [autotuner] Refine compiler choices with exact search and cache identity by @jansel in #4420
  • [benchmarks] Package fixed SDG profiles with ordinary Helion compilation by @jansel in #4421
  • [cutedsl] Search persistent and CtaGroup.TWO tiles for output-edge GEMMs by @nullplay in #4322
  • [cutedsl] Promote a wave-quantized default tile for dense tcgen05 GEMMs by @nullplay in #4323
  • [cutedsl] Charge the edge TMA-store C ring once; drain CTA pairs after dealloc by @nullplay in #4324
  • [cutedsl] Use six 64-deep AB stages for the direct-entry default by @nullplay in #4327
  • [cutedsl] Price the plain pair's (128, 32) C subtile in the paired SMEM model by @nullplay in #4329
  • [cutedsl] Extend aligned-N physical tiles to 16-bit paired GEMMs by @nullplay in #4330
  • [cutedsl] Move the half-MMA fragment knob past radix selection's shifted priority by @nullplay in #4446
  • [runtime] Declare packaging as a runtime dependency by @aditvenk in #4447
  • [cutedsl] Store fused view/permute outputs through a hierarchical TMA tensor by @nullplay in #4346
  • [autotuner] Time the final selection with trace kernel durations by @nullplay in #4347
  • [cutedsl] Admit FP32 outputs and partial last N tiles in aligned-N pairs by @nullplay in #4348
  • [cutedsl] Default to aligned-N pairs and count partial tiles in pair gates by @nullplay in #4349
  • [cutedsl] Prove bias-row allocations and add output-edge and K-tail defaults by @nullplay in #4383
  • [cutedsl] Take masks, selects and shared operands in epilogues; stage bias rows by @nullplay in #4384
  • [cutedsl] Store output-edge aux tiles through TMA and search their tile order by @nullplay in #4385
  • [cutedsl] Run persistent one-CTA N-edge tiles and prefetch partial-output rows by @nullplay in #4386
  • [cutedsl] Stage aux operands of any dtype and cp.async an unaligned M-major A by @nullplay in #4387
  • [cutedsl] Default to pairs for narrow N, small grids and residual epilogues by @nullplay in #4388
  • [cutedsl] Run batched GEMMs with column-major operands and head-merge stores by @nullplay in #4389
  • [cutedsl] Add defaults for transcendental epilogues, split K and long-N L2 by @nullplay in #4390
  • [cutedsl] Default batched GEMMs to one-CTA tiles and batch-slowest pair walks by @nullplay in #4391
  • [cutedsl] Prefetch epilogue operands and add a third warp group over short K by @nullplay in #4392
  • [cutedsl] Stage unaligned output row stores through SMEM by @nullplay in #4393
  • [cutedsl] Size every tcgen05 epilogue store by the widest output dtype by @nullplay in #4404
  • [cutedsl] Store aux-only outputs from the tcgen05 matmul epilogue by @nullplay in #4405
  • [scripts] Defer CuTe verification until editable torch is rebuilt by @jansel in #4463
  • [scripts] Ask nvidia-smi again when it fails instead of reporting no GPUs by @jansel in #4448
  • [docs] Squash sooner, answer review findings, clean up scratch, test Pallas by @jansel in #4449
  • [triton] Divide with tl.div_rn for truncated and floored quotients and addcdiv by @jansel in #4450
  • [testing] Run CuTe source checks on the CPU lane; fix order-dependent tests by @jansel in #4451
  • [autotuner] Bound worker shutdown and compiles; check NaN and fp8 outputs by @jansel in #4452
  • [compiler] Fix in-place updates, dead-code elimination, barriers and scans by @jansel in #4453
  • [compiler] Mask padded hl.arange lanes and partial tiles; fix config slots by @jansel in #4454
  • [compiler] Fix rolled reductions, tunables in device code and sort/topk by @jansel in #4455
  • [cutedsl] Fix wrong results, crashes and refusals in generated CuTe code by @jansel in #4456
  • [benchmarks] Port better-benchmark's kernels to Helion and time them vs Inductor by @jansel in #4457
  • [cutedsl] Bound the search, coalesce strided loads and privatize atomic adds by @jansel in #4458
  • [cutedsl] Add reduction lane layouts: loads-first, jammed lanes, unpadded rows by @jansel in #4459
  • [cutedsl] Serve scratch reloads from registers and drop dead scratch stores by @jansel in #4460
  • [cutedsl] Pair fp32 lane math, use native reductions, peel tails, add seeds by @jansel in #4461
  • [cutedsl] Divide as the Triton backend does: div.full.f32 by default by @jansel in #4462

Full Changelog: v1.5.2...v1.6.0