Skip to content

0.11.0-rc3

Pre-release
Pre-release

Choose a tag to compare

@jomitchellnv jomitchellnv released this 26 Jul 21:18
ad21650

Release candidate for 0.11.0

What's new
Triangle Attention: kv_lengths fast path
Added a kv_lengths argument to triangle_attention, plus a mask_to_kv_lengths
helper. Providing per-row key/value lengths routes variable-length (right-padded)
inputs to the optimized fused attention kernel on Blackwell (SM100f), instead of the
dense-mask path.
kv_lengths and mask are mutually exclusive.
Dense boolean masks are still supported; when a dense mask forces execution off the
fast path, a warning is now emitted so callers can switch to kv_lengths for best
performance.
Attention Pair Bias
New fused attention-with-pair-bias primitive with an optimized Blackwell (SM100f)
implementation.
Kernel / performance improvements
Continued improvements to the Blackwell (SM100 / SM103) fused-attention kernels.
Notes
This is a release candidate intended for validation ahead of 0.11.0.