Repository navigation
v1.13.0-release
cudnn frontend v1.13 release notes
cudnn frontend v1.13 is the preferred cudnn frontend version for cudnn version 9.11.0 and above.
New API
Introduces device descriptor, which allows for device-less compilation of cudnn graph on a target GPU. See newly added sample and documentation.
Improvements
SDPA
-
Introduced
generate_statsas a replacement foris_inference, to improve clarity. Whengenerate_statsis true, the output will contain the stats tensor. When migrating fromis_inference(which is now deprecated), note thatgenerate_statshas the opposite meaning, so pass it the negation of the bool that was passed tois_inference. -
Improved support checks for left and right diagonal bands in conjunction with the diagonal alignment.
-
Improved error handling for large head dimension (d > 128) in sdpa bprop.
Normalizations
- Added support for fused Layernorm with Relu and samples for Layernorm with relu bitmask dump
Others
-
Published improved SDPA training benchmarks for fp8 and fp16/bf16 graph patterns.
-
Enable int4 Weight only Quantization for matmul. See example
-
Allow block scale dequantize (required for low precision matmul) to take 2-D scale factor.
-
Allow reductions to accept deterministic as a attribute.
-
Added pybinds for block scale dequantize.
Bug Fixes
- Fixed the sliding window attn_score_modifier function allowing it to set true negative infinity.