ONNX Runtime v1.29.0

Announcements & Breaking Changes

• onnxruntime-web has announced the deprecation of WebGL and JSEP. The native WebGPU EP is the recommended path going forward. See the deprecation and migration plans for details (#29716, #31683).
• POSIX telemetry is now available on Linux, macOS, Android, and iOS when ONNX Runtime is built with telemetry enabled. It does not change the public ABI, WebAssembly remains telemetry-free, and setting ORT_DISABLE_TELEMETRY=1 before initialization disables non-Windows telemetry for the process (#27379, #29872).
• The unused internal onnxruntime/python/tools/tensorrt dashboard tooling was removed. This does not affect the TensorRT Execution Provider APIs (#29395).

Security Fixes

Path, bounds, and input validation

• Fixed a path traversal vulnerability in TensorRT and NvTensorRTRTX engine refitting by making external-data path validation unconditional (#29396).
• Validated the CPU MoE k attribute against the number of experts and fixed a CPU TensorScatter security issue (#29907, #29916).
• Added missing rank, shape, and parameter validation for pooling, LSTM and DynamicQuantizeLSTM, Sampling, FeatureVectorizer, SkipLayerNorm, QLinearConv, Whisper decoding, RNN activations, GridSample, contrib Range, and CropAndResize (#29254, #29255, #29265, #29579, #29595, #29605, #29871, #31636, #31671, #31675, #31676, #31684).
• Hardened CUDA indexing and buffer handling in GridSample, transpose, GatherBlockQuantized, InstanceNormalization, LayerNorm/RMSNorm, BeamSearch, DeformConv, AveragePool, and MaxPool (#29581, #29631, #29638, #31640, #31642, #31644, #31645, #31647, #31650).
• Fixed packed sub-byte tensor over-copying in OrtApi::GetValue and validated DML constant tensor byte sizes (#29157, #31665).

Supply chain and tooling

• Updated npm lockfiles, refreshed the Next.js end-to-end fixture lockfile for security advisories, and upgraded adm-zip for onnxruntime-node (#29827, #29926, #31192).

New Features

Core APIs & Runtime

• Default intra-op and inter-op thread-pool sizes can now be set with ORT_INTRA_OP_NUM_THREADS and ORT_INTER_OP_NUM_THREADS. Explicit thread settings still take precedence, and 0 preserves machine-sized defaults (#29688).
• Added weightless-model support for all initializer types, allowed zero-input EpContext nodes, and wired maximum-shape inference into workspace estimation (#29607, #29799, #31613).
• Added ONNX-domain support for rotary embedding and a fused MRotaryEmbedding contrib operator for Qwen mRoPE variants (#29261, #31728).
• Added multi-shape profiling to onnxruntime_perf_test through --data_shape, plus verbose graph-transformer tracing and broader inference-session error-path coverage (#29555, #29558, #29569, #29571).

Execution Provider ABI & Plugin EPs

• WebGPU now supports device-free compile-only sessions for offline graph transformation (#29681).
• Expanded CUDA plugin EP packaging and testing, including Windows ARM64 package and size options, updated package outputs, and aligned architecture selections across Python, C API, TensorRT, Node.js, and plugin packages (#31635, #31722, #31992).
• Improved plugin lifecycle handling by unloading failed EP library loads and fixing allocator-deleter lifetime (#29634, #29770).

Execution Provider Updates

NVIDIA CUDA EP

Attention and decoding

• Added PagedAttention with quantized KV cache, XQA decode, MLA, QK-Norm, and head-sink support (#29912).
• Extended quantized KV-cache support with attention sinks, independent and per-channel scales, sliding-window cache support, and a fused K/V dequantization launch (#29900, #29904, #31480).
• Added a cuDNN SDPA decode tier to the standard ONNX Attention CUDA kernel and enabled cuDNN SDPA for contrib Attention (#29715, #29717).
• Added attention_bias support to the GroupQueryAttention unfused path and state_window support to LinearAttention and CausalConvWithState for MTP (#29525, #31157).
• Fixed LinearAttention on GPUs with limited shared memory (#31982).

MoE and quantized GEMM

• Added NVFP4 QMoE, including native FP4xFP4 prefill on SM120, faster decode GEMV, fused routing/finalization paths, and reduced activation and weight-dequantization overhead (#29697, #29824, #29887, #29919, #31156, #31159, #31349, #31479).
• Added MatMulBlockQuantizedFp4Weight and MatMulBlockQuantizedFp8Weight, plus block-scaled tensor-core/GEMV decode paths, packed FP4 decode, M-tiling, and folded W8A8 activation QDQ (#29818, #29850, #29896, #31155, #31481).
• Improved MatMulNBits and QMoE robustness and efficiency by optimizing 8-bit dequantization, releasing raw MXFP4 initializers after prepack, and fixing subgraph prepacking and mixed FP8/FP4 build failures (#29852, #31141, #31154, #31350).

Operators and collectives

• Added LinearAttentionGate, GatedRMSNorm, and GatedAdd contrib operators (#31158, #31835).
• Added bfloat16 support to AllReduce, AllGather, and AllToAll (#31571).
• Fixed the default zero point in CUDA GatherBlockQuantized (#31693).

WebGPU EP

• Added DFT, HardSwish, Max/Min, Trilu, GRU, PRelu, MatMulBnb4, and MRotaryEmbedding support (#29454, #29587, #29828, #29833, #29840, #29845, #30512, #31976).
• Expanded integer support across Clip, Reshape, Cast, Add, Tile, Concat, Expand, Gather, CumSum, Max, and Min (#29830, #29834, #29835, #29839, #29844, #29847, #29854, #29861, #29897, #29918, #31049, #31702, #31709).
• Added the initial WebGPU PagedAttention implementation and moved Softmax and non-flash Attention to online algorithms (#29694, #29724, #31611).
• Improved MatMulNBits wide-tile accumulation precision (#29611).
• Added and extended Intel subgroup-matrix MatMul/Gemm kernels, including f16, batched-B, and odd-N support (#29592, #29749, #29813, #29893).
• Reduced cold-start and upload overhead with deferred dispatch and staging-buffer improvements; tuned FlashAttention, subgroup Gemm/MatMul, Split-K on Panther Lake, and Xe im2col-matmul (#29271, #29505, #29557, #29586, #29846, #30514).
• Upgraded Dawn and improved reliability by avoiding exceptions in Dawn callbacks, fixing a Linux adapter-failure self-deadlock, correcting Windows x86 transfer callbacks, and fixing TurboQuant batched sequence lengths (#29389, #29591, #29625, #29752, #31568).

WebNN EP

• Added uint8-packed 4-bit GatherBlockQuantized and LpNormalization, reused the shared WASM loader for Blob-backed external data, and fixed per-axis QDQ and MatMulNBits edge cases (#29475, #29801, #31151, #31152, #31197).

OpenVINO / QNN / DML / XNNPACK / TensorRT

• OpenVINO fixed float16 constant-output corruption and output-name routing, added dot-separated KV-cache names to the stateful transform, and corrected raw-data-backed float initializer handling (#29729, #29882, #29895, #31138).
• QNN added a reshape handler for split-axis reshapes (#29660).
• DML fixed wide-string handling and made fused graph kernels own their model paths (#31656, #31664).
• XNNPACK now reads dynamic Gemm M from the input tensor at compute time (#31189).
• TensorRT deduplicated context-path handling and added a build option for fused-attention cubins (#29640, #31632).

CPU & Core Optimizations

MLAS

• Added Arm64 half-precision GEMM and convolution support through KleidiAI, including FP16 MatMul/Gemm/Conv paths and asymmetric Q4 and SME2 MatMulNBits kernels (#28786, #29654, #29709, #29898).
• Added a RISC-V RVV QNBitGemm backend, an Arm64 NEON fp32 RoPE kernel, portable SVE elementwise kernels with FEXPA exp, and Arm64 UDOT routing for S8U8 QGEMM (#29537, #29787, #29836, #31145).
• Added AVX2/VNNI 2-bit weight kernels and vectorized 2-bit dequantization, and improved fp16 MatMulNBits paths by avoiding fp32 temporaries and writing fp16 output directly across 2-, 4-, and 8-bit paths (#29619, #29766, #29791, #29842, #29864, #29901).

CPU Attention & Kernels

• Improved masked Attention performance, enabled CPU FlashAttention on Linux Arm64 through L2-cache detection, and added FP16 GQA with quantized KV cache (#29621, #29719, #29825).
• Added double support to CPU Cos and int32 support to CPU Trilu, and fixed int8 QLinearSoftmax saturation and AvgPool ceil_mode/count_include_pad behavior (#28975, #29476, #29629, #29728).
• Fixed TfIdfVectorizer weight indexing and skipped MinLength logits-processor construction when eos_token_id is negative (#29604, #31649).
• Tightened K/V and cache-indirection shape contracts in CPU Attention and MultiHeadAttention, and fixed LinearAttention output shape inference for grouped-query attention (#29892, #31190, #31634).

Graph, Optimizer, and Runtime

• Extended reshape fusion, fixed double recursion in subgraph type/shape inference, and made constant-folding output deterministic (#29027, #29617, #29789).
• Fixed in-memory external initializer loading, memory-pattern allocation stream selection, and a leak in GetOverridableInitializerNames() (#29349, #29589, #29616).
• Reduced small MatMul batch allocations and redundant LUT initialization (#29085, #29690).
• Fixed static-initialization-order crashes when importing ONNX Runtime and reduced eager runtime initialization (#29880, #31964).
• Negative CPU Split axes now produce an error instead of being accepted (#31149).

Web & JavaScript

• Added on-demand loading of Blob-backed external data in JSPI builds (#29477).
• Fixed JSEP pooling output shape for ceil_mode, allowed DFT to ignore excess input data, and fixed a webpack/Terser release-build crash (#29627, #29680, #31652).

Build, Packaging & CI

• CUDA package architecture selections are now aligned across plugin EP, Python, C API, TensorRT, and Node.js pipelines. Windows arm64 is only available in CUDA plugin EP (#31992):

  OS	CUDA	CUDA architectures (all in -real form)
  Linux x64	12.8	60;70;75;80;86;89;90a;120a
  Linux x64	13.x	75;80;86;89;90a;120a
  Linux aarch64	13.x	89;90a;120a;121a
  Windows x64	12.8	61;75;86;89;120a
  Windows x64	13.x	75;80;86;89;120a
  Windows arm64	13.x	120a;121a

• Reduced CUDA compilation time and memory usage by splitting generated SM80 MoE, fpA_intB, and MatMulNBits translation units and adding two-level workspace estimation (#29614, #29699, #29811, #31834, #31837).

• Fixed CUDA 13 plugin and packaging builds on Windows, Linux, and Windows ARM64, including MSVC/TMA compatibility and CI memory limits (#31608, #31609, #31615, #31616, #31617, #31622, #31729, #31748).

• Fixed MLAS AVX2 builds on toolchains without AVX-VNNI assembler support, GCC 15 -Werror builds, and an MSVC C1001 issue in the W2 AVX-512-VNNI dispatch path (#28767, #29679, #29885).

• Improved Windows compatibility by skipping DXGI discovery when Win32k system calls are unavailable and delay-loading shell32 (#29755, #30889).

• Fixed Dawn parallel-build races, and GPU discovery in build/test environments (#29858, #29866).

Contributors

Thanks to our 63 contributors for this release!

@adrastogi, @ahsan-ca, @AngelGalindo7, @ankitm3k, @apsonawane, @blazingphoenix7, @bmehta001, @chilo-ms, @claude, @daijh, @ducviet00, @edgchen1, @elwhyjay, @eserscor, @GopalakrishnanN, @guptaishaan, @hariharans29, @Honry, @huningxin, @jchen10, @jiafatom, @jiangzhuo, @Jiawei-Shao, @JonathanC-ARM, @justinchuby, @kjg0724, @kunal-vaishnavi, @kylo5aby, @Laan33, @martin-klacer-arm, @mastryukov1990, @mcollinswisc, @miaobin, @mingmingtasd, @mirounga, @mustjab, @n1harika, @namgyu-youn, @neilmsft, @nenad1002, @nicholascelestin, @OscarFree, @prathikr, @qjia7, @quic-muchhsu, @Sammy-Dabbas, @sanaa-hamel-microsoft, @shiyi9801, @skottmckay, @tairenpiao, @TedThemistokleous, @the0cp, @tianleiwu, @titaiwangms, @velonica0, @wangw-1991, @wuisabel-gif, @xadupre, @xhcao, @xiaofeihan1, @xiaoyu-work, @yen-shi, @zlma7001

Full Changelog: v1.28.0...v1.29.0