ONNX Runtime v1.28.1

This is a patch release on top of v1.28.0, containing support for device-free WebGPU compilation, improved compatibility with sandboxed Windows processes, and targeted graph-validation fixes.

WebGPU EP

• Added support for device-free compile-only sessions, enabling offline graph transformation and optimized-model serialization without access to GPU hardware (#29681)

Bug Fixes

• Prevented an access violation in Windows processes under Win32k lockdown by skipping DXGI device discovery (#29755)
• Allowed zero-input EPContext nodes, aligning their schema with support for compiling zero-input models (#29799)
• Hardened FastGelu fusion to skip malformed Mul and Pow patterns (#32016)
• Added validation for in-memory external initializer references, rejecting unregistered or mismatched data before graph transformation (#32042)

Contributors

Thanks to our 4 contributors for this release!

@apsonawane, @shiyi9801, @adrastogi, @mingmingtasd

Full Changelog: v1.28.0...v1.28.1

ONNX Runtime CUDA Plugin EP 0.1.0

This is the first release of ONNX Runtime CUDA Plugin EP, providing CUDA execution as a separately packaged plugin execution provider.

These notes cover commits affecting CUDA Plugin EP core code, CMake integration, and its primary build and package pipeline.

Please refer to QUICK_START.md for the usage.

Highlights

Plugin Runtime

• Introduces the CUDA Plugin EP core and makes it the default CUDA provider implementation (#27816, #29544).
• Adds arena allocation, resource accounting, available-resource reporting, and IOBinding synchronization (#27931, #28028, #28103, #27919).
• Adds provider options for user compute streams, copy behavior, EP-level unified streams, and external allocators, with compatibility back to ONNX Runtime 1.24.4 through version-gated callbacks (#28603, #28824).

CUDA Graphs and Profiling

• Adds CUDA Graph capture and replay, including user compute stream support, synchronization controls, and a kernel sync stream for scratch allocation (#27958, #28002, #28686, #29221, #29244).
• Adds plugin profiling APIs and memory statistics in profiling output (#28216, #29058).

Operators and Dependencies

• Expands model coverage with NHWC hardening, cuDNN SDPA for MHA/GQA, quantized MoE kernels, block-quantized FP4 MatMul, and native NVFP4 QMoE prefill on SM120 (#28612, #28849, #28467, #29818, #29824).
• Adds cuDNN-free ArgMax, ArgMin, and ReduceSum and fixes LogSoftmax for plugin deployments (#29620).
• Reduces runtime dependencies by making cuDNN and cuFFT optional and removing the nvrtc link requirement (#29252, #29808, #29705).

Build and Packaging

• Adds dedicated build and packaging pipelines, the onnxruntime-ep-cuda Python wheel, and NuGet packages (#27959, #28152, #28299, #28378).
• Adds Windows ARM64 packages, package-size controls, and aligned package metadata and artifact naming (#28896, #31635).
• Improves CUDA build coverage and reliability across CUDA 12.8/13, SM90, and SM120 while reducing CI build time with per-architecture object libraries and quick-build support (#29711, #28736, #29776, #31608, #31609, #28645).

Contributors

Thank all contributors that contributed to the CUDA execution provider:

@abudup, @aciddelgado, @AdamLouly, @adamreeve, @adrianlizarraga, @Alexey-Kamenev, @amancini-N, @amathews-amd, @AndreyOrb, @apsonawane, @ashbhandare, @askhade, @baijumeswani, @BowenBao, @chenfucn, @chengchen666, @chilo-ms, @cloudhan, @cocotdf, @codemzs, @cschreib-ibex, @dakenf, @decahedron1, @derdeljan-msft, @duli2012, @e-tao, @edgchen1, @elwhyjay, @eserscor, @fajin-corp, @feihugis, @frank-dong-ms, @fs-eire, @fthielke, @gedoensmax, @georgthegreat, @gh-yewang, @gineshidalgo99, @GopalakrishnanN, @gramalingam, @guoyu-wang, @guoyuhong, @hadiFute, @hanbitmyths, @hariharans29, @harrysummer, @harshithapv, @HectorSVC, @hubertlu-tw, @jagadish-amd, @jambayk, @jchen351, @jeffdaily, @jessebenson, @jiafatom, @jiangzhuo, @jignparm, @jingyanwangms, @jixiongdeng, @johannes-rehm-snkeos, @jslhcl, @JTischbein, @justinchuby, @jywu-mysoft, @kailums, @ke1337, @khspear, @kit1980, @KnightYao, @kpkbandi, @krahenbuhl, @kunal-vaishnavi, @Lafi7e, @linkerzhang, @liqunfu, @liuziyue, @lukas-folle-snkeos, @mastryukov1990, @mc-nv, @micmelesse, @mika-fischer, @mindest, @mityada, @mszhanyi, @mtavenrath, @mwootton, @namgyu-youn, @naomiOvad, @natke, @neginraoof, @nenad1002, @oliviajain, @PatriceVignola, @PeixuanZuo, @pengwa, @petermcaughan, @poweiw, @pranavsharma, @praneshgo, @prathikr, @qti-jkilpatrick, @qti-kromero, @RandySheriffH, @RandyShuai, @raviskolli, @Rishi-Dave, @RyanUnderhill, @satyajandhyala, @selenayang888, @seungtaek94, @shaahji, @shahasad, @sheetalarkadam, @SherlockNoMad, @ShirasawaSama, @skottmckay, @skyline75489, @smk2007, @snnn, @sohaibiftikhar, @souptc, @stevenlix, @Stonesjtu, @suffiank, [Vishal Agarwal (@thevishalagarwal)](https...

ONNX Runtime v1.29.0

Announcements & Breaking Changes

• onnxruntime-web has announced the deprecation of WebGL and JSEP. The native WebGPU EP is the recommended path going forward. See the deprecation and migration plans for details (#29716, #31683).
• POSIX telemetry is now available on Linux, macOS, Android, and iOS when ONNX Runtime is built with telemetry enabled. It does not change the public ABI, WebAssembly remains telemetry-free, and setting ORT_DISABLE_TELEMETRY=1 before initialization disables non-Windows telemetry for the process (#27379, #29872).
• The unused internal onnxruntime/python/tools/tensorrt dashboard tooling was removed. This does not affect the TensorRT Execution Provider APIs (#29395).

Security Fixes

Path, bounds, and input validation

• Fixed a path traversal vulnerability in TensorRT and NvTensorRTRTX engine refitting by making external-data path validation unconditional (#29396).
• Validated the CPU MoE k attribute against the number of experts and fixed a CPU TensorScatter security issue (#29907, #29916).
• Added missing rank, shape, and parameter validation for pooling, LSTM and DynamicQuantizeLSTM, Sampling, FeatureVectorizer, SkipLayerNorm, QLinearConv, Whisper decoding, RNN activations, GridSample, contrib Range, and CropAndResize (#29254, #29255, #29265, #29579, #29595, #29605, #29871, #31636, #31671, #31675, #31676, #31684).
• Hardened CUDA indexing and buffer handling in GridSample, transpose, GatherBlockQuantized, InstanceNormalization, LayerNorm/RMSNorm, BeamSearch, DeformConv, AveragePool, and MaxPool (#29581, #29631, #29638, #31640, #31642, #31644, #31645, #31647, #31650).
• Fixed packed sub-byte tensor over-copying in OrtApi::GetValue and validated DML constant tensor byte sizes (#29157, #31665).

Supply chain and tooling

• Updated npm lockfiles, refreshed the Next.js end-to-end fixture lockfile for security advisories, and upgraded adm-zip for onnxruntime-node (#29827, #29926, #31192).

New Features

Core APIs & Runtime

• Default intra-op and inter-op thread-pool sizes can now be set with ORT_INTRA_OP_NUM_THREADS and ORT_INTER_OP_NUM_THREADS. Explicit thread settings still take precedence, and 0 preserves machine-sized defaults (#29688).
• Added weightless-model support for all initializer types, allowed zero-input EpContext nodes, and wired maximum-shape inference into workspace estimation (#29607, #29799, #31613).
• Added ONNX-domain support for rotary embedding and a fused MRotaryEmbedding contrib operator for Qwen mRoPE variants (#29261, #31728).
• Added multi-shape profiling to onnxruntime_perf_test through --data_shape, plus verbose graph-transformer tracing and broader inference-session error-path coverage (#29555, #29558, #29569, #29571).

Execution Provider ABI & Plugin EPs

• WebGPU now supports device-free compile-only sessions for offline graph transformation (#29681).
• Expanded CUDA plugin EP packaging and testing, including Windows ARM64 package and size options, updated package outputs, and aligned architecture selections across Python, C API, TensorRT, Node.js, and plugin packages (#31635, #31722, #31992).
• Improved plugin lifecycle handling by unloading failed EP library loads and fixing allocator-deleter lifetime (#29634, #29770).

Execution Provider Updates

NVIDIA CUDA EP

Attention and decoding

• Added PagedAttention with quantized KV cache, XQA decode, MLA, QK-Norm, and head-sink support (#29912).
• Extended quantized KV-cache support with attention sinks, independent and per-channel scales, sliding-window cache support, and a fused K/V dequantization launch (#29900, #29904, #31480).
• Added a cuDNN SDPA decode tier to the standard ONNX Attention CUDA kernel and enabled cuDNN SDPA for contrib Attention (#29715, #29717).
• Added attention_bias support to the GroupQueryAttention unfused path and state_window support to LinearAttention and CausalConvWithState for MTP (#29525, #31157).
• Fixed LinearAttention on GPUs with limited shared memory (#31982).

MoE and quantized GEMM

• Added NVFP4 QMoE, including native FP4xFP4 prefill on SM120, faster decode GEMV, fused routing/finalization paths, and reduced activation and weight-dequantization overhead (#29697, #29824, #29887, #29919, #31156, #31159, #31349, #31479).
• Added MatMulBlockQuantizedFp4Weight and MatMulBlockQuantizedFp8Weight, plus block-scaled tensor-core/GEMV decode paths, packed FP4 decode, M-tiling, and folded W8A8 activation QDQ (#29818, #29850, #29896, #31155, #31481).
• Improved MatMulNBits and QMoE robustness and efficiency by optimizing 8-bit dequantization, releasing raw MXFP4 initializers after prepack, and fixing subgraph prepacking and mixed FP8/FP4 build failures (#29852, #31141, #31154, #31350).

Operators and collectives

• Added LinearAttentionGate, GatedRMSNorm, and GatedAdd contrib operators (#31158, #31835).
• Added bfloat16 support to AllReduce, AllGather, and AllToAll (#31571).
• Fixed the default zero point in CUDA GatherBlockQuantized (#31693).

WebGPU EP

• Added DFT, HardSwish, Max/Min, Trilu, GRU, PRelu, MatMulBnb4, and MRotaryEmbedding support (#29454, #29587, #29828, #29833, #29840, [#29845](https://github.com/microsoft/...

ONNX Runtime WebGPU Plugin EP v0.2.1

Highlights

• Major performance work for attention-heavy LLMs.

  • FlashAttention decode kernels were fused and extended for any sequence length (#28389).
  • FlashAttention prefill shared-memory path was generalized (#28520).
  • Dynamic max_k_step was enabled for NVIDIA (#28511).
  • QKV bias support was added for FlashAttention in MultiHeadAttention (#28380).
  • M4 Max-specific FlashAttention optimization landed (#27780).

• Qwen3 and Gemma 4 model-path improvements.

  • QKV and MLP fusions for Qwen3-style models (#28280).
  • Q/K RMSNorm fusion into GroupQueryAttention for Qwen3-style models (#28484).
  • Opset 24 and KV-shared decoder layer support for Gemma 4 (#28501).
  • GroupQueryAttention now supports optional present-key/value outputs (#28242).

• LinearAttention and quantized-path optimizations.

  • LinearAttention subgroup optimizations and larger tile_v with subgroup support (#28412, #28519).
  • GatherBlockQuantized gained 2-bit support (#28530).

• Reliability and hardening fixes.

  • Fixes for out-of-bounds read risks in GatherBlockQuantized and Pad (#28718, #28721).
  • Fix for past_state == present_state buffer handling (#28753).
  • Fixes in QMoE numerical stability and SkipSimplifiedLayerNormalization bias behavior (#28434, #28427).
  • ConvTranspose weight shape validation improvement (#28524).

• Graph-capture and buffer-management improvements.

  • Per-graph buffer manager and session-level buffer pool updates for graph capture reuse (#28260, #28761).

Note: This section was AI-generated. It may have inaccuracies.

Contributors

Thanks to everyone who contributed to the WebGPU EP (human contributors, alphabetical):

@apsonawane, @daijh, @edgchen1, @feich-ms, @GopalakrishnanN, @guschmue, @hariharans29, @HectorSVC, @jchen10, @qjia7, @tianleiwu, @xiaofeihan1, @xenova, @yuslepukhin.

Note: This list was compiled on a best-effort basis from PRs that touched WebGPU EP-specific paths and
intentionally includes human contributors only, so it may not capture every contribution. If yours was
missed, the omission is unintentional. Your work is no less appreciated.

ONNX Runtime v1.28.0

Announcements & Breaking Changes

• Upgraded to ONNX 1.22.0 and protobuf 6.33.5 (#28754, #29606, #28967). Graph optimizer opset version checks were updated accordingly (#28966).
• cuDNN and cuFFT are now optional at runtime for the CUDA EP, and nvrtc is no longer linked, which significantly reduces the required CUDA redistributable footprint (#29252, #29808, #29705, #29620).
• An experimental C/C++ API surface was introduced. OrtModelPackageApi now lives in the experimental C API and may change in future releases (#28746, #29142, #28990).
• Deprecated / removed:
  • SkipLayerNorm strict mode is deprecated (#29388).
  • The TensorRT fused causal attention kernels were removed from the CUDA EP (#29143).
  • The dynamic WGSL generator (duktape/Node) path was removed in favor of the Python wgsl-gen implementation (#29141, #28355).
  • CUDA_QUANT_PREPROCESS is off by default (#29687).
• NPM packages are now published from the CUDA 13 pipeline (#28773).
• The CUDA 12.8 package architecture list was refreshed for this release (#29711).

Security Fixes

Memory safety & input validation

• Hardened the ORT FlatBuffer model loader against malformed buffers, and removed now-redundant table offset validation (#28186, #29068)
• Fixed type confusion in raw-pointer bind_input causing an out-of-bounds write (#28839)
• Fixed out-of-bounds pointer in TensorAt for sub-byte packed types (#28973)
• Fixed arbitrary memory read, out-of-bounds dereference, and other OOB accesses in kernels (#28991, #29011, #29012, #29014)
• Validated Col2Im inputs to prevent heap over-read (#28706)
• Hardened CropAndResize against malformed crop_size tensors (#28766)
• Validated BeamSearch vocab_size against logits width (#28774)
• Fixed bounds in WhisperDecoderSubgraph::CreateInitialFeeds (#29239)
• Validated SparseAttention CSR indices/key lengths and rejected zero-dimension block_row_indices (#29015, #29242)
• Clamped derived sequence lengths and KV-cache index in CUDA GroupQueryAttention, and fixed a CPU GQA out-of-bounds read in the past-KV buffer (#29240, #29447)
• Clamped 1D attention mask_index to valid bounds (#29449)
• Validated MaxpoolWithMask kernel rank against input spatial rank (#29253)
• Rejected CUDA BERT EmbedLayerNorm/SkipLayerNorm shapes exceeding 32-bit output indexing (#29264)
• Fixed the optional-output guard in DecoderAttention/MultiHeadAttention shape inference and negative-axis handling in ExpandDims shape inference (#29268, #29448)
• Fixed TreeEnsemble target id validation and added input validation to LinearClassifier (#29293, #29060)
• Fixed DynamicQuantizeLSTM zero-point/scale validation typos (#29462)
• Handled non-trivially-copyable types in Loop/Scan output concatenation (#29397)
• Normalized bool tensor raw_data to {0, 1} on unpack (#29238)
• Addressed hardening gaps in Resize, PadFusion, and LoRA handling (#28779, #28780, #28801)
• Fixed unbounded lifetime on WithOutputTensor in the Rust bindings (#29251)

Integer overflow & allocation size

• Guarded MlasConvPrepare working-buffer products and ConvTranspose pad computation with SafeInt (#29444, #29446)
• Fixed signed-int overflow in SamplingState::Init that could cause a heap buffer overflow (#29443)
• Hardened QMoE against integer overflow and partial K tiles (#29067)
• Validated B/scales/zero-points shape in MatMulNBits::PrePack (#29445)
• Pre-checked ConstantOfShape output size against the input initializer before constant folding (#28751)
• Fixed integer overflow in RKNPU implicit bias allocation (#29249)
• Fixed WebGPU out-of-bounds reads in Pad (int64/int32 truncation), Slice, and GatherBlockQuantized (#28721, #28704, #28718)

Supply chain & tooling

• Updated protobuf to mitigate CVE-2026-0994 and bumped ONNX/protobuf to fix additional CVEs (#28967, #29606)
• Avoided shell injection in the training helper and switched Triton compile helpers to subprocess (#28776, #28775)
• Validated archive extraction paths in the transformers tooling (#28777)
• Validated and inlined external data in node tensor attributes during session initialization (#29250)
• Enabled Spectre-mitigated MSVC libraries for BinSkim builds (#29624)
• Bumped npm dependencies: shell-quote, esbuild, tmp, ws, protobufjs, js-yaml, tar, markdown-it, @babel/core (#29022, #29044, #29055, #29057, #29061, #29062, #29063, #29079, #29090, #29156)

New Features

Execution Provider ABI & Plugin EPs

• Model Package support Phase 2, plus authoring tools, schema versioning, and folding external_data into session options (#28271, #28989, #29501)
• Added an API to select the best compiled-model compatibility info from candidate strings (#28387)
• Added crypto support: applications can supply I/O callbacks to an EP, with callback and fallback helpers (#28624)
• Implemented name-based partitioning with accompanying documentation (#28903)
• Added Linux NPU discovery through sysfs accel devices (#28703)
• Relaxed CompileModel validation to accept zero-input OrtModel graphs (#28771)
• CUDA plugin EP: user compute stream with CUDA graph, kernel sync stream exposed for scratch allocation, and Windows ARM64 packages (#29221, [#29244](https://github.com/microsoft/onnxruntime/...

ONNX Runtime v1.27.1

This is a patch release on top of v1.27.0, containing targeted bug fixes, a CUDA QMoE decode-path optimization, and CI/build infrastructure fixes.

Bug Fixes

• [MLAS] Fixed an igemm regression in the KleidiAI path (#28571)
• Fixed a QMoE CPU livelock by eliminating nested intra-op parallelism (#29081)
• Fixed a regression in graph-capture session initialization that rejected an empty graph (#29457)
• Fixed CustomOp forward compatibility by capping the version instead of rejecting it (#29574)

Performance

NVIDIA CUDA EP

• Added a QMoE GEMV fast path for batch-1 decode (#29038)

CI & Build Infrastructure

• Fixed an incorrect identity for azcopy (#29274)
• Fixed a brew install applesimutils failure by trusting the wix/brew tap (#29450)
• Upgraded to Xcode 26 (#29468)
• Stopped echoing the command when setting a VSO variable in mac-cpu-packing-jobs.yml (#29575)
• Fixed the web e2e (npm/vite) and Python DML CI pipelines (#29609)

Contributors

Thanks to our 8 contributors for this release!

@tianleiwu, @chilo-ms, @edgchen1, @adrastogi, @damdoo01-arm, @JonathanC-ARM, @martin-klacer-arm, @sanaa-hamel-microsoft

Full Changelog: v1.27.0...v1.27.1

ONNX Runtime v1.27.0

n.b. This release is targeting ONNX 1.21. ONNX 1.22 will be supported in ORT 1.28.
n.b. This changelog was generated via LLM. Only the contributor list has been verified. As always, only trust the commit history.

Announcements & Breaking Changes

• CUDA 12 package files are now explicitly named as such.
• CUDA 12 packages are deprecated, please move to CUDA 13 ASAP.

---

Security Fixes

• Fixed out-of-bounds read in SoftmaxCrossEntropyLoss via label bounds validation (#28004)
• Hardened OneHot input validation and output-size computation (#28014)
• Added SafeInt overflow protection in Expand and capped constant-folding output sizes (#28055)
• Bounded total output allocation size in Tile kernel (#28070)
• Added mask/input shape consistency checks in MaxpoolWithMask::Compute (#28223)
• Fixed BitShift UB for shift amounts greater than or equal to bit width (#28272)
• Validated sequence bounds in GQA (seqlens_k vs cos_cache) (#28277)
• Validated conv bias shape in WordConvEmbedding to prevent OOB reads (#28279)
• Fixed int32 overflow in CUDA Cast and UnaryElementWise kernels for very large tensors (#28386)
• Fixed out-of-bounds read in CropBase scale handling (#28399)
• Fixed rank-underflow bug in Inverse kernel trailing-dimension indexing (#28400)
• Added sparse tensor external file path validation and additional external-path hardening (#28408, #28709, #28725)
• Switched remaining torch.load() calls to weights_only=True (#28421)
• Added CPU cache-indirection beam-index validation (#28486)
• Added additional overflow/bounds checks and test coverage in runtime buffers (#28713, #28747)

---

New Features

Execution Provider Plugin API

• Added zero-copy I/O for plugin EPs with HOST_ACCESSIBLE memory (#28037)
• Added OrtEp::OnSessionInitializationEnd() callback (#28319)
• Added plugin EP session-options getters (#28377)
• Added CUDA Plugin EP provider options for streams and external allocators (#28603)

Core APIs & Runtime

• Added support for ONNX overloaded functions (IR v10+) (#28275)
• Added FLOAT8E8M0 datatype support in ONNX Runtime (#28381)
• Added CPU Cast support for FLOAT8E8M0 (#28435)
• Added kOrtEpDevice_EpMetadataKey_OSDriverVersion example and docs (#28282)

Quantization & Training Tooling

• Added calibration cache support to quantize_static (#28221)
• Added ActivationRestrictedAsymmetric quantization option (#28237)
• Added opset-21 block_size attribute support to QDQ quantization (#28522)
• Added CPU fallback for FusedAdam optimizer in ORT Training (#28233)

---

Execution Provider Updates

NVIDIA CUDA EP

• Added ConvTranspose-22 support (#27710)
• Filled CUDA opset gaps for LSTM, RNN, Reshape, Cast, Round/Equal, ReduceMax/ReduceMin, Sin/Cos, and Random* ops (#27737, #27743, #27742, #27744, #27754, #27755, #27756, #27759)
• Added LpNormalization support for CUDA EP (#28724)
• Added chunked dequant+GEMM for MatMulNBits to reduce peak GPU memory (#28712)
• Added QMoE tests for standard swiglu and improved decode-path routing/softmax kernels (#28741, #29026)
• Fixed CUDA Attention dispatch mismatch for GQA head-size cases (#28358)
• Fixed CUTLASS FMHA bias-loader alignment on unaligned kernel path (#28369)

WebGPU EP

• Added LSTM support on WebGPU (#27881)
• Added per-graph buffer manager for multi-graph capture (#28260)
• Added QKV and MLP layer fusions for Qwen3-style models (#28280)
• Added QKV bias support in FlashAttention for MultiHeadAttention (#28380)
• Added shader dump-to-file environment variable and nightly validation checks (#28674)
• Added opset-24 + KV-shared decoder support (Gemma 4) (#28501)
• Performance improvements: FlashAttention M4 Max optimization and subgroup-based LinearAttention tuning (#27780, #28412, #28519, #28520)
• Fixed numerical and correctness issues in QMoE, LayerNorm/SkipLayerNorm, and MatMul bias indexing (#28427, #28434, #28475)

CoreML EP

• Added support for pre-opset-13 Split (split attribute path) and scalar Gather indices (#28270, #28278)
• Added FusedConv, Identity, Ceil, Tile, Cast(bool), Sin, Cos, and GatherND support (#28289, #28293, #28595, #28596, #28598)

TensorRT / DML / QNN

• Improved TensorRT RTX compatibility (multi-GPU tests, API guards, and subgraph fixes) (#27837, #28361, #28611, #28503)
• Added diagnostics for DML failure paths (#28495)
• Updated QNN ETW log level rule handling (#27593)

---

Web & JavaScript

JavaScript / Node.js

• Updated JavaScript dependencies (next, postcss, tmp, qs, body-parser, and other npm packages) (#27705, #27894, #28304, #28547, #28644, #28683, [#28694]

---

CPU & Core Optimizations

MLAS / Quantization / Attention

• Added NHWC convolution path in MLAS to reduce transpose overhead (#26834)
• Added CPU QMoE 2-bit support and LUT GEMM fast path (#28185)
• Added quantized KV-cache support for CPU GroupQueryAttention with SIMD optimizations and tiled compute (#28576, #28578, #28606, #28695)
• Added RVV-optimized NCHWc convolution/pooling and LLM operators for RISC-V (#28411, #28518)
• Paralleli...

ONNX Runtime WebGPU Plugin EP v0.1.0

We're excited to ship the first release of the WebGPU Execution Provider as a plugin EP for ONNX Runtime. Instead of being baked into the core onnxruntime binary, the WebGPU EP is now distributed as a standalone artifact that registers with an existing ONNX Runtime installation at runtime.

Highlights

• Broad operator coverage on WebGPU. Native WebGPU kernels for the operators needed by common transformer, vision, and generative workloads — including Conv variants, MatMul/Gemm, normalizations, attention (Attention, MultiHeadAttention, GroupQueryAttention), rotary embeddings, quantized matmul, quantized Mixture-of-Experts (QMoE), and more. See the Operator coverage section below for a summary.
• Quantized & accelerated kernels. DP4A and subgroup-matrix MatMulNBits, a FlashAttention kernel, and vendor-optimized Intel MatMul/Gemm paths. See the Performance features section below.
• Plugin EP packaging. WebGPU support now ships as a separate, independently versioned library (onnxruntime_providers_webgpu) that plugs into a compatible ONNX Runtime (1.24.4 or newer) at runtime. Users can adopt WebGPU acceleration without switching their core ORT package, and the EP can iterate on its own cadence.
• Cross-platform native binaries for Windows x64/arm64 (bundled with dxil.dll / dxcompiler.dll), Linux x64, and macOS arm64.
• Language packages.
  • Python: onnxruntime-ep-webgpu wheel, installed alongside the onnxruntime package, registered via onnxruntime.register_execution_provider_library(...). See package page for details on installation and usage.
  • .NET: Microsoft.ML.OnnxRuntime.EP.WebGpu NuGet package, referenced alongside Microsoft.ML.OnnxRuntime, registered via OrtEnv.RegisterExecutionProviderLibrary(...). See package page for details on installation and usage.

Operator coverage

The WebGPU EP registers kernels for the majority of ONNX standard-domain operators used by mainstream model architectures, plus a curated set of com.microsoft contrib operators. Highlights by category:

• Math, normalization & reduction: MatMul, Gemm, Softmax, LayerNormalization, RMSNormalization, InstanceNormalization, BatchNormalization, LpNormalization, unary/binary elementwise ops, all standard reductions (ReduceMean, ReduceSum, ReduceMax, ...), CumSum, Einsum, TopK, ArgMax/ArgMin.
• Neural network: Conv, ConvTranspose, MaxPool/AveragePool (and Global* variants), plus a FusedConv contrib op.
• Tensor manipulation: Transpose, Reshape, Slice, Concat, Split, Gather/GatherElements/GatherND, ScatterElements/ScatterND, Pad, Tile, Cast, Resize, GridSample, Where, Flatten, Squeeze, Identity, Shape, and more.
• Transformer / LLM contrib ops: Attention, MultiHeadAttention, GroupQueryAttention, RotaryEmbedding, SkipLayerNormalization, SkipSimplifiedLayerNormalization, SimplifiedLayerNormalization, BiasAdd, BiasGelu, BiasSplitGelu, FastGelu, Gelu, QuickGelu, CausalConvWithState, LinearAttention.
• Quantization: DequantizeLinear, MatMulNBits (with DP4A and subgroup-matrix paths), GatherBlockQuantized, QMoE.

For the authoritative list, see the kernel registrations in webgpu_execution_provider.cc and webgpu_contrib_kernels.cc.

Performance features

• DP4A and subgroup-matrix MatMulNBits paths for accelerated quantized matmul on supported hardware.
• FlashAttention kernel for attention-heavy workloads.
• Intel-optimized MatMul/Gemm code paths for improved performance on Intel GPUs.
• Program caching to amortize shader compilation costs across runs.
• Optional PIX frame capture and WebGPU profiler integration for performance investigation.

Known limitations

• Platform support in this release is limited to the platforms listed above (no mobile, no Linux arm64, no macOS x64).

Acknowledgments

This initial release is the result of contributions from engineers at Microsoft, Intel, and the broader community. Thank you to everyone who built, reviewed, and tested the WebGPU plugin EP — including (in alphabetical order):

@aciddelgado, @adrastogi, @adrianlizarraga, @chilo-ms, @daijh, @derdeljan-msft, @edgchen1, @eserscor, @feich-ms, @fs-eire, @guschmue, @HectorSVC, @ingyukoh, @jchen10, @jiangzhaoming, @Jiawei-Shao, @jing-bao, @justinchuby, @kunal-vaishnavi, @mindest, @prathikr, @qjia7, @satyajandhyala, @shaoboyan091, @sheetalarkadam, @skottmckay, @snnn, @sushraja-msft, @tianleiwu, @titaiwangms, @TomCrypto, @vraspar, @wenqinI, @xenova, @xhcao, @xiaofeihan1, @yuslepukhin.

Special thanks to the Intel team for the vendor-optimized MatMul/Gemm kernels.

Note: This list was compiled on a best-effort basis from PRs that touched WebGPU EP-specific paths, so it may not capture every contribution. If yours was missed, the omission is unintentional — your work is no less appreciated.

ONNX Runtime v1.26.0

n.b. The following was generated via LLM from Git history. Only the contributor list has been verified.

Announcement - Breaking Changes

• Support for CUDA 12 will be removed in 1.27.0.
  • CUDA 13 will continue to be published as onnxruntime-<os>-<arch>-gpu_cuda13-<version>.<ext>
• CUDA runtime will be moving soon to a dedicated Execution Provider (EP) instead of a published package from ORT core.

Highlights

• Added optional memory mapping for .ort model loads (#28164).
• Added RISC-V Vector (RVV) support for CPU EP (#28261).
• OpenVINO EP upgraded for 1.26.0 development release (#28297).
• WebGPU gained GridSample support (#28264) and Split-K improvements (#28151).
• CUDA plugin EP gained graph support (#28002), profiling API (#28216).

Security and Reliability Hardening

• Replaced unrestricted Python setattr configuration with an allowlist (#28083).
• Hardened multiple OOB and overflow scenarios across ML and core ops:
  • Attention mask index OOB write (#27789).
  • MaxPoolGrad indices bounds validation (#27903).
  • SVM and TreeEnsemble bounds/security fixes (#27950, #27951, #27952, #27989).
  • RNN sequence_lens OOB read and integer overflow handling (#28052, #28003).
  • GroupQueryAttention seqlens_k bounds validation and compatibility follow-up (#28031, #28259).
  • MatMulBnb4 and ML coefficient SafeInt checks (#27995, #28001).
  • CUDA Gather int32 overflow fix (#28108).
  • GridSample float->int64 cast hardening for NaN/Inf/out-of-range coords (#28302).
• Fixed session logger use-after-free during EP teardown under verbose logging (#28274).

CUDA, Attention, and MLAS

• Filled CUDA opset/operator gaps and extended support:
  • Transpose opset 23 -> 25 (#27740).
  • QuantizeLinear/DequantizeLinear opset 25 (#28046).
  • CUDA TopK INT8/INT16/UINT8 support (#27862).
  • LabelEncoder CUDA support for numeric types (#28045).
• Attention/GQA improvements:
  • Fixed ONNX Attention min-bias alignment crash on SM<80 and masked-batch NaN behavior (#27831).
  • Added FP32 QK accumulation path for unfused GQA attention (#28198).
  • Added CUDART_VERSION reduction compatibility in GQA attention (#28296).
  • Fixed CUDA 13 build error in GQA unfused attention (#28309).
  • PagedAttention fallback for SM<80 fp16 (#28200).
• MLAS updates:
  • FP16 Gelu enablement (#26815).
  • Arm64 BF16 fast-math conv kernels for NCHW/NCHWc paths (#27878).

WebGPU, WebNN, and JavaScript

• WebGPU feature and correctness updates:
  • Added GridSample (#28264).
  • Split-K support for batch size > 1 (#28151).
  • MatMulNBits refactor and batching improvements (#28109, #28197).
  • MHA correctness fix when present outputs are not requested (#28027).
  • Buffer upload overflow fix (#27948).
  • Position ID bounds validation in WebGPU/JS RotaryEmbedding (#28214).
• WebNN change:
  • Renamed pool2d property roundingType -> outputShapeRounding (#28172).
• JavaScript ecosystem maintenance:
  • Multiple dependency bumps.

Plugin EP and EP Ecosystem

• CUDA plugin EP:
  • Graph capture/replay support ported and expanded (#27958, #28002).
  • Sync support for IOBinding (#27919).
  • Profiling API implementation (#28216).
  • Resource accounting integration (#28028).
• WebGPU plugin EP:
  • Pipeline updates and API init error handling fixes (#28121, #28211).
• Other EP updates:
  • CoreML: HardSigmoid and QuickGelu support; Pad reflect support/fixes (#28182, #28184, #28073, #28062).
  • NvTensorRTRTX compatibility and diagnostics updates (#28263, #27577).
  • QNN file-mapping guard improvements (#27871).

Contributors

@tianleiwu, @yuslepukhin, @edgchen1, @vraspar, @hariharans29, @skottmckay, @eserscor, @xadupre, @sanaa-hamel-microsoft, @elwhyjay, @Rishi-Dave, @titaiwangms, @adrianlizarraga, @jatinwadhwa921, @jchen10, @Jiawei-Shao, @maxwbuckley, @preetha-intel, @qjia7, @qti-hungjuiw, @RajeevSekar, @umangb-09, @adrastogi, @akote123, @amd-genmingz, @ankitm3k, @apsonawane, @bachelor-dou, @baijumeswani, @bopeng1234, @chilo-ms, @chwarr, @Craigacp, @dccarmo, @derdeljan-msft, @ericcraw, @fdwr, @fs-eire, @gaugarg-nv, @gblong1, @GopalakrishnanN, @Honry, @intbf, @ishwar-raut1, @Jaswanth51, @javier-intel, @JonathanC-ARM, @julia-thorn, @justinchuby, @jwludzik, @Kevin-Taha, @Kotomi-Du, @MayureshV1, @mdvoretc-intel, @miaobin, @milpuz01, @mingyueliuh, @mklimenk, @n1harika, @prathikr, @psakhamoori, @qti-yuduo, @quic-calvnguy, @RyanMetcalfeInt8, @sfatimar, @sgbihu, @ShirasawaSama, @ssam18, @susbhere, @sushraja-msft, @TejalKhade28, [Thomas Hühn (@th)...

ONNX Runtime v1.25.1

n.b. This changelog is LLM generated. Only the contributor listing has been verified.

ONNX Runtime Release 1.25.1

📢 Announcements & Breaking Changes

ONNX Op Updates

• Enhanced ONNX operator support with new opset versions: Reshape (opset 25), Transpose (opset 24) (#27752)

---

✨ New Features

📊 New ONNX Ops & Model Support

• LinearAttention and CausalConvState operators for Qwen3.5 model support (#27907)
• RotaryEmbedding (RotEMB) and RMSNorm operators added (#27752)
• Linear Attention signature support (#27842)

---

🌐 Web & JavaScript

WebGPU EP

• Qwen3.5 model support on WebGPU execution provider (#27996)
• QMoE 1-token decode path optimization — fused operations to reduce GPU dispatches for improved performance (#27998)

---

🐛 Bug Fixes

Core Runtime Fixes

• Improved filesystem error messages during Linux device discovery for better debugging experience (#27289)
• Fixed missing include for SetRawDataInTensorProto in NVIDIA TensorRT RTX tests (#28065)

---

🙏 Contributors

Thanks to our 7 contributors for this release:
@guschmue, @sanaa-hamel-microsoft, @apsonawane, @eserscor, @ishwar-raut1, @qjia7, @theHamsta

Full Changelog: v1.25.0...v1.25.1