Highlights

Expanded the Python wheels into a C++ SDK, with linkable libraries for the
CUDA, Core ML, MLX, OpenVINO, and Qualcomm delegates and packaged TorchAO
kernels.
Expanded LLM serving with multi-method export, batched request scheduling,
bounded cancellation, off-graph KV-cache layouts, and new Qwen3.5 MoE, Muse
Glimmer, Supertonic, and Voxtral workflows.
Added ROCm support to the AOTI CUDA backend, including packed W4 execution
and end-to-end Voxtral Realtime coverage.
Expanded Arm and Cortex-M support with explicit-layout lowering and kernels,
additional Ethos-U targets and demos, broader VGF profiling, and more model
and operator coverage.
Expanded Qualcomm LLM
quantization-aware distillation, additional operators, Windows guidance, and
stronger validation and error handling.
Improved Vulkan quantized execution, dispatch safety, external-memory
integration, and operator coverage.

Runtime and Core

Added multi-method model support to the export pipeline and per-backend load
options.
Added capacity-aware memory planning over target memory maps and support for
shared allocations with offsets.
Added schema-version checks for PTE and PTD files and clearer failures for
invalid device-planned copies and missing delegate data.
Added shared pre-decomposition and graph-transform hooks, including common
view/permute propagation and reinplace passes.
Improved constant propagation, symbolic-shape handling, and device placement
for memory-planned buffers.

Added a C++ SDK to the Python wheel and split the runtime from the Python
bindings extension.
Added CUDA wheel publishing on Pytorch.org
Improved delegate/runtime bundling in the published wheels.
Added packaged TorchAO and quantized-kernel libraries.
Added Windows ARM64 build coverage.
Reduced wheel size and improved portability by stripping local macOS symbols,
removing unreachable vendored sources, and dropping unusable Linux MKL
search paths.
Lowered the supported CMake floor so CMake 3.26 through 3.28 builds work
again.

Arm

Added opt-in Cortex-M explicit-layout lowering with convolution and pooling kernels.
Added Arm export-recipe support, a pre-decomposition partitioner pipeline,
and broader symbolic-shape and dim-order handling.
Added Ethos-U65 support, Ethos-U85 KV-cache export, and SmolLM2,
TinyStories-42M, Silero VAD, MobileSAM, and Stable Diffusion workflows.
Expanded VGF profiling and environment preflight validation tools, neural statistics, automatic operator-support generation, support for int8 KV cache, FP16,
BF16, MXFP8, and quantized model coverage.
Improved Cortex-M memory planning, CMSIS-Pack coverage, in-place activation
fusion, and arithmetic, pooling, and activation lowering.

Qualcomm

Added LLM quantization-aware distillation and Gemma 4 text-model enablement.
Added support for pdist, sort, reflection_pad3d, as_strided,
scatter_reduce, scatter_add, and Hadamard-transform flows.
Added QNN-on-Windows guidance and improved SDK setup, import behavior,
delegate bounds checks, graph-I/O validation, and quantized metadata
preservation.

CUDA and ROCm

Added opt-in ROCm support to the AOTI backend, packed W4 execution, AMD
pointwise examples, and Voxtral Realtime coverage on gfx1100 and gfx950.
Added shared-memory targeting for cross-architecture CUDA inference and
INT4, INT5, and INT6 short-query kernels.
Improved CUDA delegate integration with per-thread streams, a warm memory
pool, cross-method weight sharing, and clearer missing-weight errors.

Core ML and MLX

Added linkable Core ML and MLX libraries to Apple frameworks and SwiftPM,
plus ETDump profiling support.
Added MLX off-graph KV-cache flat, ring, and cell layouts with shared pools
and cross-thread execution fixes.
Expanded MLX lowering with fused attention, native_group_norm,
upsample_nearest2d, flip, and trunc support.
Added Metal addmm and split_copy support and improved Core ML handling of
tied embeddings and causal attention masks.

Vulkan

Added external image and buffer wrapping with caller-provided dimensions.
Added device-safe dispatch geometry, two-dimensional dispatch helpers, and
quantized convolution paths using unsigned dot products.
Added payload-bounded constant sharding, scalar comparisons, and broader
arange, clamp, indexing, and batch support.

NXP

Upgraded the eIQ Neutron SDK to 3.2.1 and added an MCUXpresso example build.
Added Neutron support for rsqrt, Hardswish, softmax, reflected padding,
scalar elementary arguments, and AdaptiveAvgPool1d.
Added Neutron operator-suite and MLPerf Tiny classification coverage.

XNNPACK

Added BF16 fully connected delegation and BF16 activation support in LLM
Export.
Bumped the XNNPACK pin to 92a7ad501b9516f9fecae119e0146dd1f58e54c1 to support the above
Improved dynamic-quantized
channels-last handling.
Fixed pooling with default strides and single-element kernels, convolution
dimensionality inference, and shared-observer handling for slices.

Added a quantized MoE feed-forward operator with INT4 packing, fused
projections, source transformations, and numerical coverage.
Added a quantized stacked-halves RoPE operator and BF16-by-FP32 GEMV decode
support.
Optimized BF16 SDPA and fixed NaNs in CUDA split-K attention for Voxtral.
Added faster contiguous reductions and fixed non-contiguous reduction bounds,
broadcast rank mismatches, and dim-order validation for several operators.

Fixed CUDA low-memory external-weight export, model weight lookup, stream
selection, and cross-method AOTI library collisions.
Fixed QNN crashes on AMD hosts, invalid graph I/O, stale SDK bundles, and
setup side effects during import.
Fixed Arm convolution, pooling, rescale, grid-sampling, symbolic-shape, and
quantization edge cases across TOSA, VGF, and Ethos-U.
Fixed Vulkan delegation beyond supported graph regions and several dispatch,
resize, and quantization synchronization issues.
Fixed XNNPACK pooling, channels-last, placeholder-renaming, and BF16 export
issues.
Fixed Core ML and MLX attention, thread-affinity, and build-integration
issues.