This is a patch release on top of v1.30.0, containing targeted model-validation and memory-safety fixes, improved Plugin EP data transfers, and build and packaging fixes. Highlights Bug Fixes Hardened contrib-op shape inference with additional rank, attribute, initializer, and overflow validation for CausalConvWithState, GatedDeltaNet, GroupQueryAttention, and SparseAttention (#32607). Prevented o…
ONNX Runtime WebGPU Plugin EP 0.5.0 expands attention, mixture-of-experts, and quantized operator support, improves GPU kernel performance, and strengthens runtime integration and correctness. Highlights Performance Enabled subgroup-matrix kernels on AMD GPUs (#32667). Added an FP16 GEMV specialization for single-row decode projections (#33070). Improved data movement in Concat, Slice, Transpose,…
ONNX Runtime 1.31.1 is a patch release that fixes an OpenVINO-enabled Python build regression and corrects WASM Debug external-data tests. These notes cover changes since ONNX Runtime 1.31.0. Highlights Fixed a provider-info namespace mismatch that caused compilation failures when building with both OpenVINO and Python enabled (#33327). Fixed WASM Debug test failures by skipping file-backed extern…
ONNX Runtime 1.31.0 stabilizes model-package and EPContext data APIs, improves CPU model loading and quantized MoE inference, expands device-based execution-provider selection, and strengthens model-loading and runtime reliability. These notes cover changes since ONNX Runtime 1.30.0. CUDA and WebGPU kernel updates are summarized here; detailed provider notes are covered by their separate plugin EP…
This release brings sparse attention and packed continuous-batching support, INT2 and mixed-width QMoE execution, decode-kernel tuning, and reliability fixes to the ONNX Runtime CUDA Plugin EP. Highlights include new QSA/CSA attention operators, expanded paged speculative decoding, opt-in MatMul auto-tuning, and improved MXFP4 support on Blackwell consumer GPUs. Compatibility The minimum ONNX Runt…
This release brings expanded quantized inference and attention support, kernel performance improvements, and reliability fixes to the ONNX Runtime CUDA Plugin EP. Highlights include 2-bit MatMulNBits, INT4 paged KV caches, speculative decoding improvements, compact recurrent-state updates, and safer CUDA Graph execution across sessions. Please refer to the CUDA Plugin EP Quick Start for usage and…
This is a patch release on top of v1.28.2, containing memory-safety and model-validation fixes, a plugin execution fix, and packaging infrastructure updates. Highlights Bug Fixes Fixed an out-of-bounds read in EmbedLayerNormalization shape inference when beta is a scalar (#32610). Fixed a heap buffer overflow when reusing a packed sub-byte buffer for a full-byte tensor (#32611). Fixed an out-of-bo…
This release brings kernel performance improvements, expanded operator support, and reliability fixes to the ONNX Runtime WebGPU Plugin EP. Highlights Performance Optimized MatMulNBits wide-tile execution using subgroup shuffle (#31703) Supplied fused activation parameters to Conv/MatMul as uniforms, and fused eight more activations into Conv (#32116, #32117) Added fused activation support to the…
ONNX Runtime 1.30.0 expands generative AI inference, improves CPU and GPU performance, adds Go bindings, and strengthens runtime reliability. These notes cover changes since ONNX Runtime 1.29.1. Highlights Expanded CUDA inference support with variable-length causal convolution for continuous batching, speculative decoding in paged XQA, and INT4 paged KV caches with per-channel scales (#32168, #323…
This is a patch release on top of v1.29.0, containing GroupQueryAttention capability and KV-cache layout improvements, plugin Execution Provider performance tooling updates, and targeted graph and optimizer fixes. GroupQueryAttention Added bidirectional GroupQueryAttention support on CPU and CUDA through a backward-compatible causal attribute, with explicit handling for unsupported execution paths…
This is a patch release on top of v1.28.1, containing a targeted fix for Compile API model serialization. Highlights Bug Fixes Fixed Compile API callback serialization to prevent duplicate graph nodes, inputs, outputs, and value information in emitted optimized models, including models with embedded or external initializers (#32303) Contributors Thanks to our contributor for this release! @adrasto…
ONNX Runtime WebGPU Plugin EP 0.3.0 expands model and data-type coverage, improves generative-model performance, and strengthens configuration, reliability, and release tooling. These release notes were drafted with AI assistance. Highlights Model and operator coverage Added initial PagedAttention support, MRotaryEmbedding, GRU, DFT, PRelu, HardSwish, Trilu, Max and Min, and MatMulBnb4. (#31611, #…
This is a patch release on top of v1.28.0, containing support for device-free WebGPU compilation, improved compatibility with sandboxed Windows processes, and targeted graph-validation fixes. WebGPU EP Added support for device-free compile-only sessions, enabling offline graph transformation and optimized-model serialization without access to GPU hardware (#29681) Bug Fixes Prevented an access vio…
This is the first release of ONNX Runtime CUDA Plugin EP, providing CUDA execution as a separately packaged plugin execution provider. These notes cover commits affecting CUDA Plugin EP core code, CMake integration, and its primary build and package pipeline. Please refer to QUICK_START.md for the usage. Highlights Plugin Runtime Introduces the CUDA Plugin EP core and makes it the default CUDA pro…
Announcements & Breaking Changes onnxruntime-web has announced the deprecation of WebGL and JSEP. The native WebGPU EP is the recommended path going forward. See the deprecation and migration plans for details (#29716, #31683). POSIX telemetry is now available on Linux, macOS, Android, and iOS when ONNX Runtime is built with telemetry enabled. It does not change the public ABI, WebAssembly remains…
n.b. The following was generated via LLM from Git history. Only the contributor list has been verified. Announcement - Breaking Changes Support for CUDA 12 will be removed in 1.27.0. CUDA 13 will continue to be published as onnxruntime-<os>-<arch>-gpu_cuda13-<version>.<ext> CUDA runtime will be moving soon to a dedicated Execution Provider (EP) instead of a published package from ORT core. Highlig…
Highlights Major performance work for attention-heavy LLMs. FlashAttention decode kernels were fused and extended for any sequence length (#28389). FlashAttention prefill shared-memory path was generalized (#28520). Dynamic max_k_step was enabled for NVIDIA (#28511). QKV bias support was added for FlashAttention in MultiHeadAttention (#28380). M4 Max-specific FlashAttention optimization landed (#2…
Announcements & Breaking Changes Upgraded to ONNX 1.22.0 and protobuf 6.33.5 (#28754, #29606, #28967). Graph optimizer opset version checks were updated accordingly (#28966). cuDNN and cuFFT are now optional at runtime for the CUDA EP, and nvrtc is no longer linked, which significantly reduces the required CUDA redistributable footprint (#29252, #29808, #29705, #29620). An experimental C/C++ API s…
This is a patch release on top of v1.27.0, containing targeted bug fixes, a CUDA QMoE decode-path optimization, and CI/build infrastructure fixes. Bug Fixes [MLAS] Fixed an igemm regression in the KleidiAI path (#28571) Fixed a QMoE CPU livelock by eliminating nested intra-op parallelism (#29081) Fixed a regression in graph-capture session initialization that rejected an empty graph (#29457) Fixed…
n.b. This release is targeting ONNX 1.21. ONNX 1.22 will be supported in ORT 1.28. n.b. This changelog was generated via LLM. Only the contributor list has been verified. As always, only trust the commit history. Announcements & Breaking Changes CUDA 12 package files are now explicitly named as such. CUDA 12 packages are deprecated, please move to CUDA 13 ASAP. Security Fixes Fixed out-of-bounds r…
We're excited to ship the first release of the WebGPU Execution Provider as a plugin EP for ONNX Runtime. Instead of being baked into the core onnxruntime binary, the WebGPU EP is now distributed as a standalone artifact that registers with an existing ONNX Runtime installation at runtime. Highlights Broad operator coverage on WebGPU. Native WebGPU kernels for the operators needed by common transf…
n.b. This changelog is LLM generated. Only the contributor listing has been verified. ONNX Runtime Release 1.25.1 📢 Announcements & Breaking Changes ONNX Op Updates Enhanced ONNX operator support with new opset versions: Reshape (opset 25), Transpose (opset 24) (#27752) ✨ New Features 📊 New ONNX Ops & Model Support LinearAttention and CausalConvState operators for Qwen3.5 model support (#27907) Ro…
📢 Announcements & Breaking Changes Build & Platform C++20 is now required to build ONNX Runtime from source. Minimum toolchains: MSVC 19.29+, GCC 10+, Clang 10+. Users of prebuilt packages are unaffected. (#27178) CUDA minimum version raised to 12.0 — CUDA 11.x is no longer supported. Users pinned to CUDA 11.x should stay on ORT 1.24.x or upgrade their CUDA toolkit/driver. (#27570) ONNX upgraded t…
This is a patch release for ONNX Runtime 1.24, containing bug fixes and execution provider updates. Bug Fixes Core: Added PCI bus fallback for Linux GPU device discovery in containerized environments (e.g., AKS/Kubernetes) where nvidia-drm is not loaded but GPU PCI devices are still exposed via sysfs. (#27591) Plugin EP: Fixed null pointer dereference when iterating output spans in GetOutputIndex.…
This is a patch release for ONNX Runtime 1.24, containing bug fixes, security improvements, performance enhancements, and execution provider updates. Security Fixes Core: Fixed GatherCopyData integer truncation leading to heap out-of-bounds read/write. (#27444) Core: Fixed RoiAlign heap out-of-bounds read via unchecked batch_indices. (#27543) Core: Prevent heap OOB from maliciously crafted Lora Ad…
This is a patch release for ONNX Runtime 1.24, containing several bug fixes, security improvements, and execution provider updates. Bug Fixes NuGet: Fixed native library loading issues in the ONNX Runtime NuGet package on Linux and macOS. (#27266) macOS: Fixed Java support and Jar testing on macOS ARM64. (#27271) Core: Enable Robust Symlink Support for External Data for Huggingface Hub Cache. (#27…
📢 Announcements & Breaking Changes Platform Support Changes Python 3.10 wheels are no longer published — Please upgrade to Python 3.11+ Python 3.14 support added Free-threaded Python (PEP 703) — Added support for Python 3.13t and 3.14t in Linux (#26786) x86_64 binaries for macOS/iOS are no longer provided and minimum macOS is raised to 14.0 API Version ORT_API_VERSION updated to 24 (#26418) ✨ New…
ORT 1.23.2 cherrypick 1 (#26368) Adds the following commits to the release-1.23.2 branch for ORT 1.23.2: - [TensorRT] Fix DDS output bug during engine update - PR: #26272 - commit id: 00e85dd - Fix shape inference failure with in-memory external data - PR: #26263 - commit id: d955476 - [CUDA] replace 90a-virtual by 90-virtual for forward compatible - PR: #26230 - commit id: b58911f - [QNN-EP] Fix…