ONNX Runtime
Clone or download
skottmckay Simplify logic around creating relationship between nodes for implici…
…t NodeArg usage. Allows using an initializer from multiple levels up to not fail. We would need to accumulate a list of initializers from all levels up otherwise, and doing so doesn't add any value. (#200)

Improve a comment to clarify when the parent graph NodeArg lookup kicks in.
Latest commit beb326f Dec 18, 2018
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github/ISSUE_TEMPLATE Update issue templates (#62) Nov 29, 2018
cmake update onnx (#209) Dec 18, 2018
csharp Disable csharp pretrained tests temporarily (#207) Dec 18, 2018
dockerfiles Update Dockerfile for ARM build and the build instruction (#93) Dec 5, 2018
docs Miscellaneous fixes (#123) Dec 7, 2018
include/onnxruntime/core More C header naming changes (#202) Dec 18, 2018
onnxruntime Simplify logic around creating relationship between nodes for implici… Dec 18, 2018
package/rpm Rel 0.1.5 (#70) Dec 1, 2018
tools update onnx (#209) Dec 18, 2018
.clang-format Initial bootstrap commit. Nov 20, 2018
.clang-tidy Add remaining build options and make minor changes in documentation (#39 Nov 28, 2018
.gitattributes Initial bootstrap commit. Nov 20, 2018
.gitignore Update the documentation, run all examples during the generation of t… Dec 5, 2018
.gitmodules add gemmlowp as submodule. (#206) Dec 18, 2018
BUILD.md Update onnx_backend_test_series.py (#146) Dec 13, 2018
CODEOWNERS Fix codeowners file Nov 28, 2018
CONTRIBUTING.md Miscellaneous fixes (#123) Dec 7, 2018
LICENSE Initial bootstrap commit. Nov 20, 2018
README.md Reduce whitespace at top of readme for easier reading (#148) Dec 11, 2018
ThirdPartyNotices.txt Initial bootstrap commit. Nov 20, 2018
VERSION_NUMBER Rel 0.1.5 (#70) Dec 1, 2018
build.amd64.1411.bat Initial bootstrap commit. Nov 20, 2018
build.bat Initial bootstrap commit. Nov 20, 2018
build.sh Initial bootstrap commit. Nov 20, 2018
rename_manylinux.sh Bug bash (#43) Nov 28, 2018
requirements-dev.txt Initial bootstrap commit. Nov 20, 2018
requirements-doc.txt Update the documentation, run all examples during the generation of t… Dec 5, 2018
requirements.txt Initial bootstrap commit. Nov 20, 2018
setup.py Minor documentation changes (#78) Dec 3, 2018


Windows CPU Windows GPU Linux CPU Linux GPU MacOS CPU
Build Status Build Status Build Status Build Status Build Status

ONNX Runtime is an open-source scoring engine for Open Neural Network Exchange (ONNX) models.

ONNX is an open format for machine learning (ML) models that is supported by various ML and DNN frameworks and tools. This format makes it easier to interoperate between frameworks and to maximize the reach of your hardware optimization investments. Learn more about ONNX on https://onnx.ai or view the Github Repo.

Why use ONNX Runtime

ONNX Runtime is an open architecture that is continually evolving to adapt to and address the newest developments and challenges in AI and Deep Learning. We will keep ONNX Runtime up to date with the ONNX standard, supporting all ONNX releases with future compatibliity while maintaining backwards compatibility with prior releases.

ONNX Runtime continuously strives to provide top performance for a broad and growing number of usage scenarios in Machine Learning. Our investments focus on these 3 core areas:

  1. Run any ONNX model
  2. High performance
  3. Cross platform

Run any ONNX model

Alignment with ONNX Releases

ONNX Runtime provides comprehensive support of the ONNX spec and can be used to run all models based on ONNX v1.2.1 and higher. See ONNX version release details here.

As of November 2018, ONNX Runtime supports the latest released version of ONNX (1.3). Once 1.4 is released, ONNX Runtime will align with the updated spec, adding support for new operators and other capabilities.

Traditional ML support

ONNX Runtime fully supports the ONNX-ML profile of the ONNX spec for traditional ML scenarios.

High Performance

You can use ONNX Runtime with both CPU and GPU hardware. You can also plug in additional execution providers to ONNX Runtime. With many graph optimizations and various accelerators, ONNX Runtime can often provide lower latency and higher efficiency compared to other runtimes. This provides smoother end-to-end customer experiences and lower costs from improved machine utilization.

Currently ONNX Runtime supports CUDA and MKL-DNN (with option to build with MKL) for computation acceleration. To add an execution provider, please refer to this page.

We are continuously working to integrate new execution providers to provide improvements in latency and efficiency. We have ongoing collaborations to integrate the following with ONNX Runtime: * Intel MKL-DNN and nGraph * NVIDIA TensorRT

Cross Platform

ONNX Runtime offers:

  • APIs for Python, C#, and C (experimental)
  • Available for Linux, Windows, and Mac 

See API documentation and package installation instructions below.

Looking ahead: To broaden the reach of the runtime, we will continue investments to make ONNX Runtime available and compatible with more platforms. These include but are not limited to:

  • C# for Linux, Mac
  • C# supporting GPU
  • C packages
  • ARM

Getting Started

If you need a model:

  • Check out the ONNX Model Zoo for ready-to-use pre-trained models.
  • To get an ONNX model by exporting from various frameworks, see ONNX Tutorials.

If you already have an ONNX model, just install the runtime for your machine to try it out. One easy way to deploy the model on the cloud is by using Azure Machine Learning. See detailed instructions here.


APIs and Official Builds

API Documentation CPU package GPU package
Python Windows
C# Windows
Linux - Coming Soon
Mac - Coming Soon
Coming Soon
C (experimental) Coming Soon Coming Soon

Build Details

For details on the build configurations and information on how to create a build, see Build ONNX Runtime.


See more details on API and ABI Versioning and ONNX Compatibility in Versioning.

Design and Key Features

For an overview of the high level architecture and key decisions in the technical design of ONNX Runtime, see Engineering Design.

ONNX Runtime is built with an extensible design that makes it versatile to support a wide array of models with high performance.


We welcome your contributions! Please see the contribution guidelines.


For any feedback or to report a bug, please file a GitHub Issue.

Code of Conduct

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.


MIT License