Here you will find various samples, tutorials, and reference implementations for using ONNX Runtime. For a list of available dockerfiles and published images to help with getting started, see this page.

• Python
• C#
• C/C++
• Java

Inference only

• Basic Model Inferencing (single node Sigmoid) on CPU
• Model Inferencing (Resnet50) on CPU
• Model Inferencing on CPU using ONNX-Ecosystem Docker image
• Model Inferencing on CPU using ONNX Runtime Server (SSD Single Shot MultiBox Detector)
• Model Inferencing using NUPHAR Execution Provider

Inference with model conversion

• SKL Pipeline: Train, Convert, and Inference
• Keras: Convert and Inference

Inference and deploy through AzureML

• Inferencing on CPU using ONNX Model Zoo models:

  • Facial Expression Recognition
  • MNIST Handwritten Digits
  • Resnet50 Image Classification

• Inferencing on CPU with model conversion step for existing models:

  • TinyYolo

• Inferencing on CPU with PyTorch model training:

  • MNIST

  For aditional information on training in AzureML, please see AzureML Training Notebooks

• Inferencing on GPU with TensorRT Execution Provider (AKS)

  • FER+

Inference and Deploy wtih Azure IoT Edge

• Intel OpenVINO
• NVIDIA TensorRT on Jetson Nano (ARM64)
• ONNX Runtime with Azure ML

Other

• Running ONNX model tests
• Common Errors with explanations

• Inferencing Tutorial

• C - Inferencing (SqueezeNet)
• C++ - Inferencing (SqueezeNet)
• C++ - Inferencing (MNIST)

• Inference Tutorial
• MNIST inference