An easy-to-use public AI benchmark designed and developed by BenchmarkXPRT community

---

About

AIXPRT is an AI benchmark tool that makes it easier to evaluate a system's machine learning inference performance by running common image-classification, object-detection, and recommender system workloads.

AIXPRT includes support for the Intel OpenVINO, TensorFlow, and NVIDIA TensorRT toolkits to run image-classification and object-detection workloads with the ResNet-50 and SSD-MobileNet v1 networks, as well as a Wide and Deep recommender system workload with the Apache MXNet toolkit. The test reports FP32, FP16, and INT8 levels of precision. Test systems must be running Ubuntu 18.04 LTS or Windows 10, and the minimum CPU and GPU requirements vary by toolkit. You can find more detail on hardware and software requirements in the installation package's ReadMe files.

Support

SDK/Frameworks	OpenVINO	TensorRT	TensorFlow	MXNet
Hardware	Intel CPU, Intel GPU, Intel Neural Compute Stick 2, Intel Vision Accelerator	NVIDIA GPU, NVIDIA Xavier	AMD CPU, AMD GPU, Intel CPU, NVIDIA GPU, NVIDIA Xavier	AMD CPU, Intel CPU, NVIDIA GPU
OS	Ubuntu 18.04, Windows 10	Ubuntu 18.04, Windows 10	Ubuntu 18.04, Windows 10	Ubuntu18.04

Run the latest version of AIXPRT

Please use the package selector tool to download the appropriate one for your test system. A ReadMe file is provided along with each package with instructions for how to set up and run the benchmark.

Results

1. Below is how a snapshot of sample result summary. More details about the results can be found in the package's README.md file.

2. For already available results visit AIXPRT results page.
3. To submit results to our page, please follow these instructions.

Contribution guidelines

• Instructions for downloading the AIXPRT repository

The AIXPRT repository contains large files, over 50MB in size, so the package, git-lfs, must be installed and the repository must be cloned. (A zip file of the repository will not include the large files.)

Install git lfs and clone. Instructions are found at https://packagecloud.io/github/git-lfs/install and are listed in the following 3 steps

1. curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash
2. sudo apt-get install git-lfs
3. git clone https://github.com/BenchmarkXPRT/AIXPRT.git (You may need to enter your credentials for each large file)
4. For environment setup, follow the steps in the README.md file in the Modules/Deep-Learning/ directory for each platform

• Add a workload

1. Workloads on this git repository are grouped by framework. To begin, please pick a framework for the new workload. If it's a new framework, create a new folder with the framework name.
2. Follow the guidelines provided in this document to edit an existing workload or add a new one.
3. Once the workload is ready, contact BenchmarkXPRTsupport@principledtechnologies.com with your submission.

Resources

• Understanding AIXPRT batch size
• Understanding AIXPRT precision settings
• Understanding concurrent instances in AIXPRT
• Understanding AIXPRT results
• Navigating the AIXPRT results table

Licensing and legal information

For legal and licensing information, please see the following file:

• LICENSE