AIXPRT is an AI benchmark tool that makes it easier to evaluate a system's machine learning inference performance by running common image-classification, object-detection, and recommender system workloads.
AIXPRT includes support for the Intel OpenVINO, TensorFlow, and NVIDIA TensorRT toolkits to run image-classification and object-detection workloads with the ResNet-50 and SSD-MobileNet v1 networks, as well as a Wide and Deep recommender system workload with the Apache MXNet toolkit. The test reports FP32, FP16, and INT8 levels of precision. Test systems must be running Ubuntu 18.04 LTS or Windows 10, and the minimum CPU and GPU requirements vary by toolkit. You can find more detail on hardware and software requirements in the installation package's ReadMe files.
Intel Neural Compute Stick 2,
Intel Vision Accelerator
Run the latest version of AIXPRT
Please use the package selector tool to download the appropriate one for your test system. A ReadMe file is provided along with each package with instructions for how to set up and run the benchmark.
- Below is how a snapshot of sample result summary. More details about the results can be found in the package's README.md file.
- For already available results visit AIXPRT results page.
- To submit results to our page, please follow these instructions.
Instructions for downloading the AIXPRT repository
The AIXPRT repository contains large files, over 50MB in size, so the package, git-lfs, must be installed and the repository must be cloned. (A zip file of the repository will not include the large files.)
Install git lfs and clone. Instructions are found at https://packagecloud.io/github/git-lfs/install and are listed in the following 3 steps
- curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash
- sudo apt-get install git-lfs
- git clone https://github.com/BenchmarkXPRT/AIXPRT.git (You may need to enter your credentials for each large file)
- For environment setup, follow the steps in the README.md file in the Modules/Deep-Learning/ directory for each platform
Add a workload
- Workloads on this git repository are grouped by framework. To begin, please pick a framework for the new workload. If it's a new framework, create a new folder with the framework name.
- Follow the guidelines provided in this document to edit an existing workload or add a new one.
- Once the workload is ready, contact BenchmarkXPRTsupport@principledtechnologies.com with your submission.
- Understanding AIXPRT batch size
- Understanding AIXPRT precision settings
- Understanding concurrent instances in AIXPRT
- Understanding AIXPRT results
- Navigating the AIXPRT results table
Licensing and legal information
For legal and licensing information, please see the following file: