Skip to content


Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time
December 5, 2022 21:48
August 9, 2019 22:31

license CircleCI reference Forum Discord


This repo contains code for benchmarking deep learning solutions, including RedisAI. This code is based on a fork of work initially made public by TSBS at

Current DL solutions supported:

  • RedisAI: an AI serving engine for real-time applications built by Redis Labs and Tensorwerk, seamlessly plugged into ​Redis.
  • Nvidia Triton Inference Server: An open source inference serving software that lets teams deploy trained AI models from any framework (TensorFlow, TensorRT, PyTorch, ONNX Runtime, or a custom framework), from local storage or Google Cloud Platform or AWS S3 on any GPU- or CPU-based infrastructure.
  • TorchServe: built and maintained by Amazon Web Services (AWS) in collaboration with Facebook, TorchServe is available as part of the PyTorch open-source project.
  • Tensorflow Serving: a high-performance serving system, wrapping TensorFlow and maintained by Google.
  • Common REST API serving: a common DL production grade setup with Gunicorn (a Python WSGI HTTP server) communicating with Flask through a WSGI protocol, and using TensorFlow as the backend.

Current use cases

Currently, aibench supports two use cases:

  • creditcard-fraud [details here]: from Kaggle with the extension of reference data. This use-case aims to detect a fraudulent transaction based on anonymized credit card transactions and reference data.

  • vision-image-classification[details here]: an image-focused use-case that uses one network “backbone”: MobileNet V1, which can be considered as one of the standards by the AI community. To assess inference performance we’re recurring to COCO 2017 validation dataset (a large-scale object detection, segmentation, and captioning dataset).

Current DL solutions supported per use case:

Use case/Inference Server model RedisAI TensorFlow Serving Torch Serve Nvidia Triton Rest API
Vision Benchmark (CPU/GPU) (details) mobilenet-v1 (224_224) ✔️ Not supported Not supported ✔️ Not supported
Fraud Benchmark (CPU) (details) Non standard Kaggle Model with the extension of reference data ✔️ docs ✔️ docs ✔️ docs Not supported ✔️ docs


The easiest way to get and install the go benchmark programs is to use go get and then issuing make:

# Fetch aibench and its dependencies
go get
cd $GOPATH/src/


Blogs/White-papers that reference this tool


AIBench, a tool for comparing and evaluating AI serving solutions. forked from [tsbs]( and adapted to AI serving use case








No packages published