YOLOv5 benchmarking

GPU (Colab-NVIDIA T4)

Based on 5000 inference iterations after 50 iterations of warmups. Includes Image Preprocessing (letterboxing etc.), Model Inference and Output Postprocessing (NMS, Scale-Coords, etc.) time.

Backend	FPS
TensorRT	157-165
Torch	150-155
ONNX	92-96
TensorFlow	43-47

GPU (Colab-NVIDIA P4)

Based on 5000 inference iterations after 50 iterations of warmups. Includes Image Preprocessing (letterboxing etc.), Model Inference and Output Postprocessing (NMS, Scale-Coords, etc.) time.

Backend	FPS
TensorRT	82-86
Torch	65-69
ONNX	62-66
TensorFlow	39-42

CPU (Colab)

Note: We are performing more benchmarks, we will update info later on.

Based on 500 inference iterations after 10 iterations of warmups. Includes Image Preprocessing (letterboxing etc.), Model Inference and Output Postprocessing (NMS, Scale-Coords, etc.) time.

Backend	FPS
ONNX	5.4-7.6
TFLite	4.0-5.1
TensorFlow	3.5-4.2
Torch	3.2-4.0