Low GPU utilization

[ls0h]

Hi!

I'm trying to use MTCNN face detector from this example: https://github.com/onnx/models/blob/master/models/face_recognition/ArcFace/arcface_inference.ipynb
The detector works, but GPU usage is very low, only about 14%.
And it gives out only 6 FPS for FullHD image.

Why is the detector so slow? How to make it use 100% of GPU?

Here is my code:

import time
import cv2
import mxnet as mx
import mtcnn_detector
ctx = mx.gpu()
detector = mtcnn_detector.MtcnnDetector("path_to_model", ctx=ctx)
image = cv2.imread("1920x1080.png")
frame_id = 0
time_deltas: = [0 for _ in range(10)]
while True:
    time1 = time.time()
    detector.detect_face(image)
    time2 = time.time()
    td = time2 - time1
    time_deltas[frame_id % 10] = td
    if frame_id % 10 == 0:
        fps = 1 / (sum(time_deltas) / len(time_deltas))
        print(fps)
    frame_id += 1

My system:
GPU: 2 x GTX 1050Ti, CPU: Xeon E5-2660 v2 @ 2.20GHz, RAM: 32GB
Python: Python 3.6.5, mxnet-cu91mkl, Nvidia driver: 390.48
Ubuntu 18.04

[prasanthpul]

Have you tried using ONNX Runtime GPU to run the model?