FPS issue with movenet_lighning_int8 on docker container

[fabzit]

What happened?

Hi everyone, i'm trying to test the performance of different pose estimation models over a raspberry and over a docker container on pc. I measure FPS and Accuracy.
I have an issue on movenet_lightning_int8 on pc docker container. This model processed only 1 frame for second, worst than movenet thunder. Why?

Relevant code

pose_detector = Movenet('movenet_lightning8')
score_sum = 0
num_frames = 100 

# Start capturing video input from the file
cap = cv2.VideoCapture(cv2.samples.findFile(directory+input))
if not cap.isOpened():
    print('Unable to open: ' + input)
        exit(0)
# Start time
start_time = time.time()
# Grab a few frames
for i in range(0, num_frames) :
    success, image = cap.read()
    print(i) # with this model it prints once per second :S
    image.flags.writeable = False
    list_persons = [pose_detector.detect(image)]
    score_sum += list_persons[0].score
# End time
end_time = time.time()
    
# Release video
cap.release()
cv2.destroyAllWindows()

seconds = end_time-start_time
fps = int(num_frames / seconds)
mean_score = score_sum / num_frames

Relevant log output

the variable fps returns only 1.

tensorflow_hub Version

0.12.0 (latest stable release)

TensorFlow Version

2.8 (latest stable release)

Other libraries

No response

Python Version

3.x

OS

Linux

[singhniraj08]

@fabzit,

Could you try the cropping algorithm mentioned in official documentation and see if it enhances the FPS and prediction quality. Thank you!

[fabzit]

@fabzit,

Could you try the cropping algorithm mentioned in official documentation and see if it enhances the FPS and prediction quality. Thank you!

ok, i will try this, but the point is that with the same code, movenet thunder works with 15-16 fps and movenet Lightning int 8 works at only 1 fps. It's not normal, i think.

[singhniraj08]

@fabzit,

Are you trying to set the model up in TF Lite? Please refer to example tutorial here. Just click on Open Cloab Notebook.

Can you please share the minimal working code for us to debug the issue? Thank you!

[fabzit]

@singhniraj08 ,
You can see the code on Colab here. Unfortunately it doesn't work with thunder model and i don't know why, it gives this error when loading the model "# ValueError: Mmap of '57' at offset '0' failed with error '22'."
However the only part written by me is the final one.

On pc stock or in a docker container the thunder model returns much più fps then other model.

[singhniraj08]

@fabzit,

I was unable to run your code as the input video file didn't worked for me. I tried running the example notebook on TFHub site and observed that both lightning and thunder performs at ~4 FPS. Please find attached gist here.

As per this TF blog, this model achieves 30+ FPS by running the model completely client-side, in the browser using TensorFlow.js with no server calls needed after the initial page load and no dependencies to install.

@alenarepina, Can you please share some insights on why lightning performs at ~ 4 FPS while running on colab with GPU

Thank you!

[WGierke]

Closing due to inactivity.