What is the norminal processing speed of mediapipe's hand solution?

[aidengk-iris]

System information (Please provide as much relevant information as possible)

• Have I written custom code : Yes, custom code but only post processing on mediapipe's results. Mediapipe's hand solution is untouched.

• OS Platform and Distribution:

  Windows 10 Version 10.0.19044 Build 19044 for Intel based desktop.
  Windows 11 Pro Version 10.0.22000 Build 22000 for AMD based laptop

• MediaPipe version: 0.8.10.1

• Bazel version: Sorry, not sure how to check it. I pip installed the ready to use python package

• Solution : Hands, using CPU only.

• Programming Language and version: Python 3.9.12

Describe the expected behavior:

Dear mediapipe dev team and all,

This is not an issue but a question on the processing speed of hand solution.

I am using the hand solution with model complexity set to 1 in real-time with some processing of the landmarks and annotation on the video.
I timed mediapipe's processing on individual frames (1280x480 pixels) for detection of two hands and plotted 900 of them.

Timing was done on two computers

As you can see, the processing time is on average a little longer than the time between two camera frames (33.3 ms since the camera captures at 30 FPS)

Please note that I timed only mediapipe's processing time, no additional processing. You can see the code used for timing below

Processing time plot on i7 8809G @ 3.1GHz, 16G RAM, Win10

Processing time plot on AMD Ryzen 7 6800HS @ 3.2GHz, 16G RAM, Win11 Pro

The x-axis is the 900 processed frames.

Explaination of the plots: the spikes are likely re-detection of the palm.

At the beginning, no hand was seen by the camera, therefore the first 200 frames or less took less time to detect (because no palm detected).

After ~200 frames, my hand enters the camera's view, and the processing time increased obviously.
This makes sense as the palm detection and finger detection models are active.

Question: Is the processing time normal? I expect the inference to be faster.
If this is the limit, is there something I could do to speed up the inference?

Thank you much for your insights.
P.S. Mediapipe is a beutiful project. Thanks for making it happen!

Code used to time the processing speed

[sureshdagooglecom]

Hi @aidengk-iris ,
Models do take significant time to load, up to several seconds .
Therefore, 5 to 10 seconds may be expected on some devices.

[aidengk-iris]

Hi @sureshdagooglecom
Thanks for getting back to me. I was thinking of the time it takes for the hand solution to process one video frame, after loading the models.
I should have been more clear that the models are already loaded when I timed the processing time of a frame

[sureshdagooglecom]

Hi @aidengk-iris ,
Can we close this issue if the issue has resolved.

[aidengk-iris]

@sureshdagooglecom
Yeah I guess ~35ms is the best the hand solution can do at this moment. Let's close the issue.
I still look forward to the community's sharing of the processing time though.
Many thanks.

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No