Hand detection and gestures

[nicoeiris11]

Hi Vlad!

While testing hand features (in the web demo), I noticed that the MediaPipe detector handdetect.json result is always empty.
Do you mind checking?

I'd like to have your opinion regarding hand model alternatives in terms of accuracy and performance.

1. Default ones are your recommendation? Why is victordibia implementation the default one and not MediaPipe's?
2. I see in the model's wiki table you detail hand alternatives, do you mind explaining the difference between MediaPipe Hand Landmark, MediaPipe HandPose (HandDetect), and MediaPipe HandPose (HandSkeleton) models?
3. Is it possible to integrate MediaPipe's gesture recognition? I checked out their demo and works great. It'd be useful for my use cases to have good hand keypoint detection and gestures.

Thank you so much!

[vladmandic]

re: mediapipe - one is just hand box detector, second actually analyzes whats inside the box (actual keypoints) and third is combined one by mediapipe, but how they did the glue two models together uses proprietary ops from mediapipe which makes it a non-portable option.

why i'm not using mediapipe hand detector by default? because it behaves like it assumes there is hand in the frame. so if you try to run with hand not in frame, it will report false-positives high degree of confidence. its great if hand is visible, otherwise its horrible - and since human does not assume anything much be visible, that explains my choice.

now, why are you getting always empty results - i'll check that. can you open an issue so i can track it. a bit overloaded with other stuff at the moment and if its just a discussion item, it may get forgotten.

[nicoeiris11]

Thank you Vlad!

Here is the issue: #353

And what about integrating something like MediaPipe gesture recognition? Is this a non-portable option?

[nicoeiris11]

why i'm not using mediapipe hand detector by default? because it behaves like it assumes there is hand in the frame. so if you try to run with hand not in frame, it will report false-positives high degree of confidence. its great if hand is visible, otherwise its horrible and since human does not assume anything much be visible, that explains my choice.

How different is the MediaPipe hand model from human and this demo? In their demo I don't see any false positives but a very accurate detection of hands.

[vladmandic]

on a controlled input its very precise. on an uncontrolled one its horrible with false positives.