[WIP] Implement Gestures

[pwab]

WIP proposal

---

Just some short points:

• For now we habe pinch and grab.
• There are some simple static gestures (static hand, only one hand and only finger positions are important e.g. thumbs up or showing numbers) and a lot of complex gestures (moving hands and arms combined with gesture transitions e.g. waving, counting, clapping etc.)
• Most of this is already researched and can be implemented with simple algorithms (based on angle calculations) to complex algorithms (based on machine learning).
• There already are a lot of simple implementations. For example: https://github.com/mahaveerverma/hand-gesture-recognition-opencv
• We could expose angle values and relative positions of the fingers to make it easier to detect for external analysis approaches or detect it in the leap motion itself and only send signals.
• Maybe also raycasts for pointing gestures with the fingers or arm (lower arm is tracked by leap motion isn't it?)

[BastiaanOlij]

I really hope we can find someone who is delving into this sort of stuff to contribute. I think a number of those aren't even restricted to leap motion but could be added to the godot-xr-tools library as more generic features.

I know Holger (Neospark) has been dealing with gestures with hand tracking on the quest, I'd advise having a chat with him about his experiences.

Are you on discord? The Godot Discord has an excellent active #XR channel where it would be good to discuss these things.

[pwab]

I really hope we can find someone who is delving into this sort of stuff to contribute.

Well this proposal is also written for me. I plan to implement this stuff somehow. At least in GDscript for the leap motion hand. But you are right that this could be implemented for a broader audience.

I know Holger (Neospark) has been dealing with gestures with hand tracking on the quest,

Thanks for the contact. Will return to this topic if the more important bullet collision problems are solved. I also know some guys that have implemented gestures for the Kinect and established a hand gestures catalogue for human-computer-interfaces.

Are you on discord?

Yes I was online two days ago. Will come back there soon.