WebXR option to display hand mesh instead of controllers

[homanchou]

Is your feature request related to a problem? Please describe.

As a player of a first person shooter game, when playing WebXR immersive mode, I would like to see hand meshes representing my players hands even though i'm holding on to Quest Controllers, so that I can see my hands animate when I make a fist, point, wave hi at someone, grab onto objects, hold a gun, hold a sword, etc.

Describe the solution you'd like

I would like the default webXR experience to have an option for setting up default generic hands for me. useDefaultHands: true or something like that. I would like the mesh hands to automatically animate to "a pointing" finger if the grip button is down and the trigger button is up. Or a "fist" if both the trigger and the grip button are held down.

In terms of games, I think I might need the ability to dial in certain hand shapes when certain actions take place.

Idea is to be able to dictate: When hand mesh position is close to lever mesh position and grip button is down, then close the fist by 50% (like pulling down a lever).

Just a handful of stock poses might just cover 90% of all hand use cases in which the hands hold things (relaxed hand, fingers spread, closed fist, index finger point, gun grip). For example fingers spread wide can hold a softball. Then we just parent the hand to the thing we want to hold and dial in the pose to 30% fingers bent. Whereas holding a sword or a bat has a smaller diameter so we parent the hand to the object we want to hold then dial the pose down to 75% of a closed fist.

I would also like a clear API for adding additional animations or overriding a mesh with my own custom hand meshes.

Discussion
https://forum.babylonjs.com/t/are-there-any-existing-models-for-webxr-hands-available/14584/7

[RaananW]

Keeping it for future releases ATM, because i can't still estimate how many tasks I will finish by code freeze for 4.2.

[rohitatcosm]

Hello All,
Is there any update on this ? Can we have Hand Menu instead of controllers ?

[RaananW]

You can use the controllers on the hand menu as well (enabling the near-interactions feature). Or is your goal to show hands instead of controllers?

[rohitatcosm]

Hey Raanan, can u please share HandMenu Playground (with near-interaction feature on).. I am able to see hands instead of controllers but i want some kind of menu (3D GUI buttons or anything else) to open on hands gesture

[NickFernandez]

This would be really helpful for the baseball game I'm working on. I'd like the controllers to look like hands so that I could hold the ball and bat. The hand tracking feature is cool, but I feel like that would turn off certain people from playing.

[RaananW]

that's not an easy feature to implement, and IMO should actually come directly from the vendors. Otherwise we will need to have hand positions for each type of available controller. I don't remember seeing this feature suggested for the WebXR specs, but I will try researching some more

[thomlucc]

Require supports from vendors.