Gesture recognition block

[Aki-07]

• Introduced an opt-in gesture recognition subsystem: developers can enable it via options.enableGestures() and configure providers, thresholds, and supported gestures.
• Implemented a reusable GestureRecognitionBlock that analyzes WebXR hand joints to detect pinch, open-palm, fist, thumbs-up, point, and spread gestures, emitting consistent start/update/end events other blocks can subscribe to.
• Updated the hands template and README to showcase the new gesture event stream, including simple visual feedback so teams can quickly validate the behavior.

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

[dli7319]

Code looks good! I'll test it out on my devices when I get into the office on Monday.

[Aki-07]

That works! @dli7319

[ruofeidu]

Thank you Akilesh!

@zhongyi-zhou for visibility and double check as well ;)

[zhongyi-zhou]

Very good way to detect gestures using heuristics! Thanks @Aki-07

[dli7319]

Hi Akilesh,

I tried it on Quest 3 today. Most of the gestures are working well!
However, for me personally, I couldn't get fist to trigger at all.
Additionally I had some reliability issues with the heuristic gesture:

1. When turning my hand, the thumbs up gesture gets triggered.
2. When making a fist, punch will get triggered.
3. Spread is a little bit difficult to trigger.

quest3_heuristic_gestures_xrblocks_20251020.mp4

For the template, I think it's hard to tell what's going on just based on the color of the cylinder, since a user wouldn't know the mapping between the gesture and the color and both hands affect the cylinder.
Could we make a separate template, e.g. heuristic_hand_gestures, and have some basic UI which shows all the detected gestures for each hand?

Some minor nits from me:

1. We can rename GestureRecognitionBlock to just GestureRecognition since no other classes use Block.
2. I see your intent to support different gesture providers. Since the gesture providers will need to use webxr hand joints, could we rename webxr to heuristics and move the heuristics code to a separate file outside of GestureRecognitionBlock?

[Aki-07]

Sure will get the comments resolved EOD tomorrow @dli7319

[Aki-07]

Swapped out the old GestureRecognitionBlock for a leaner GestureRecognition, pulled the heuristics into their own module (defaulting to heuristics), tuned the gesture scoring so fists/thumbs-up/point/spread aren’t nearly as flaky anymore, and refreshed the docs while adding a templates/heuristic_hand_gestures sample that shows a live dashboard for each hand, build still passes with npm run build, so holler if the Quest needs more tweaks. do review it @dli7319
Sorry got late on thisss

[dli7319]

Thanks for the refactoring and the new template.

We'll probably need to rework the template in a future PR since any HTML UI won't show up in WebXR.
For now can we have the template do a console.log whenever a gesture is started or stopped?

The heuristics are still somewhat buggy for me.
On quest 3:

1. Thumbs up still triggers when I'm not doing a thumbs up, and now does not trigger when I'm doing a thumbs up 😆.
2. Open-palm is always triggered
3. Fist is never triggered.

On the positive side, spread is easier for me to trigger now and pinch is still working well.

@ruofeidu Could you also test on Quest 3 to see if it's just me?

The same things happen in the simulator: the thumbs up gesture triggers with every prerecorded gesture except thumbs up.

Since we have a new template just for the heuristic gestures now, can we revert the hands template?
We want to keep the templates as simple as possible. We can add your new template to the docs site if you want.

For the main README.md file, could we move the ### Gesture recognition section out into a separate file, perhaps under docs/docs/manual/hand_gestures.md. You can leave the new bullet point in the list of features.

[ruofeidu]

Thanks! Please delegate this to Xun to test the gestures on Quest -- he should have a spare one or I'll share one in next weeks.

[Aki-07]

Tried tweaking a bit more. When you get a minute @dli7319 , please re-test on Quest 3 (and the simulator) to see if the confidence curves feel better, happy to keep tweaking if anything still acts up.

[dli7319]

The heuristics still need some improvement but we can get this submitted to get the structure in and update the heuristics in future PRs. Thanks for being an early contributor to XR Blocks!