Disclaimer: Please note that this project only allows for opening and closing each finger into a fist, as well as spreading them sideways. It does not support hand position tracking on its own. For hand position tracking, additional controllers or trackers are required. If you're using PCVR, you can try using a custom SteamVR driver that provides hand position tracking. When combined with this project, it enables both finger and hand tracking, making it compatible with full body tracking. You can watch a video demo that demonstrates the combination of these two software programs below.
Using a Leap Motion Controller from Ultraleap for finger tracking in VRChat.
minordaimyo-leapmotion-osc-example.mp4
Example of minordaimyo using leapmotion-osc 1.1.0 in combination with Driver Leap for both hand and finger tracking.
Example of parameter input by ThatGuyThimo.
This project uses OscCore and the OpenUPM Ultraleap Unity Plugin, to send finger tracking data from a Leap Motion Controller to VRChat via Open Sound Control.
Download and run the installer file of the latest release. If you want, you can also download the archive file marked as the windows build, unzip it and run the executable file it contains.
Download the archive file marked as the linux build, extract it and run the application with:
./LeapmotionOSC.x86_64This is not tested yet, please open a new issue if this does not work.
If you want to build the project yourself, you will need to clone this respository and install OscCore & Ultraleaps OpenUPM (Ultraleap Unity Plugin version 6.1.0). This repository includes a prefab that you can use to set up the latest configuration in your hierarchy. Unity version used for the current release is Unity 2021.3.16f1.
The tracking optimization for the controllers is set to HMD, using Edit Time Pose Head Mounted A. This application uses the standard VRChat OSC port 9000. If there is a discrepancy, you will need to create a new build, as there are no configuration options yet.
Your avatar receives OSC inputs for 20 float parameters that you need to add to your avatar. You also need to create or use the animation controller provided in the 1.1.0 release to create blendtrees that transition from the animations of each finger.
- leftThumb (float)
- leftIndex (float)
- leftMiddle (float)
- leftRing (float)
- leftPinky (float)
- rightThumb (float)
- rightIndex (float)
- rightMiddle (float)
- rightRing (float)
- rightPinky (float)
- leftThumbSpread (float)
- leftIndexSpread (float)
- leftMiddleSpread (float)
- leftRingSpread (float)
- leftPinkySpread (float)
- rightThumbSpread (float)
- rightIndexSpread (float)
- rightMiddleSpread (float)
- rightRingSpread (float)
- rightPinkySpread (float)
To do this, we created two layers in the gesture layer (one hand each) and added a custom bool parameter called "leapmotion" to turn OSC value playback on and off ingame. We recommend using a threshold of 0.4 to 1.0 for the blendtrees, as we have had the best results with this. Please note that the animations provided in the repo are customized to our avatars and have not been tested on other avatars yet. In the worst case you will have to set up the animations and blendtrees yourself. You will probably need to adjust the thresholds to fit your avatars hands correctly.
- Philipp initial project release.
- Ollie original NodeJS version.
- -Thimo- move to Unity standalone version, implementation of finger spreading. Repository owner.
- 2nd October 2023
- 1.1.0
- Added finger spreading thanks to -Thimo-.
- 1.0.3
- Moved to standalone Unity application.
- 1.0.2
- Added config file for server / client and leapjs settings. added oscRefreshRate as config variable, only send parameter when change happens.
- 1.0.1
- Split distance calculation and OSC sending to separate functions.
- 1.0.0
- Pull changes from Ollie, cleanup code, update license.
- 0.0.1
MIT License: Check the LICENSE file