The easiest way to mocap your hands!
No artists had to suffer to make this Blender animation. It was all mocapped using a Meta Quest Pro!
We developed this tool in order to streamline the process of capturing hand movements from Meta Quest headsets and bringing them into Blender for use in animations.
- The hands in this concept video were mocapped using handy!
- The hands and face in this live demo were also mocapped using handy!
The code behind that live demo is available here, in case you are interested in how mocap data can be played in the browser.
- Unity 2022.1.23 or later
- Meta Quest (1, 2, or Pro)
- Windows or macOS
If you've never built and installed a Unity project on a Meta Quest headset, start by reading the beginner's guide. It will walk you through everything you need to do!
We wrote it so that people that don't have a technical background can run this project too.
We developed two workflows for this project. The Optimized Workflow is fast, but it requires an internet connection so that the Handy app can send mocap recordings from the headset to the Unity editor to automatically generate Alembic files. The Manual Workflow is slow, but it doesn't require an internet connection.
Click to expand
- First, build the
ClientSceneand install it to the headset. You can find it here:
To build and install it you can simply go to File -> Build And Run.
- Now, switch over to the
ServerScene- double-click it to open it, then hit play in the editor. You can find the scene here:
Play in the editor looks like this:
-
Run the
Handyapp on the headset. -
Start and stop recording by pinching your left thumb and index finger together and holding the pinch until the red recording indicator appears or disappears at your left wrist.
The red sphere at the left wrist indicates whether you are recording or not.
- Every time you stop recording, the client (the
Handyapp) will send a.jsonlinesfile to the server (the Unity editor). The server will then immediately start playing back the recording (you will see your hands moving in Unity's viewport). Once it finishes playing the recording (you will see your hands freeze in Unity's viewport), it will output a finished.abcfilename in Unity's console.
The .abc files will always be placed in a folder called data_output which is located at the root of your clone of this repository.
-
If you start and stop recording multiple times in a row, a queue will form in the server, so you will have to wait for it to process all your recordings.
-
Load your exported
.abcfiles in Blender. You will see two hands and a cube that acts as a placeholder for the headset, which we also record!
Click to expand
- First, build the
CaptureSceneand install it to the headset. You can find it here:
Double-click it to open it, and then go to File -> Build Settings..., select any scenes that are in the Scenes In Build box, right click them and select Remove Selection.
After that simply click the Add Open Scenes button and the CaptureScene should be added to the list. You are now ready to press the Build And Run button.
- Run the
Handyapp on the headset. - Start and stop recording by pinching your left thumb and index finger together and holding the pinch until the red recording indicator appears or disappears at your left wrist.
The red sphere at the left wrist indicates whether you are recording or not.
- Connect the headset to your computer and download the
.jsonlinesfiles that were recorded. You can find them here:
- Open the
PlaybackScenein the Unity editor by double-clicking it. You can find it here:
- Click on the
PlaybackManagerobject in the scene hierarchy. In thePlaybackManagercomponent of that object, input the path of the.jsonlinesfile that you want to export as an Alembic file. In theAlembicExportercomponent of the same object, specify the location where you want the Alembic file to be generated and its name.
- Hit play in the editor and wait for the animation to complete. Play in the editor looks like this:
- Load your exported
.abcfile in Blender. You will see two hands and a cube that acts as a placeholder for the headset, which we also record!
This project is licensed under the MIT License - see the LICENSE file for details.







