[Demo] [Related paper]
Web application for real time animation of a virtual actor. The character can be controlled using a webcam (default) or a gamepad.
This project is developed in WebGLStudio and uses the Beyond Reality Face (v4) library for facial landmark tracking.
tags: JavaScript, WebGL, Puppeteering, Facial retargeting, Animation, Embodied Virtual Actor, Gestures, Facial Expressivity, Virtual Characters, Real-time, 3D graphics
Before start, you must pre-initialize a neutral face. That means you have to put a neutral expression looking into the screen and press "Calibrate" button. You can also modify some thresholds if you active "Show Parameters".
The available expressions are represented in a 2d circumplex space (Valence-Arousal representation) as is shown in the following image. You can move in this space with the left joystick of the gamepad.
In the gamepad mode you can also activate the lip synchronization with your voice. For this, a microphone is required.