A 3D Mixed Reality Experience for the Microsoft HoloLense to learn sign language.
Learning a new sign is as easy as 1-2-3:
As in the video 1) the sign is demonstrated by the avatar, 2) demonstrated from your own perspective by the hands of the avatar and 3) repeated by the learner.
"Hello" gesture
YouTube videos can be misleading or error-prone when learning sign language because you always see the instructors hands back-to-front.
Therefore we created a mixed reality environment in which an avatar not only stands in front of you demonstrating the gesture, but also moves his arms exactly there where your arms should go. Now you can just repeat the movement the avatar did.
- ✅ Interaction with friendly avatar.
- ✅ Shows gestures from your own perspective.
- Future🔮: Checks your hand-position with a kinect or a laser to check whether you did it right and to correct your mistakes.
How the sign for "What's up?" is learned:
"What's up?" gesture
- Microsoft HoloLense
- Unity Engine
- Motion Capture