Unity augmented (mixed) reality application for Google Tango Lenovo Phab 2 Pro for exposure therapy with snake, spider and rat.
The developed augmented reality system is based on a Google Tango smartphone (Lenovo Phab 2 Pro). This smartphone is equipped with motion tracking and depth sensors. It allows the system to integrate depth information in a scene efficiently. The system reconstructs the surface in the scene including its illuminant condition and detects the ground plane of the scene and other planes as well with the support of Tango SDK. Then, a 3D model of a virtual animal adds to the reconstructed scene, and the scene with the animal renders using the Unity game engine. The rendered image superimposed on the video stream in real-time so that the image is seen by an observer using a VR (virtual reality) headset (e.g., Google cardboard).
We selected Vipera Nikolskii snake for ophidiophobia (fear of snakes) study.
List of animations (snakeall_anim):
- fr[0 - 74] - Bind Pose → Start Movement P.;
- fr[75 - 299] Movement;
- fr[300 - 374] Movement → IDLE
- fr[375 - 454] Tongue Movement 2 Times;
- fr[455 - 534] IDLE;
- fr[535 - 678] Tongue Movement 1 Time;
- fr[679 - 854] Response → Action;
- fr[855 - 1000]: IDLE → movement.
Selected spider is an Argiope bruennichi (wasp spider) female. We chose to use this spider because it is one of the most common spiders in Russia. Recognizing this spider in his habitation area is straightforward. Argiope bruennichi has a very bright wasp-like colour.
- file[ idle ]: IDLE;
- file[ hesitating ]: Hesitating;
- file[ fear_start ]: Defending Pose Start;
- file[ fear_interm ]: Defending Pose;
- file[ fear_end ]: Defending Pose End;
- file[ move_01_start ]: Movement start;
- file[ move_01_move ]: Movement;
- file[ move_01_end ]: Movement end.
We selected rat as one of the disgusting and frequently faced animal in Russia. For example, it is possible to meet a rat in the Moscow subway, near garbage cans, in cellars. Some people consider rats as potential disease carriers what cause fear and phobic avoidance of this animal.
List of animations (rat_60fps):
- fr[0 - 1] : Bind Pose;
- fr[2 - 160]: IDLE (Starts with Bind P. and ends with the same P.);
- fr[161-199]: Bind Pose → Stand;
- fr[200-318]: Itches;
- fr[319-355]: Stand P. → Bind Pose;
- fr[356-414]: Bind Pose → Movement;
- fr[415-490]: Movement;
- fr[491-520]: Movement → Bind Pose.
This application will work just on Lenovo Phab 2 Pro.
Be aware, before cloning you need intalled git-lfs.
The project was developed during BSc studies and we would like to thank our scientific supervisor Tadamasa Sawada and advisor Tomas Jurcik for their involvement in this project.
Nikita Sergeev | Andrew Skuratov Copyright 2018
The MIT licence does not apply to FBX animal models. All animal models were developed in Autodesk Maya Student Version and you must not use them for commercial purposes, check the link for details.