This project demonstrates how to use voice commands to interact with and navigate aneurysm data.
The project is optimized for use with the Valve Index for a complete experience. However, a debug version with limited features is available, allowing execution without a Head-Mounted Display (HMD).
- Clone the Git repository from GitHub — Voice-User-Interface
- Download the Docker repository from Zenodo — Zenodo Repository
- Download Unity Hub from Unity Hub
- You will need a Unity account to log in.
- Once logged in, navigate to the Projects tab and click Add.
- Select the Unity folder from the cloned Git repository.
- Unity will prompt you to install the correct version for this project (Unity 2019.4.40f1).
- Install the specified Unity version.
- Note: Installing Visual Studio is optional and only required if you plan to edit code.
- After the installation is complete, open the project through Unity Hub.
- Within the project, navigate to the Scenes folder and double-click on Navigation_Demo to load the scene displayed in the screenshot above.
-
Download and install Docker Desktop — Docker Desktop
-
Unzip the downloaded Docker repository.
-
Open a command prompt and navigate to the .../src directory within the unzipped Docker repository and run:
bash docker-compose up --build- Note: A running instance of Docker Desktop in the background is required to execute this command.
- This process may take some time as all necessary components are installed.
- Ensure Docker is running as described in the installation section.
- Open the Navigation_Demo scene in Unity, following the instructions in the installation section.
- In Unity, press Play in the Unity Editor.
- For users with a Valve Index HMD:
- Press the left-hand trigger to activate the microphone.
- For users without an HMD (debug version):
- Hold down the Spacebar to activate the microphone.
- While the microphone is activated, issue verbal commands to the system. For example:
- "Rotate the object 90° around the X-axis."
- Depending on the specifications of your system, commands may take 2 to 10 seconds to execute.
If you use our project in a scientific publication please use the following citation:
@article{hombeck2024voice, title={Voice user interfaces for effortless navigation in medical virtual reality environments}, author={Hombeck, Jan and Voigt, Henrik and Lawonn, Kai}, journal={Computers & Graphics}, volume={124}, pages={104069}, year={2024}, publisher={Elsevier} }