Build voice-based conversational interfaces for AR/VR.
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.


Proof-of-concept Unity application using arbitrary voice commands to control the placement of objects on the screen.

Copyright (c) 2016 Aaron Faucher MIT License

NOTE has received a major overhaul since Facebook Messenger's bot release earlier this year. Currently, Wit3D uses the legacy API response format, in which JSON responses are parsed into intents and entities. I will be working in the coming months to reformat Wit3D to work with stories and actions, the new features. Until then, unless you have a pre-existing app using the legacy reponse format, Wit3D won't work for you. Sorry for the inconvenience, and stay tuned!


View a demo of Wit3D in action here:

Augmented reality proof-of-concept demonstration:


  1. Download and open the Unity project folder.
  2. Open the 02_More Objects scene.
  3. Run the scene.
  4. Voice a command to the scene by:
  • Press the spacebar
  • Voice your command i.e. "Put the box on the chair."
  • Release the spacebar

How it works

While the spacebar is being held, Unity records a sample of your voice from the microphone. On release of the spacebar, it streams this file to, where the command is processed and interpreted. returns a JSON response containing a 'subject' [i.e. box] and 'destination' [i.e. chair] from the command. This JSON response is parsed by Unity, which identifies the GameObjects by name in the scene.


While this is a relatively simple example, building a bridge between Unity and allows developers to leverage the powerful NLP engine for their AR/VR applications. End users can now interact with AR/VR scenes using their voice, with taking on the role of interpreting these commands.