Skip to content
Switch branches/tags

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time



This project integrates Microsoft's LUIS API into Unity for Natural Language understanding in HoloLens applications. The HoloLens portion of the code was adapted from the HoloToolkit code from Microsoft The LUIS API in the sample folder is using the ALIEBot API ( that answers questions about the show The 100.


Katherine "Kat" Harris - Technical Evangelist at Microsoft. @KATVHARRIS - Twitter

Project Setup

To set up dictation and LUIS drag the revised DictationController (aka Communicator) and LUISManager_Hololens prefabs into your scene. They can be found as Prefabs in the Holograms folder.

Future Feature List

  • Create Inspector attribute to plug-in any LUIS library
  • Create Sample Scene Folder and Sample Scene

Technical Notes

A blog post about getting HoloLens Speech and Dictation working with LUIS can be found here:

A blog post about the project can be found here: Currently integrated with API for ALIEbot. The user can currently type request to get back infromation about a character. Next step is to use Voice Commands to Trigger LUIS calls.

07/06/16 The LUIS endpoint is working with direct ping from UnityWeb Request

07/05/16 Currently looking into Hololens and UAP Speech to Text API. Currently there is a listener to trigger certain commands in Hololens, but those are one word solutions, we want a more robust natural language processing to interact with a bot framework.

Acessing LUIS directly can also be an option - EXAMPLE: Making a HTTP request to the LUIS endpoint

This will return a JSON object that can be read for the next command.

07/03/16 Built Unity Project with the Hololens settings and tried integrating the Microsoft.Bot.Builder into the project. Errors occured: Microsoft.Bot.Builder 1.2.5 is not compatible with UAP,Version=v10.0. Potential Solution - Downgrade UAP version? (This might cause errors for Unity and Hololens) Potential Solution - Wait for Bot Builder to be upgraded Potential Solution 3 - Use the Node.js SDK to create a server for Unity to ping and access.

Currently is a simple bubble pop game.

Using the #WIN_UWP tag for UWP specific code, aka calling the bot framework. This tag spins off a seperate thread to handle UWP specific API calls. Will need to return from this thread to the Unity Thread if we want to implement Chain Dialogs.


  • Input - Gaze, Tap, Voice
  • Shared - Multiplayer interactions


Project for integrating Microsoft Bot framework into Unity for Natural Language understanding in HoloLens applications.







No releases published


No packages published