A sample Unity project for Microsoft HoloLens that lets you interact with a floating robot head using speech recognition, and getting answers spoken back by the "bot" using Text-to-Speech. The commands sent to the bot are powered by the Microsoft Bot Framework and LUIS from Microsoft Cognitive Services.
Switch branches/tags
Nothing to show
Clone or download
Pull request Compare This branch is 1 commit ahead, 40 commits behind ActiveNick:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
Assets
ProjectSettings
Screenshots
UWP
.gitignore
Assembly-CSharp-Editor.csproj
Assembly-CSharp.csproj
HoloBot.CSharp.Editor.csproj
HoloBot.CSharp.csproj
HoloBot.sln
README.md

README.md

HoloBot

A sample Unity project for Microsoft HoloLens that lets you interact with a floating robot head using speech recognition, and getting answers spoken back by the "bot" using Text-to-Speech. The commands sent to the bot are powered by the Microsoft Bot Framework and LUIS from Microsoft Cognitive Services.

To get started with HoloLens, visit the Windows Holographic Dev Center.

Features

  • Hovering bot head (aka HoloBot) with looping sound
  • Tap the HoloBot to move it to a different location, tap again to place
  • Gaze at the HoloBot to trigger the speech recognizer, you will hear a "Ping" sound
  • Speak "commands" using natural language
  • The HoloBot speaks back to you using Speech Synthesis (aka Text-to-Speech, or TTS)
  • All sounds and speech use spatial audio that originate from the HoloBot
  • The "brain" of the HoloBot can be any public bot built with the Microsoft Bot Framework

Video Demonstration

ScreenShot

The bot demonstrated in this video is The Maker Show Bot, also found here on GitHub.

Instructions / Notes

All

Connecting your Bot to HoloBot

  • Create and register your bot as per the intructions at https://dev.botframework.com. Since HoloBot uses free natural language dictation, it is highly recommended that your bot support NLP via Natural Language Understanding Service (LUIS) from Microsoft Cognitive Services
  • Enable the Direct Line channel on your bot
  • Generate and copy your Direct Line secret (aka API key)
  • Open BotService.cs in the /Scripts folder of the HoloBot Unity project and paste your Direct Line secret in the _APIKEY private string

All

Acknowledgments

I want to offer special thanks to the following people who have helped me in building this sample app:

  • Jarez Bienz, for the Text-to-Speech component that he wrote the the HoloLens Unity Toolkit, and for his help in integrating my UWP Bot Framework code into Unity
  • Kat Haris, for her awesome Unity skills and helping me with audio sources triggers
  • Vanessa Arnauld & Sara Nagy, for being incredible holographic "enablers" :)
  • The whole Microsoft Holographic Academy team & mentors - especially Pat - for their awesome training, resources, patience and help

Follow Me