Skip to content

A sample Unity project for Microsoft HoloLens that lets you interact with a floating robot head using speech recognition, and getting answers spoken back by the "bot" using Text-to-Speech. The commands sent to the bot are powered by the Microsoft Bot Framework and LUIS from Microsoft Cognitive Services.

master
Switch branches/tags
Go to file
Code
This branch is 1 commit ahead, 77 commits behind ActiveNick:master.

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
UWP
 
 
 
 
 
 
 
 

README.md

HoloBot

A sample Unity project for Microsoft HoloLens that lets you interact with a floating robot head using speech recognition, and getting answers spoken back by the "bot" using Text-to-Speech. The commands sent to the bot are powered by the Microsoft Bot Framework and LUIS from Microsoft Cognitive Services.

To get started with HoloLens, visit the Windows Holographic Dev Center.

Features

  • Hovering bot head (aka HoloBot) with looping sound
  • Tap the HoloBot to move it to a different location, tap again to place
  • Gaze at the HoloBot to trigger the speech recognizer, you will hear a "Ping" sound
  • Speak "commands" using natural language
  • The HoloBot speaks back to you using Speech Synthesis (aka Text-to-Speech, or TTS)
  • All sounds and speech use spatial audio that originate from the HoloBot
  • The "brain" of the HoloBot can be any public bot built with the Microsoft Bot Framework

Video Demonstration

ScreenShot

The bot demonstrated in this video is The Maker Show Bot, also found here on GitHub.

Instructions / Notes

All

Connecting your Bot to HoloBot

  • Create and register your bot as per the intructions at https://dev.botframework.com. Since HoloBot uses free natural language dictation, it is highly recommended that your bot support NLP via Natural Language Understanding Service (LUIS) from Microsoft Cognitive Services
  • Enable the Direct Line channel on your bot
  • Generate and copy your Direct Line secret (aka API key)
  • Open BotService.cs in the /Scripts folder of the HoloBot Unity project and paste your Direct Line secret in the _APIKEY private string

All

Acknowledgments

I want to offer special thanks to the following people who have helped me in building this sample app:

  • Jarez Bienz, for the Text-to-Speech component that he wrote the the HoloLens Unity Toolkit, and for his help in integrating my UWP Bot Framework code into Unity
  • Kat Haris, for her awesome Unity skills and helping me with audio sources triggers
  • Vanessa Arnauld & Sara Nagy, for being incredible holographic "enablers" :)
  • The whole Microsoft Holographic Academy team & mentors - especially Pat - for their awesome training, resources, patience and help

Follow Me

About

A sample Unity project for Microsoft HoloLens that lets you interact with a floating robot head using speech recognition, and getting answers spoken back by the "bot" using Text-to-Speech. The commands sent to the bot are powered by the Microsoft Bot Framework and LUIS from Microsoft Cognitive Services.

Resources

Releases

No releases published

Packages

No packages published