HoloToolkit.Sharing

Joe van den Heuvel edited this page Mar 25, 2016 · 4 revisions

Overview

HoloToolkit.Sharing is a library that allows applications to span multiple devices.

Originally developed for OnSight, a collaboration between SOTA (a Microsoft studio) and NASA to enhance their existing Mars rover planning tool with HoloLens, HoloToolkit.Sharing enables users to use multiple devices for a task by allowing the apps running on each device communicate and stay in sync seamlessly in real time. Users can collaborate with other users who may also be using multiple devices, and may be in the same room or working remotely.

The Sharing library is used internally by the Microsoft HoloLens team for its own demos and prototypes. We are continuously updating it based on the needs of our projects, but we do not claim that it is complete or bug-free. We wanted to open it up to the community in this unpolished form because we found it useful, and hope you do too.

HoloToolkit.Sharing consists of

  • A client library that can plug in to any application. The client library allows apps to connect directly to other HoloToolkit-enabled apps or to the Sharing Service
  • A server executable (Sharing Service) that facilitates discovery and connecting remote users. The server can run as a Windows Service or as a command-line application.

Features

Multi-Platform, Multi-language

To allow its use in as many apps as possible, HoloToolkit.Sharing is written in portable, cross-platform C++ native code. Wrappers APIs are then generated automatically from the native code for other programming languages. For wrapper languages that support reference counting, the wrappers are generated in such a way that the reference counting of the native objects is correctly shared between the native code and wrapper language, ensuring correct object lifetime.

Currently supported Platforms: Win32, Windows Universal, Mac OSX
Currently supported Languages: C++, C#, Java

Lobby & Session system

  • Discover available sessions or create your own
  • Permanent or "until empty" session lifetime
  • See user status: current session, mute state
  • Easy to discover and hop between sessions

Anchor Sharing

  • Users in a session can be in the same or different physical rooms
  • Users can share the location of Holographic 'anchors' they place in their room with other users in the same room
  • Users joining late can download all anchors in the session
  • Allows multiple users to see shared holograms

Synchronization System

Synchronize data across all participants in session

  • Everyone in session guaranteed to see the same thing
  • Automatic conflict resolution for simultanious conflicting remote changes
  • Real-time: See remote changes as they happen
  • Shared data sets automatically merged when new users join a session
  • Responsive: no delay in your own changes
  • Ownership: your data leaves session when you do

Group Voice Chat

Support for VOIP is built-in

  • Server-based mixing lowers processing and bandwidth requirements for clients

Visual Pairing

Connect devices just by looking at them

  • One device displays a QR code with connection info and security code
  • Other device sees QR code, connects, and validates with security code
  • Can also detect location in 3D space using built-in fiducial marker support

Profiler

Profiling and debugging an experience that spans multiple devices is challenging. So HoloToolkit.Sharing provides an app that can connect to multiple devices at once and aggregates their timings and debug output in a single place