HoloCT

A medical imaging holographic application for viewing CT data in Microsoft's HoloLens device and dynamically loading holographic 'cases' from blob storage. Holographic cases are in XML format with mesh data stored in GTLF2 format, compressed in base 64.
Made in collaboration with Great Ormond Street Hospital, NHS ‘Digital Research, Informatics and Virtual Environments’ unit (DRIVE): https://www.gosh.nhs.uk/news/great-ormond-street-hospital-partner-microsoft-transform-healthcare-using-artificial-intelligence and UCL industry exchange (IXN) network: http://ixn.org.uk/.

Video demonstration here:
https://youtu.be/nXyaozmQpT8

Authors

This project was developed by UCL Computer Science students as part of the UCL Industry Exchange Network (http://ixn.org.uk) which pairs university students with industry as part of their curriculum.
Inventor and project lead - Dr Dean Mohamedally, d.mohamedally@ucl.ac.uk
MSc Student - Dr Caroline Smith

Code and Storage

Key files are in the assets folder, with the following structure:
/Assets
    /Holograms – The fetal meshes used in the app (in .fbx format)
    /Resources - Images, xml files, materials for meshes
    /Scripts – The C# scripts for the app
    /Scenes – The .unity files that contain the main app scenes

Example .MHIF (medical holographic interchange format) files and glTF meshes can be found in /MHIF_Cases

Built With

• Unity 2017.4.1f1.
• Visual Studio 2017: ‘Universal Windows Platform development’ & ‘Game Development with Unity’ workloads.
• Windows 10 Education with SDK update from April 6th 2018, in developer mode with Hyper-v supported.
• HoloLens Emulator and HoloLens device.

Prerequisites

See Microsoft Mixed Reality Academey for full specification of tools required for HoloLens development: https://docs.microsoft.com/en-us/windows/mixedreality/install-the-tools

Dependencies

The following libraries are used in the app and need to be installed via Unity for the app to function:
• Microsoft’s Mixed reality tool kit for Unity
• RESTClient for Unity.
• StorageServices for Unity.
• UnityGLTFLib (uses JSON.NET plugin for Unity)

Azure Blob Storage is also required for dynamically loading meshes. Once set up - enter storage account name, container name and key in the 'dynamic' scene.

Installation & Deployment

Opening in Unity:
• Download or clone this repository. In Unity – select ‘open’ and locate the ‘HoloCT’ folder to open in Unity.
• Click on the different scenes to load them and explore. All singleton components are in the ‘preload’ scene including mixed reality camera parent and input manager which are needed in every scene.
• Enter the azure storage details in the ‘dynamic load’ scene – storage acocunt name, access key and contiainer needed.
Build Settings:
• With the mixed reality toolkit already in the project, the correct settings for a mixed reality project can be applied by selecting Mixed Reality Toolkit -> Configure -> Apply Mixed Reality Project Settings. Important to target for Windows Universal UWP and enable .net scripting backend. • Add all the scenes to the build by going to file -> build settings, add open scenes. Preload scene should be the first scene (0) and menu should be second (1).
Capabilities
• The app must declare the appropriate capabilities in its manifest. The settings are found in Player Settings > Settings for Universal Windows Platform > Publishing Settings > Capabilities. InternetClient & microphone are needed.

For more information on setting up for mixed reality app:
https://github.com/Microsoft/MixedRealityToolkit-Unity/blob/master/GettingStarted.md
For how to build app to emulator:
https://docs.microsoft.com/en-us/windows/mixed-reality/holograms-101e
For how to build app to HoloLens device:
https://docs.microsoft.com/en-us/windows/mixed-reality/holograms-101

* HoloCT is provided under a GNU AFFERO GENERAL PUBLIC LICENSE and all terms of that licence apply (see LICENSE.txt). Use of the HoloCT or code is entirely at your own risk. Neither the Carolos nor DRIVE accept any responsibility for loss or damage to any person, property or reputation as a result of using the software or code. No warranty is provided by any party, implied or otherwise. This software and code is not guaranteed safe to use in a clinical or other environment and you should make your own assessment on the suitability for such use. Installation of any HoloCT software, indicates acceptance of this disclaimer. A supported and maintained version of HoloCT is available via Carolos partner DRIVE.