ENH: Add OpenXRRemoting external project #2

jcfr · 2023-04-07T14:51:55Z

Download v2.9.1 of Microsoft.Holographic.Remoting.OpenXR

References:

https://www.nuget.org/packages/Microsoft.Holographic.Remoting.OpenXr/2.9.1

Download v2.9.1 of Microsoft.Holographic.Remoting.OpenX References: * https://www.nuget.org/packages/Microsoft.Holographic.Remoting.OpenXr/2.9.1

…zard

jcfr · 2023-04-28T05:10:29Z

Running the emulator requires special permission:

jcfr · 2023-04-28T05:15:12Z

Notes on installing Visual Studio components to support building the "player" application referenced in the VTK - OpenXR Holographic Remoting documentation¹.

See https://discourse.slicer.org/t/windows-preview-build-delayed-visual-studio-2022-updates/29045

https://docs.vtk.org/en/latest/vtk_modules/vtk-modules/Rendering/OpenXRRemoting/README.html ↩

jcfr · 2023-04-28T05:58:18Z

Step by step

Starts the emulator

Open the solution associate with the player application. After it has been built (see below), it reports the following in the Visual Studio output tab.

ConfigurationParser/Constructor/EnumerateEmulators: START
    NumberOfDevicesFound = 1
    ConfigurationParser/Constructor/EnumerateEmulators/Emulator
	    Id = 32b69331-0d59-2dec-951b-9b4e72ae9a2a
	    Name = HoloLens 2 Emulator 10.0.20348.1535
	    Type = Emulator
	    Version = 10.0.20348.1535
	    UwpVersion = 10.0.20348.1535
ConfigurationParser/Constructor/EnumerateEmulators: END (Success, 0:00:00.146)
ConfigurationParser/Constructor: END (Success, 0:00:00.218)
IsEmulatorSupported: START
    Is64BitOperatingSystem = True
DetectSlat: START
    isHyperVRunning = True
DetectSlat: END (Success, 0:00:00.105)
    SlatState = SlatAndHWVEnabled
    IsEmulatorSupported = True
IsEmulatorSupported: END (Success, 0:00:00.109)

After attempting to start debugging session to run the player app on the emulator, we only see the option Download New emulators

Building the player app

References

Prerequisites

Update Visual Studio 2022 installation (Tools -> Get Tools and Features):

MSVC v143 - VS 2022 C++ ARM64 build tools (Latest)
MSVC v143 - VS 2022 C++ ARM64 Spectre-mitigated libs (Latest)
C++ Universal Windows Platform support for v143 build tools (ARM64)

To address the error message "Sorry, something went wrong"`, see these instructions.

Building

On Windows, build the "player application"

git clone https://github.com/microsoft/MixedReality-HolographicRemoting-Samples

Open player/sample/SamplePlayer.sln with right-click and select Open With -> Visual Studio 2022

On first use, right-click the solution and select Restore NuGet Packages

Select Release

…zation

…ctivate Update vtkSlicerMixedRealityLogic: * Add SetMixedRealityConnected/GetMixedRealityConnected * Add SetMixedRealityActive/GetMixedRealityActive * SetDefaultReferenceView

Fixes: ``` Exception thrown: read access violation. this was nullptr. Call stack: vtkRendering-9.1.dll!vtkRenderer::SetActiveCamera(vtkCamera * cam) Line 770 C++ ```

When using remoting, the OpenXR loader must be built dynamically to allow the remoting package to override this loader at runtime. (see RemotingXR.json) Fixes: ``` ERROR [GENERAL | xrGetInstanceProcAddr | OpenXR-Loader]: No active xrInstance handle. ```

Fixes: Unable to open openXR action file : [...]/externals/vtkRenderingOpenXRvtk_openxr_actions.json

This requires VTK MR-10449 (see [1]) backported to Slicer/VTK through the Slicer PR-7190 (see [2]). [1] https://gitlab.kitware.com/vtk/vtk/-/merge_requests/10449 [2] Slicer/Slicer#7190

jcfr · 2023-08-25T16:55:36Z

Pull requests implementing required changes:

ENH: Update VTK to support distribution of OpenXRRemoting in extension Slicer/Slicer#7190
COMP: Expand <CMAKE_CFG_INTDIR> in extension additional launcher settings Slicer/Slicer#7191

Ensure that the "openxr_loader.dll" library is found when loading the MixedReality module from a build tree using the "SlicerWithSlicerMixedReality.exe" launcher.

jcfr · 2023-08-25T23:00:16Z

Pull request implementing useful changes:

ENH: Update VTK to backport OpenXR error handling improvements Slicer/Slicer#7193

…nfigured Adapted from KitwareMedical/SlicerVirtualReality@173b1a837

VTK_INSTALL_RUNTIME_DIR or VTK_INSTALL_LIBRARY_DIR CMake variables were completely removed in VTK 9.1.

List of changes: $ git shortlog 10d08094..b03e854a4 --no-merges Jean-Christophe Fillion-Robin (3): Add SuperBuild support Remove redundant BUILD_TESTING option COMP: Support configuring install directories

Adapted from KitwareMedical/SlicerVirtualReality@a4d465b73 Remove the "externals" sub-directory that was specific to branch Slicer/VTK@slicer-v9.1.20220125-efbe2afc2 integrating commit Slicer/VTK@0ba7af630 ([Backport MR-8123] vtk_module_build: Ensure external modules build directory are not overwritten) Since the approach originally proposed in VTK MR-8123 was superseded by MR-9097 there is no need to specify the "externals" sub-directory.

jcfr · 2023-08-26T02:58:36Z

Pull request implementing required changed:

ENH: Update VTK backporting RenderingVR and RenderingOpenXR fixes Slicer/Slicer#7195

Especially the backport of MR-10452: OpenXR: Consistently name manifest install sub-directory

…gers

Note that the vtkMixedRealityViewInteractorStyle still derives from vtkOpenXRInteractorStyle whereas the VirtualReality counterpart derives from vtkMRMLViewInteractorStyle.

jcfr · 2023-08-28T07:25:07Z

The extension can now be built on Windows (and even packaged & installed) without requiring the developer to copy files and libraries.

Next step we will need to refactor SlicerVirtualReality extension so that relevant functionalities can be reused while minimizing duplication:

Refactoring to support OpenVR and OpenXR, and streamline reuse in SlicerMixedReality SlicerVirtualReality#121

We will also have to improve the OpenXR integration so that we can retrieve the position of each controllers & trackers:

Refactor transform tracking to support OpenXR SlicerVirtualReality#122

cc: @LucasGandel @lassoan @cpinter @adamrankin

ENH: Add OpenXRRemoting external project

09c27f1

Download v2.9.1 of Microsoft.Holographic.Remoting.OpenX References: * https://www.nuget.org/packages/Microsoft.Holographic.Remoting.OpenXr/2.9.1

jcfr linked an issue Apr 7, 2023 that may be closed by this pull request

Add external project to build vtkRenderingOpenXRRemoting #1

Closed

5 tasks

jcfr added 3 commits April 7, 2023 10:57

COMP: Ensure vtkRenderingOpenXRRemoting finds OpenXRRemoting

fe93619

COMP: Remove FooBarWidget originally tgenerated using the ExtensionWi…

1b591f2

…zard

DOC: Update license headers to be consistent and reference Apache 2.0

6b995ee

jcfr force-pushed the integrate-microsoft-holographic-remoting branch 5 times, most recently from 648006e to 43c3a46 Compare April 10, 2023 04:03

jcfr added 6 commits May 19, 2023 06:40

WIP: Add initial skeleton (view node and displayable manager factory)

b713706

WIP: Add logic for instantiating view node and associated render window

9e62069

ENH: Add support for setting PlayerIPAddress node property

3f2c589

BUG: Ensure action manifest files are found during interctor initiali…

8669f84

…zation

WIP: Update vtkSlicerMixedRealityLogic adding methods for connect & a…

d31a007

…ctivate Update vtkSlicerMixedRealityLogic: * Add SetMixedRealityConnected/GetMixedRealityConnected * Add SetMixedRealityActive/GetMixedRealityActive * SetDefaultReferenceView

WIP: Add debug statements

30541a4

jcfr force-pushed the integrate-microsoft-holographic-remoting branch from 071a631 to 30541a4 Compare May 19, 2023 10:41

LucasGandel added 4 commits May 26, 2023 13:24

BUG: Add missing instantiation of Renderer and RenderWindow

a0872f2

Fixes: ``` Exception thrown: read access violation. this was nullptr. Call stack: vtkRendering-9.1.dll!vtkRenderer::SetActiveCamera(vtkCamera * cam) Line 770 C++ ```

BUG: Fix action manifest path with trailing slash

2cf9eae

Fixes: Unable to open openXR action file : [...]/externals/vtkRenderingOpenXRvtk_openxr_actions.json

WIP: Process XR events

b772173

LucasGandel mentioned this pull request Aug 2, 2023

Doc Request: Add FAQ and/or Roadmap KitwareMedical/SlicerVirtualReality#127

Closed

4 tasks

jcfr added 2 commits August 25, 2023 04:27

COMP: Install VTK Rendering libraries: VR, OpenXR and OpenXRRemoting

898ea6d

ENH: Copy and install OpenXRRemoting libraries along side VTK libraries

b4610cf

This requires VTK MR-10449 (see [1]) backported to Slicer/VTK through the Slicer PR-7190 (see [2]). [1] https://gitlab.kitware.com/vtk/vtk/-/merge_requests/10449 [2] Slicer/Slicer#7190

COMP: Fix launcher setting to ensure OpenXR-SDK library is found

a41caf6

Ensure that the "openxr_loader.dll" library is found when loading the MixedReality module from a build tree using the "SlicerWithSlicerMixedReality.exe" launcher.

jcfr added 7 commits August 25, 2023 19:29

COMP: Ensure vtkRendering(XR|OpenXR|OpenXRRemoting) modules are re-co…

ba65781

…nfigured Adapted from KitwareMedical/SlicerVirtualReality@173b1a837

COMP: Fix install dirs in VTK module external project to support VTK9

ca3d863

VTK_INSTALL_RUNTIME_DIR or VTK_INSTALL_LIBRARY_DIR CMake variables were completely removed in VTK 9.1.

COMP: Update VTKExternalModule

309a206

List of changes: $ git shortlog 10d08094..b03e854a4 --no-merges Jean-Christophe Fillion-Robin (3): Add SuperBuild support Remove redundant BUILD_TESTING option COMP: Support configuring install directories

COMP: Conditionally install vtkRenderingOpenXRRemoting libraries

ef1eb99

COMP: Customize generated config adding "vtkRenderingOpenXRRemoting_DIR"

694ca43

COMP: Add install rules for OpenXR library

f31ada6

jcfr force-pushed the integrate-microsoft-holographic-remoting branch 8 times, most recently from fa3d863 to ac01e64 Compare August 28, 2023 01:47

jcfr added 2 commits August 27, 2023 21:48

ENH: Register Markups, Tranform and Volume Rendering displayable mana…

c8a1274

…gers

ENH: Add MixedReality Interactor and InteractorStyle classes

ab2bb48

Note that the vtkMixedRealityViewInteractorStyle still derives from vtkOpenXRInteractorStyle whereas the VirtualReality counterpart derives from vtkMRMLViewInteractorStyle.

jcfr force-pushed the integrate-microsoft-holographic-remoting branch 4 times, most recently from 290c64c to 45a8513 Compare August 28, 2023 03:30

ENH: Add support for connect/disconnect

c8cdf68

jcfr force-pushed the integrate-microsoft-holographic-remoting branch from a4685f5 to c8cdf68 Compare August 28, 2023 05:05

jcfr merged commit 3ae5ff3 into main Aug 28, 2023

jcfr deleted the integrate-microsoft-holographic-remoting branch August 28, 2023 05:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ENH: Add OpenXRRemoting external project #2

ENH: Add OpenXRRemoting external project #2

jcfr commented Apr 7, 2023 •

edited

jcfr commented Apr 28, 2023 •

edited

jcfr commented Apr 28, 2023 •

edited

jcfr commented Apr 28, 2023 •

edited

jcfr commented Aug 25, 2023

jcfr commented Aug 25, 2023

jcfr commented Aug 26, 2023

jcfr commented Aug 28, 2023

ENH: Add OpenXRRemoting external project #2

ENH: Add OpenXRRemoting external project #2

Conversation

jcfr commented Apr 7, 2023 • edited

jcfr commented Apr 28, 2023 • edited

jcfr commented Apr 28, 2023 • edited

Footnotes

jcfr commented Apr 28, 2023 • edited

Step by step

Building the player app

References

Prerequisites

Building

jcfr commented Aug 25, 2023

jcfr commented Aug 25, 2023

jcfr commented Aug 26, 2023

jcfr commented Aug 28, 2023

jcfr commented Apr 7, 2023 •

edited

jcfr commented Apr 28, 2023 •

edited

jcfr commented Apr 28, 2023 •

edited

jcfr commented Apr 28, 2023 •

edited