Update XR Rig prefab to better support HoloLens 2 scenarios

[AMollis]

This issue has been migrated a new MRTK repository, and the status of this issue will now be tracked at the following location:

• Update XR Rig prefab to better support HoloLens 2 scenarios MixedRealityToolkit/MixedRealityToolkit-Unity#323

---

Overview

We recently hit issues where the default XR Rig settings didn't work with Azure Spatial Anchors and Azure Remote Rendering on HoloLens 2. We should improve the experience for HoloLens 2 so that these scenarios work out of the box.

Tasks

• Figure out correct defaults for MRKT3's XR Rig prefab. At the moment Tracking Origin Mode is set to Not Specified and Camera Y Offset set to 1.6. This doesn't seem ideal for HoloLens2, but perhaps it is correct given the Transform.Position.Y value is 1.6. However, these settings didn't work correctly with ASA and ARR; need to understand why.
• Update MRTK3's XR Rig prefab as needed
• Maybe create a HoloLens 2 specific XR Rig prefab?
• Add MSFT's EyeLevelSceneOrigin to HL2 specific XR Rig prefab?

Current Defaults:

More Context

For ASA and ARR to function correctly on HoloLens 2, we had to set Tracking Origin Mode to Device and Camera Y Offset to 0.

Some useful information from the MSFT OpenXR Team:

Tracking origin mode sets the origin of XR content in the Unity scene to either be centered in your headset's starting position (Device) or centered on the floor below your starting position (Floor) 

The Camera Y Offset is how far the camera is translated above that XR origin within the Unity scene. 

Typically, we do TrackingOriginMode=Device, Camera Y Offset=0, which makes sure the experience starts around your head (assuming the Unity app is built around the origin). The other common (older) configuration I've seen is TrackingOriginMode=Floor, Camera Y Offset=1.6, which places the origin on the floor (roughly) and the camera still where your head is.

Also from OpenXR Team:

The device mode should be set to "Unbounded" not just "Device".

The "Device" setting uses "Local" tracking. The MSFT OpenXR plugin has a component to force the app into an "Unbounded" tracking. See the EyeLevelSceneOrigin class for details. Perhaps the MRTK XR Rig prefab for HoloLens should use this.

[keveleigh]

Are you seeing an issue with the current defaults on HL2? When set to "Not Specified", the XR Rig allows the device/runtime to determine its optimal mode. The Camera Y Offset is only used when the device/runtime decides to use "Floor", so this value should not affect HL2.

As a side note, I had a request out to Unity to add "Unbounded" to the XR Origin component, so it should show up in a future iteration. I don't remember the timeline.

[AMollis]

Are you seeing an issue with the current defaults on HL2?

Using the default configuration, both ASA and ARR don't function correctly. ASA recalls anchors 1.6m too high. ARR renders content 1.6m too high. Setting tracking origin to Device and offset to 0 addressed this problem.

We need to understand the issue a bit better before we can determine the correct course of action here. Whatever is chosen we'll do our best not to prohibit usage on non-HoloLens devices.

[keveleigh]

The Camera Y Offset is only used when the device/runtime decides to use "Floor", so this value should not affect HL2.

Just realized I had this exactly backwards, my bad! It actually only takes effect when using "Device" 😅 this is likely the issue you're seeing, but it sounds like a bug in ASA and ARR to me. They should be placing/transforming their content relative to the Camera Offset (or really, probably the Trackables Parent object for anchors), similar to how ARFoundation's components (like ARMeshManager and ARAnchorManager) work. Depending on the Unity/ARFoundation versions they support, they'll be looking for the XROrigin or ARSessionOrigin components.

This snippet from the 4.2.7 release of ARAnchorManager:

MRTK3 surfaces this as a static member via PlayspaceUtilities.XROrigin and some helpful pose transform methods:

MixedRealityToolkit-Unity/com.microsoft.mrtk.core/Utilities/PlayspaceUtilities.cs

Lines 41 to 54 in 9941264

	/// <summary>
	/// Transforms a <see cref="Pose"/> from OpenXR scene-origin-space to Unity world-space.
	/// Uses the XROrigin's CameraFloorOffsetObject transform.
	/// </summary>
	public static Pose TransformPose(Pose pose)
	{
	// Null-checking Unity objects can be expensive. Caching this here cuts two null checks into one.
	// Here, we use CameraFloorOffsetObject, because poses are reported local to the floor offset.
	Transform origin = XROrigin.CameraFloorOffsetObject.transform;
	return new Pose(
	origin.TransformPoint(pose.position),
	origin.rotation * pose.rotation
	);
	}

For correctness with a broader configuration of apps, ASA and ARR should also account for this offset object.

[srinjoym]

@keveleigh I'm trying to better understand the motivation behind setting the default "Camera Y Offset" to 1.6, and the default Y Position on the Camera Offset GameObject to 1.6:

This seems like it has no net effect (GameObject moves up 1.6m then XROrigin moves it back down 1.6) other than allowing the user to build the scene at a different origin height (the MRTK content appear to be positioned at a height of 1.769m).

This same effect could be achieved by setting both the GameObject Y Position to 0, the "Camera Y Offset" to 0, and moving the scene content to 0 height. Do you recall if there any reason the scene content was positioned at the 1.769m height in MRTK 3? The example scenes in MRTK 2 all had content centered around the default origin at 0.

I'm still looking into the ARR and ASA bugs separately - agree that they should respect the offset if it's there.

[keveleigh]

This seems like it has no net effect (GameObject moves up 1.6m then XROrigin moves it back down 1.6) other than allowing the user to build the scene at a different origin height (the MRTK content appear to be positioned at a height of 1.769m).

The GameObject is at 1.6m at edit time to better visualize the scene you're building through Unity's various camera previews, as it matches where the camera will relatively be at runtime.

At runtime, Unity will set the Camera Offset GameObject's height to 1.6m according to the XROrigin's setting if the tracking origin mode is Device (so, it effectively won't move in this configuration because it was already set that way in the editor, but that's good because it matches what a dev was seeing in the editor).
If the tracking origin mode is Floor, XROrigin sets the Camera Offset GameObject's height to 0m. Setting it to 0m allows the device to directly localize itself above the runtime's floor space, which will be somewhere in a similar range to 1.6m above the scene origin (1.6m is ~5' 3", so slightly different depending on a user's actual height), so the scene content should still be relatively in a predictable location.

This is in XROrigin's source:

As far as why we've preferred this offset as opposed to localizing everything directly around the head is largely to do with XRI's locomotion/teleportation implementation. It assumes there will be some floor representation (and thus, some offset between the head and the floor), whether it's an artificial 1.6m or one actually determined by the runtime.

https://docs.unity3d.com/Packages/com.unity.xr.interaction.toolkit@2.3/manual/locomotion.html

This same effect could be achieved by setting both the GameObject Y Position to 0, the "Camera Y Offset" to 0, and moving the scene content to 0 height.

While this is true for devices that default to XR_REFERENCE_SPACE_TYPE_LOCAL or XR_REFERENCE_SPACE_TYPE_UNBOUNDED_MSFT, devices that have a built-in XR_REFERENCE_SPACE_TYPE_STAGE (like desktop WMR and Quest) will now have their content embedded in the physical world's floor.

The example scenes in MRTK 2 all had content centered around the default origin at 0.

This is true! And it led to some special casing in the teleportation logic where we tried to guesstimate where we should put the camera post-teleport that didn't really work in all cases.

MixedRealityToolkit-Unity/Assets/MRTK/Services/TeleportSystem/MixedRealityTeleportSystem.cs

Lines 300 to 302 in b026319

	float height = targetPosition.y;
	targetPosition -= CameraCache.Main.transform.position - MixedRealityPlayspace.Position;
	targetPosition.y = height;

Regarding the floor-embedded content I mentioned above, this also led to us creating a special MonoBehaviour that adjusted the app's content's height at runtime, but I've found the XROrigin approach to be more robust in both cases (as well as being a Unity-created component instead of something custom!).

[srinjoym]

Thanks for the detailed explanation! The rationale for using the Y Offset makes sense. I'm working on validating the issues with our services and checking the behavior of this with OpenXR's unbounded space.

ASA already has a fix coming and I've validated that this is no longer an issue.

[srinjoym]

Confirmed ARR does not respect the Y Offset on the XR Origin, following up with them to create a bug. I think based on the investigation here we can go ahead and close this issue as there's no changes needed on MRTK.