Controller devicePose and pointer Poses

[fight4dream]

Is your feature request related to a problem?

OpenXR has standardized the grip pose and pointer pose.

Grip pose is a pose that allows applications to reliably render a virtual object held in the user’s hand, whether it is tracked directly or by a motion controller.

This grip pose can be useful for InteractorFacade to set its grab attach point location, and/or help orient the controller and hand models.

Pointer pose is a pose that allows applications to point in the world using the input source, according to the platform’s conventions for aiming with that kind of source.

The pointer pose can be useful for PointerFacade to follow. This shall be the device/platform recommended pointer location and orientation.

Describe the solution you'd like

Add to the Input action map the corresponding binding for the devicePose, pointer, and/or other unity extended such as devicePosition, deviceOrientation.

We need a CallbackContextToPose for the UnityEngine.XR.OpenXR.Input.Pose struct, and then PosePositionExtractor to extract position and PoseEulerAngleExtractor to extract euler angles.

Additional context

The OpenXR Specification: 6.3.2. Input subpaths
OpenXR Plugin: Pose Data

[fight4dream]

Use case 1:
We want the LeftHand and RightHand be tracking the device poses directly.

1. Add CameraRigs.UnityXRPluginFramework
2. Add interactors and change the hand model
3. Orient the hand models such that the Grab Attach Point
   a. is at the palm centroid
   b. its +X axis away from palm for left hand, -X away from palm for right hand
   c. its +Z runs along the head line of the palm towards thumb/index finger

3. Add UnityInputSystem.Mappings.GenericXR, assume we have the CallbackContextToPose setup for the device poses
4. Add a PoseProvider to the Left_DevicePose and store the pose returned from callback context into the pose provider
5. Drag the pose provider to the LeftHand Tracked Pose Driver 'Use Pose Provider'
6. Repeat with RightHand and Right_DevicePose

Now the LeftHand and RightHand should track the grab point, and our hand models is the same position of our real life hands.

namespace Tilia.Input.UnityInputSystem.Transformation.Conversion
{
    using System;
    using UnityEngine;
    using UnityEngine.Events;
    using UnityEngine.InputSystem;
    using Pose = UnityEngine.XR.OpenXR.Input.Pose;

    /// <summary>
    /// Transforms a <see cref="InputAction.CallbackContext"/> to a <see cref="Pose"/>.
    /// </summary>
    public class CallbackContextToPose : CallbackContextTransformer<Pose, CallbackContextToPose.UnityEvent>
    {
        /// <summary>
        /// Defines the event with the transformed <see cref="Pose"/> value.
        /// </summary>
        [Serializable]
        public class UnityEvent : UnityEvent<Pose> { }

        /// <summary>
        /// Transforms the given input <see cref="InputAction.CallbackContext"/> to the equivalent <see cref="Pose"/> value.
        /// </summary>
        /// <param name="input">The value to transform.</param>
        /// <returns>The transformed value.</returns>
        protected override Pose Process(InputAction.CallbackContext input)
        {
            return input.ReadValue<Pose>();
        }
    }
}

using UnityEngine;
using UnityEngine.Experimental.XR.Interaction;
using UnityEngine.SpatialTracking;

public class PoseProvider : BasePoseProvider
{
    public UnityEngine.XR.OpenXR.Input.Pose Pose { get; set; }

    public override PoseDataFlags GetPoseFromProvider(out Pose output)
    {
        PoseDataFlags flags = PoseDataFlags.NoData;
        output = default;
        if (Pose.trackingState.HasFlag(UnityEngine.XR.InputTrackingState.Position))
        {
            flags |= PoseDataFlags.Position;
            output.position = Pose.position;
        }
        if (Pose.trackingState.HasFlag(UnityEngine.XR.InputTrackingState.Rotation))
        {
            flags |= PoseDataFlags.Rotation;
            output.rotation = Pose.rotation;
        }
        return flags;
    }
}

[fight4dream]

#28

[fight4dream]

I found that the default left/right hand tracking is already using devicePosition and deviceRotation.
Also there is a new Tracked Pose Driver that comes with the new Input System. It can directly bind to pointerPosition and pointerRotation. Thus no need for additional custom scripts or modification of the current prefab.

Added two guides instead
How to use the OpenXR controller Pointer pose
How to orient the hand model under OpenXR Specifications

[thestonefox]

This PR implements this I believe

#39