MRTK3 should expose a method to query if HoloLens 2 eye tracking is calibrated

[AMollis]

This issue has been migrated a new MRTK repository, and the status of this issue will now be tracked at the following location:

• MRTK3 should expose a method to query if HoloLens 2 eye tracking is calibrated MixedRealityToolkit/MixedRealityToolkit-Unity#107

---

Overview

MRTK2 had exposed a way to query if HoloLens eye tracking is calibrated. This is useful, because eye tracking doesn't function on HoloLens 2 if the user hasn't calibrated, and apps may want to change UX based on this lack of calibration.

Tasks

• Add an API to query if eye tracking is calibrated.
• Add usage of new API to Eye Gaze sample scene
• Add manual test for HL2

API Recommendation

Add standalone helper class like EyeCalibrationChecker into Input package. Log a "warning" message if this is used on non-UWP platform.

Other Possibilities

1. Extend ActionBasedController and create a GazeController, and add bool IsCalibrated() to GazeController
2. Possibly add bool IsCalibrated() member function to IGazeInteractor
3. Something else?

Workaround

The following code can be used to query this information:

// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.

using System;
using UnityEngine;
using UnityEngine.Events;
#if WINDOWS_UWP
using Windows.Perception;
using Windows.Perception.People;
using Windows.Perception.Spatial;
using Windows.UI.Input.Spatial;
#endif

namespace Microsoft.MixedReality.Toolkit.Examples
{
    /// <summary>
    /// Checks whether the user is calibrated and prompts a notification to encourage the user to calibrate.
    /// </summary>
// TODO make sure this component menu is the same as other "input" things
    [AddComponentMenu("Scripts/MRTK/Input/EyeCalibrationChecker")]
    public class EyeCalibrationChecker : MonoBehaviour
    {
        [Tooltip("For testing purposes, you can manually assign whether the user is eye calibrated or not.")]
        [SerializeField]
        private bool editorTestUserIsCalibrated = true;

        public UnityEvent OnEyeCalibrationDetected;
        public UnityEvent OnNoEyeCalibrationDetected;

        private bool? prevCalibrationStatus = null;

        private void Update()
        {
            bool? calibrationStatus;

            if (Application.isEditor)
            {
                calibrationStatus = editorTestUserIsCalibrated;
            }
            else
            {
                calibrationStatus = CheckCalibrationStatus();
            }

            if (calibrationStatus.HasValue)
            {
                if (prevCalibrationStatus != calibrationStatus)
                {
                    if (!calibrationStatus.Value)
                    {
                        OnNoEyeCalibrationDetected.Invoke();
                    }
                    else
                    {
                        OnEyeCalibrationDetected.Invoke();
                    }
                    prevCalibrationStatus = calibrationStatus;
                }
            }
        }

        private bool CheckCalibrationStatus()
        {
#if WINDOWS_UWP

            if (MixedReality.OpenXR.PerceptionInterop.GetSceneCoordinateSystem(Pose.identity) is SpatialCoordinateSystem worldOrigin)
            {
                SpatialPointerPose pointerPose = SpatialPointerPose.TryGetAtTimestamp(worldOrigin, PerceptionTimestampHelper.FromHistoricalTargetTime(DateTimeOffset.Now));
                if (pointerPose != null)
                {
                    EyesPose eyes = pointerPose.Eyes;
                    if (eyes != null)
                    {
                        return eyes.IsCalibrationValid;
                    }
                }
            }
#endif // WINDOWS_UWP

            return true;
        }
    }
}

Tasks

Beta Give feedback

1. Add an API to query if eye tracking is calibrated.
2. Add usage of new API to Eye Gaze sample scene
3. Add Unit Tests. Since Unit Tests will run on non-HL2s, tests should only validate behavior for non-HL2 devices.

[marlenaklein-msft]

Closed with #11664