Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MRTK3 should expose a method to query if HoloLens 2 eye tracking is calibrated #107

Closed
3 tasks done
IssueSyncBot opened this issue Aug 20, 2023 · 1 comment
Closed
3 tasks done
Assignees
Labels
Type: Feature Request A request for a new feature that can be included with the next minor version release.

Comments

@IssueSyncBot
Copy link

IssueSyncBot commented Aug 20, 2023

Original issue opened by:

@AMollis AMollis


Overview

MRTK2 had exposed a way to query if HoloLens eye tracking is calibrated. This is useful, because eye tracking doesn't function on HoloLens 2 if the user hasn't calibrated, and apps may want to change UX based on this lack of calibration.

Tasks

  • Add an API to query if eye tracking is calibrated.
  • Add usage of new API to Eye Gaze sample scene
  • Add manual test for HL2

API Recommendation

Add standalone helper class like EyeCalibrationChecker into Input package. Log a "warning" message if this is used on non-UWP platform.

Other Possibilities

  1. Extend ActionBasedController and create a GazeController, and add bool IsCalibrated() to GazeController
  2. Possibly add bool IsCalibrated() member function to IGazeInteractor
  3. Something else?

Workaround

The following code can be used to query this information:

// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.

using System;
using UnityEngine;
using UnityEngine.Events;
#if WINDOWS_UWP
using Windows.Perception;
using Windows.Perception.People;
using Windows.Perception.Spatial;
using Windows.UI.Input.Spatial;
#endif

namespace Microsoft.MixedReality.Toolkit.Examples
{
    /// <summary>
    /// Checks whether the user is calibrated and prompts a notification to encourage the user to calibrate.
    /// </summary>
// TODO make sure this component menu is the same as other "input" things
    [AddComponentMenu("Scripts/MRTK/Input/EyeCalibrationChecker")]
    public class EyeCalibrationChecker : MonoBehaviour
    {
        [Tooltip("For testing purposes, you can manually assign whether the user is eye calibrated or not.")]
        [SerializeField]
        private bool editorTestUserIsCalibrated = true;

        public UnityEvent OnEyeCalibrationDetected;
        public UnityEvent OnNoEyeCalibrationDetected;

        private bool? prevCalibrationStatus = null;

        private void Update()
        {
            bool? calibrationStatus;

            if (Application.isEditor)
            {
                calibrationStatus = editorTestUserIsCalibrated;
            }
            else
            {
                calibrationStatus = CheckCalibrationStatus();
            }

            if (calibrationStatus.HasValue)
            {
                if (prevCalibrationStatus != calibrationStatus)
                {
                    if (!calibrationStatus.Value)
                    {
                        OnNoEyeCalibrationDetected.Invoke();
                    }
                    else
                    {
                        OnEyeCalibrationDetected.Invoke();
                    }
                    prevCalibrationStatus = calibrationStatus;
                }
            }
        }

        private bool CheckCalibrationStatus()
        {
#if WINDOWS_UWP

            if (MixedReality.OpenXR.PerceptionInterop.GetSceneCoordinateSystem(Pose.identity) is SpatialCoordinateSystem worldOrigin)
            {
                SpatialPointerPose pointerPose = SpatialPointerPose.TryGetAtTimestamp(worldOrigin, PerceptionTimestampHelper.FromHistoricalTargetTime(DateTimeOffset.Now));
                if (pointerPose != null)
                {
                    EyesPose eyes = pointerPose.Eyes;
                    if (eyes != null)
                    {
                        return eyes.IsCalibrationValid;
                    }
                }
            }
#endif // WINDOWS_UWP

            return true;
        }
    }
}
### Tasks
- [ ] Add an API to query if eye tracking is calibrated.
- [ ] Add usage of new API to Eye Gaze sample scene
- [ ] Add Unit Tests. Since Unit Tests will run on non-HL2s, tests should only validate behavior for non-HL2 devices.

ISSUE MIGRATION

Issue migrated from: microsoft/MixedRealityToolkit-Unity#11570

@IssueSyncBot IssueSyncBot added MRTK2 Type: Feature Request A request for a new feature that can be included with the next minor version release. labels Aug 20, 2023
@IssueSyncBot
Copy link
Author

Original comment by:

@marlenaklein-msft marlenaklein-msft


Closed with #11664

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: Feature Request A request for a new feature that can be included with the next minor version release.
Projects
None yet
Development

No branches or pull requests

2 participants