Changing ActiveProfile at runtime does not work

[chbecker-ms]

(Ported to GitHub, reference 21522491)

Issue encountered using MRTK V2.0.0 RC1 Refresh

There's a property on MRTK.Instance called ActiveProfile, which supposedly will
swap out the profile and reset the configuration, but when I attempted to call
this I ran into a bunch of the below errors, and fixing just the camera related
errors exposed a lot more.  I managed to "fix" these locally by actively by
hunting for objects that weren't cleaned up and fixing them up.  Below is most
of the set of changes I had to make to allow swapping active profiles.

Changes I made to get this working in my project:

• MixedRealityToolkit.cs:UpdateAllServices

  • Removal of activeSystems, and registeredMixedRealityServices in foreach
    loop was causing loop to fail due to the collection changing.  Added a
    ToList() on each to fix.

  [davidkline-ms] addressing with try/catch(InvalidOperationException) for better performance.

• UIRaycastCamera

  • Had to change destroy call to DestroyImmediate in FocusProvider.cs from
    Destroy, as we needed to clean up references in a single frame.

  • Added IsNull check in MixedRealityInputModule.cs:
    GetMousePointerEventDataForMrtkPointer to check for a destroyed
    RaycastCamera, and find the new one in the scene if so.  Also since
    there is some delay I added a short circuit return for when
    RaycastCamera was null.

• ShellHandRayPointer

  • Any hand ray or controller pointer could become stale when swapping
    profiles, because we do not disable the linerenderer 100% of the time
    when setting a shell hand ray pointer inactive.  Had to add a foreach
    loop to OnPostSceneQuery in the !IsActive branch to manually disable all
    linerenderers.

• Gaze Cursor

  • Had to add setting the GazeCursor in GazeProvider.OnEnable, since it was not swapping gaze cursors when swapping profiles.

[davidkline-ms] Solving using DestroyImmediate to ensure correct re-creating.

• WindowsMixedRealityDeviceManager: Disable()

  • Had to set gestureRecognizer, and NavigationRecognizer to null after
    dispose.  We were trying to ref the disposed instances.

• GestureRecognizreExtensions

  • Had to add null check for recognizer to avoid accessing disposed or
    destroyed instance.

• Disabling the gaze cursor in the input profile does not work

  • Added a force override for force disabling the gaze cursor in BaseCursor

• InputSimulationService::OnHandDataChanged

  • Added null checks for second handDataProvider since calling
    UpdateHandInputSource on Left would cause handDataProvider to be
    destroyed, so UpdateHandInputSource on the right hand would throw a null
    ref

Issues I encountered but did not solve:

• Gaze cursor behavior is wonky after swapping input profiles.  Gaze cursor
  would not become active until the first time the user air taps.

• Gaze provider sometimes falls back to gaze when swapping to eye tracking.

There's already a UIRaycastCamera in the scene. It will be ignored, so please
delete it to avoid confusion.

UnityEngine.Debug:LogError(Object, Object)
Microsoft.MixedReality.Toolkit.Input.FocusProvider:Initialize() (at
Assets/MixedRealityToolkit.Services/InputSystem/FocusProvider.cs:416)
Microsoft.MixedReality.Toolkit.MixedRealityToolkit:InitializeAllServices() (at
Assets/MixedRealityToolkit/Services/MixedRealityToolkit.cs:931)
Microsoft.MixedReality.Toolkit.MixedRealityToolkit:InitializeServiceLocator()
(at Assets/MixedRealityToolkit/Services/MixedRealityToolkit.cs:477)
Microsoft.MixedReality.Toolkit.MixedRealityToolkit:ResetConfiguration(MixedRealityToolkitConfigurationProfile)
(at Assets/MixedRealityToolkit/Services/MixedRealityToolkit.cs:116) 

 

MissingReferenceException: The object of type 'Camera' has been destroyed but
you are still trying to access it.
Your script should either check if it is null or you should not destroy the
object.
Microsoft.MixedReality.Toolkit.Input.MixedRealityInputModule.GetMousePointerEventDataForMrtkPointer
(Microsoft.MixedReality.Toolkit.Input.MixedRealityInputModule+PointerData
pointerData) (at
Assets/MixedRealityToolkit.Services/InputSystem/MixedRealityInputModule.cs:176)
Microsoft.MixedReality.Toolkit.Input.MixedRealityInputModule.GetMousePointerEventData
(System.Int32 pointerId) (at
Assets/MixedRealityToolkit.Services/InputSystem/MixedRealityInputModule.cs:164)
UnityEngine.EventSystems.StandaloneInputModule.ProcessMouseEvent (System.Int32
id) (at
C:/buildslave/unity/build/Extensions/guisystem/UnityEngine.UI/EventSystem/InputModules/StandaloneInputModule.cs:504)
Microsoft.MixedReality.Toolkit.Input.MixedRealityInputModule.Process () (at
Assets/MixedRealityToolkit.Services/InputSystem/MixedRealityInputModule.cs:117)
UnityEngine.EventSystems.EventSystem.Update () (at
C:/buildslave/unity/build/Extensions/guisystem/UnityEngine.UI/EventSystem/EventSystem.cs:377)

[chbecker-ms]

Comment from Julia:

Seems like an issue that has provided significant pain for customer in the past

[wiwei]

At this point, I don't think that we can get this in for this iteration, so I'm removing this tag

[david-c-kline]

I have fixes for the invalidoperationexception when the collection changes as well as a nullreferenceexception caused by the gazeProviderPointingData being null after the profile is changed.

Continuing to look at the other issues as they are encountered.

[david-c-kline]

Added a fix for the ui raycast camera not being property recreated.

Encountering an issue where the DefaultCursor is being duplicated when switching profiles

[david-c-kline]

Moving to the 2.2.0 release.

Current status:

• Partial fix for #4289 (change active profile at runtime) #6043 contains a partial fix that eliminates the exceptions (ex: invalid operation due to collection changes) encountered when changing profiles.
• After talking the changes in Partial fix for #4289 (change active profile at runtime) #6043, switching between profiles can result in two default cursor instances (Changing ActiveProfile duplicates DefaultCursor object in hierarchy #6041)
• More thorough test coverage needs to be performed, including writing regression tests

[michalek-marcin]

@davidkline-ms Will this fix be available in MRTK 2.4.0? PR is still open. When do you plan to release new MRTK version?

[stale]

This issue has been marked as stale by an automated process because it has not had any recent activity. It will be automatically closed in 30 days if no further activity occurs. If this is still an issue please add a new comment with more recent details and repro steps.

[waldobronchart]

It would be great if this was put on the plan for the next release!