MixedRealityInputModule throws KeyNotFoundException from OnPointerUp

[matthejo]

Describe the bug

In Remote Assist, we sometimes see a KeyNotFoundException thrown from MixedRealityInputModule.OnPointerUp. After this exception is thrown, input stops working for the remainder of the app session due to unhandled NullReferenceExceptions being thrown from

To reproduce

Reproduction is difficult, but I believe it is necessary to be holding a pressed hand state with a far-interaction pointer when the hand goes out of scope (OnSourceLost).

Expected behavior

There should not be KeyNotFoundExceptions thrown from this callstack.

Your setup (please complete the following information)

• Unity Version 2018.4.16f1
• MRTK Version 2.2.20191003.3

Target platform (please complete the following information)

• HoloLens 2

Additional context

Based on my debugging, I believe that the following sequence of events are happening:

1. An OnSourceLost event occurs, probably due to a hand going out of view.
   2/ The MixedRealityInputModule's OnSourceLost handler handles the event first, and removes the pointer data from its Dictionary
2. The LinePointer gets disabled from its OnSourceLost handler (BaseControllerPointer.OnDisable). When IsSelectPressed is true, the BaseControllerPointer causes an immediate raising of the OnPointerUp event.
3. The MixedRealityInputModule's OnPointerUp handler handles the event, and asserts that the pointer must be in its pointer data collection. However, the pointer has just been removed from the OnSourceLost execution sequence. MixedRealityInputModule throws a KeyNotFoundException, preventing the remainder of the OnSourceLost stack from continue to execute
4. Subsequent frames have to deal with objects that were not cleaned up correctly, and continue to throw NullReferenceExceptions every frame

[matthejo]

Here are the relevant sequences of callstacks from the Unity log, starting with the first unhandled exception. Some frames are unavailable due to .NET Native compilation and inlining.

KeyNotFoundException: Arg_KeyNotFound
at Microsoft.MixedReality.Toolkit.Input.MixedRealityInputModule.Microsoft.MixedReality.Toolkit.Input.IMixedRealityPointerHandler.OnPointerUp(MixedReality.Toolkit.Input.MixedRealityPointerEventData eventData)
at Microsoft.MixedReality.Toolkit.Input.MixedRealityInputSystem.<>c.<.cctor>b__196_14(MixedReality.Toolkit.Input.IMixedRealityPointerHandler handler, EventSystems.BaseEventData eventData)
at RemoteAssist!+0x1be9f13
at RemoteAssist!+0x1c8104f
at RemoteAssist!+0x1d2bbe7
at Microsoft.MixedReality.Toolkit.Input.MixedRealityInputSystem.RaisePointerUp(MixedReality.Toolkit.Input.IMixedRealityPointer pointer, MixedReality.Toolkit.Input.MixedRealityInputAction inputAction, MixedReality.Toolkit.Utilities.Handedness handedness, MixedReality.Toolkit.Input.IMixedRealityInputSource inputSource)
at Microsoft.MixedReality.Toolkit.Input.BaseControllerPointer.OnDisable()
at Microsoft.MixedReality.Toolkit.Input.LinePointer.OnDisable()
at Microsoft.MixedReality.Toolkit.Input.LinePointer.$Invoke4(Int64 instance, Int64* args)
at UnityEngine.Internal.$MethodUtility.InvokeMethod(Int64 instance, Int64* args, IntPtr method)
(Filename: Line: 0)

KeyNotFoundException: Arg_KeyNotFound
at Microsoft.MixedReality.Toolkit.Input.MixedRealityInputModule.Microsoft.MixedReality.Toolkit.Input.IMixedRealityPointerHandler.OnPointerUp(MixedReality.Toolkit.Input.MixedRealityPointerEventData eventData)
at Microsoft.MixedReality.Toolkit.Input.MixedRealityInputSystem.<>c.<.cctor>b__196_14(MixedReality.Toolkit.Input.IMixedRealityPointerHandler handler, EventSystems.BaseEventData eventData)
at RemoteAssist!+0x1be9f13
at RemoteAssist!+0x1c8104f
at RemoteAssist!+0x1d2bbe7
at Microsoft.MixedReality.Toolkit.Input.MixedRealityInputSystem.RaisePointerUp(MixedReality.Toolkit.Input.IMixedRealityPointer pointer, MixedReality.Toolkit.Input.MixedRealityInputAction inputAction, MixedReality.Toolkit.Utilities.Handedness handedness, MixedReality.Toolkit.Input.IMixedRealityInputSource inputSource)
at Microsoft.MixedReality.Toolkit.Input.BaseControllerPointer.OnSourceLost(MixedReality.Toolkit.Input.SourceStateEventData eventData)
at Microsoft.MixedReality.Toolkit.Input.MixedRealityInputSystem.<>c.<.cctor>b__196_1(MixedReality.Toolkit.Input.IMixedRealitySourceStateHandler handler, EventSystems.BaseEventData eventData)
at RemoteAssist!+0x1be9f13
at RemoteAssist!+0x1c8104f
at RemoteAssist!+0x1c84313
at RemoteAssist!+0x1c83fe3
at Microsoft.MixedReality.Toolkit.Input.MixedRealityInputSystem.RaiseSourceLost(MixedReality.Toolkit.Input.IMixedRealityInputSource source, MixedReality.Toolkit.Input.IMixedRealityController controller)
at Microsoft.MixedReality.Toolkit.WindowsMixedReality.Input.WindowsMixedRealityDeviceManager.RemoveController(XR.WSA.Input.InteractionSource interactionSource)
at Microsoft.MixedReality.Toolkit.WindowsMixedReality.Input.WindowsMixedRealityDeviceManager.InteractionManager_InteractionSourceLost(XR.WSA.Input.InteractionSourceLostEventArgs args)
at System.Action<UnityEngine.XR.WSA.Input.InteractionSourceDetectedEventArgs>.Invoke(XR.WSA.Input.InteractionSourceDetectedEventArgs obj)
at RemoteAssist!+0x21fe903
at RemoteAssist!+0x21fe839
at UnityEngine.Internal.$MethodUtility.InvokeMethod(Int64 instance, Int64* args, IntPtr method)
(Filename: Line: 0)

Mixed Reality Controller Right was never registered with the Input Manager!

(Filename: C:\buildslave\unity\build\Runtime/Export/Debug.bindings.h Line: 45)

KeyNotFoundException: Arg_KeyNotFound
at Microsoft.MixedReality.Toolkit.Input.MixedRealityInputModule.Microsoft.MixedReality.Toolkit.Input.IMixedRealityPointerHandler.OnPointerUp(MixedReality.Toolkit.Input.MixedRealityPointerEventData eventData)
at Microsoft.MixedReality.Toolkit.Input.MixedRealityInputSystem.<>c.<.cctor>b__196_14(MixedReality.Toolkit.Input.IMixedRealityPointerHandler handler, EventSystems.BaseEventData eventData)
at RemoteAssist!+0x1be9f13
at RemoteAssist!+0x1c8104f
at RemoteAssist!+0x1d2bbe7
at Microsoft.MixedReality.Toolkit.Input.MixedRealityInputSystem.RaisePointerUp(MixedReality.Toolkit.Input.IMixedRealityPointer pointer, MixedReality.Toolkit.Input.MixedRealityInputAction inputAction, MixedReality.Toolkit.Utilities.Handedness handedness, MixedReality.Toolkit.Input.IMixedRealityInputSource inputSource)
at Microsoft.MixedReality.Toolkit.Input.BaseControllerPointer.OnSourceLost(MixedReality.Toolkit.Input.SourceStateEventData eventData)
at Microsoft.MixedReality.Toolkit.Input.MixedRealityInputSystem.<>c.<.cctor>b__196_1(MixedReality.Toolkit.Input.IMixedRealitySourceStateHandler handler, EventSystems.BaseEventData eventData)
at RemoteAssist!+0x1be9f13
at RemoteAssist!+0x1c8104f
at RemoteAssist!+0x1c84313
at RemoteAssist!+0x1c83fe3
at Microsoft.MixedReality.Toolkit.Input.MixedRealityInputSystem.RaiseSourceLost(MixedReality.Toolkit.Input.IMixedRealityInputSource source, MixedReality.Toolkit.Input.IMixedRealityController controller)
at Microsoft.MixedReality.Toolkit.WindowsMixedReality.Input.WindowsMixedRealityDeviceManager.UpdateInteractionManagerReading()
at Microsoft.MixedReality.Toolkit.WindowsMixedReality.Input.WindowsMixedRealityDeviceManager.Update()
at Microsoft.MixedReality.Toolkit.BaseDataProviderAccessCoreSystem.Update()
at Microsoft.MixedReality.Toolkit.MixedRealityToolkit.<>c.b__60_0(MixedReality.Toolkit.IMixedRealityService service)
at Microsoft.MixedReality.Toolkit.MixedRealityToolkit.ExecuteOnAllServices(Collections.Generic.IEnumerable<Microsoft.MixedReality.Toolkit.IMixedRealityService> services, Action<Microsoft.MixedReality.Toolkit.IMixedRealityService> execute)
at Microsoft.MixedReality.Toolkit.MixedRealityToolkit.UpdateAllServices()
at Microsoft.MixedReality.Toolkit.MixedRealityToolkit.$Invoke57(Int64 instance, Int64* args)
at UnityEngine.Internal.$MethodUtility.InvokeMethod(Int64 instance, Int64* args, IntPtr method)
(Filename: Line: 0)

Mixed Reality Controller Right was never registered with the Input Manager!

(Filename: C:\buildslave\unity\build\Runtime/Export/Debug.bindings.h Line: 45)

NullReferenceException:
at UnityEngineProxy.InternalCalls.Component_Get_Custom_PropGameObject(Object self)
at Microsoft.MixedReality.Toolkit.Input.BaseHandVisualizer.Microsoft.MixedReality.Toolkit.Input.IMixedRealitySourceStateHandler.OnSourceLost(MixedReality.Toolkit.Input.SourceStateEventData eventData)
at Microsoft.MixedReality.Toolkit.Input.MixedRealityInputSystem.<>c.<.cctor>b__196_1(MixedReality.Toolkit.Input.IMixedRealitySourceStateHandler handler, EventSystems.BaseEventData eventData)
at RemoteAssist!+0x1be9f13
at RemoteAssist!+0x1c8104f
at RemoteAssist!+0x1c84313
at RemoteAssist!+0x1c83fe3
at Microsoft.MixedReality.Toolkit.Input.MixedRealityInputSystem.RaiseSourceLost(MixedReality.Toolkit.Input.IMixedRealityInputSource source, MixedReality.Toolkit.Input.IMixedRealityController controller)
at Microsoft.MixedReality.Toolkit.WindowsMixedReality.Input.WindowsMixedRealityDeviceManager.UpdateInteractionManagerReading()
at Microsoft.MixedReality.Toolkit.WindowsMixedReality.Input.WindowsMixedRealityDeviceManager.Update()
at Microsoft.MixedReality.Toolkit.BaseDataProviderAccessCoreSystem.Update()
at Microsoft.MixedReality.Toolkit.MixedRealityToolkit.<>c.b__60_0(MixedReality.Toolkit.IMixedRealityService service)
at Microsoft.MixedReality.Toolkit.MixedRealityToolkit.ExecuteOnAllServices(Collections.Generic.IEnumerable<Microsoft.MixedReality.Toolkit.IMixedRealityService> services, Action<Microsoft.MixedReality.Toolkit.IMixedRealityService> execute)
at Microsoft.MixedReality.Toolkit.MixedRealityToolkit.UpdateAllServices()
at Microsoft.MixedReality.Toolkit.MixedRealityToolkit.$Invoke57(Int64 instance, Int64* args)
at UnityEngine.Internal.$MethodUtility.InvokeMethod(Int64 instance, Int64* args, IntPtr method)

[RichLogan]

Sorry to be commenting on an old and closed issue but I believe I'm running into this, or a very similar issue, on MRTK v2.4.0:

System.Collections.Generic.Dictionary`2[TKey,TValue].get_Item (TKey key) (at <00000000000000000000000000000000>:0)
Microsoft.MixedReality.Toolkit.Input.MixedRealityInputModule.Microsoft.MixedReality.Toolkit.Input.IMixedRealityPointerHandler.OnPointerUp (Microsoft.MixedReality.Toolkit.Input.MixedRealityPointerEventData eventData) (at <00000000000000000000000000000000>:0)
Microsoft.MixedReality.Toolkit.Input.MixedRealityInputSystem+<>c.<.cctor>b__240_14 (Microsoft.MixedReality.Toolkit.Input.IMixedRealityPointerHandler handler, UnityEngine.EventSystems.BaseEventData eventData) (at <00000000000000000000000000000000>:0)
System.Data.DataColumnChangeEventHandler.Invoke (System.Object sender, System.Data.DataColumnChangeEventArgs e) (at <00000000000000000000000000000000>:0)
Microsoft.MixedReality.Toolkit.BaseEventSystem.HandleEvent[T] (UnityEngine.EventSystems.BaseEventData eventData, UnityEngine.EventSystems.ExecuteEvents+EventFunction`1[T1] eventHandler) (at <00000000000000000000000000000000>:0)
Microsoft.MixedReality.Toolkit.Input.MixedRealityInputSystem.DispatchEventToGlobalListeners[T] (Microsoft.MixedReality.Toolkit.Input.BaseInputEventData baseInputEventData, UnityEngine.EventSystems.ExecuteEvents+EventFunction`1[T1] eventHandler) (at <00000000000000000000000000000000>:0)
Microsoft.MixedReality.Toolkit.Input.MixedRealityInputSystem.HandlePointerEvent[T] (UnityEngine.EventSystems.BaseEventData eventData, UnityEngine.EventSystems.ExecuteEvents+EventFunction`1[T1] eventHandler) (at <00000000000000000000000000000000>:0)
Microsoft.MixedReality.Toolkit.Input.MixedRealityInputSystem.RaisePointerUp (Microsoft.MixedReality.Toolkit.Input.IMixedRealityPointer pointer, Microsoft.MixedReality.Toolkit.Input.MixedRealityInputAction inputAction, Microsoft.MixedReality.Toolkit.Utilities.Handedness handedness, Microsoft.MixedReality.Toolkit.Input.IMixedRealityInputSource inputSource) (at <00000000000000000000000000000000>:0)
Microsoft.MixedReality.Toolkit.Input.BaseControllerPointer.OnSourceLost (Microsoft.MixedReality.Toolkit.Input.SourceStateEventData eventData) (at <00000000000000000000000000000000>:0)
Microsoft.MixedReality.Toolkit.Input.MixedRealityInputSystem+<>c.<.cctor>b__240_1 (Microsoft.MixedReality.Toolkit.Input.IMixedRealitySourceStateHandler handler, UnityEngine.EventSystems.BaseEventData eventData) (at <00000000000000000000000000000000>:0)
System.Data.DataColumnChangeEventHandler.Invoke (System.Object sender, System.Data.DataColumnChangeEventArgs e) (at <00000000000000000000000000000000>:0)
Microsoft.MixedReality.Toolkit.BaseEventSystem.HandleEvent[T] (UnityEngine.EventSystems.BaseEventData eventData, UnityEngine.EventSystems.ExecuteEvents+EventFunction`1[T1] eventHandler) (at <00000000000000000000000000000000>:0)
Microsoft.MixedReality.Toolkit.Input.MixedRealityInputSystem.DispatchEventToGlobalListeners[T] (Microsoft.MixedReality.Toolkit.Input.BaseInputEventData baseInputEventData, UnityEngine.EventSystems.ExecuteEvents+EventFunction`1[T1] eventHandler) (at <00000000000000000000000000000000>:0)
Microsoft.MixedReality.Toolkit.Input.MixedRealityInputSystem.HandleEvent[T] (UnityEngine.EventSystems.BaseEventData eventData, UnityEngine.EventSystems.ExecuteEvents+EventFunction`1[T1] eventHandler) (at <00000000000000000000000000000000>:0)
Microsoft.MixedReality.Toolkit.Input.MixedRealityInputSystem.RaiseSourceLost (Microsoft.MixedReality.Toolkit.Input.IMixedRealityInputSource source, Microsoft.MixedReality.Toolkit.Input.IMixedRealityController controller) (at <00000000000000000000000000000000>:0)
Microsoft.MixedReality.Toolkit.WindowsMixedReality.Input.WindowsMixedRealityDeviceManager.RemoveController (UnityEngine.XR.WSA.Input.InteractionSource interactionSource) (at <00000000000000000000000000000000>:0)
Microsoft.MixedReality.Toolkit.WindowsMixedReality.Input.WindowsMixedRealityDeviceManager.InteractionManager_InteractionSourceLost (UnityEngine.XR.WSA.Input.InteractionSourceLostEventArgs args) (at <00000000000000000000000000000000>:0)
System.Action`1[T].Invoke (T obj) (at <00000000000000000000000000000000>:0)
UnityEngine.XR.WSA.Input.InteractionManager.OnSourceEvent (UnityEngine.XR.WSA.Input.InteractionManager+EventType eventType, System.IntPtr statePtr, UnityEngine.XR.WSA.Input.InteractionSourcePressType pressType) (at <00000000000000000000000000000000>:0)
UnityEngine.Debug:LogException(Exception)
Microsoft.MixedReality.Toolkit.BaseEventSystem:HandleEvent(BaseEventData, EventFunction`1)
Microsoft.MixedReality.Toolkit.Input.MixedRealityInputSystem:DispatchEventToGlobalListeners(BaseInputEventData, EventFunction`1)
Microsoft.MixedReality.Toolkit.Input.MixedRealityInputSystem:HandlePointerEvent(BaseEventData, EventFunction`1)
Microsoft.MixedReality.Toolkit.Input.MixedRealityInputSystem:RaisePointerUp(IMixedRealityPointer, MixedRealityInputAction, Handedness, IMixedRealityInputSource)
Microsoft.MixedReality.Toolkit.Input.BaseControllerPointer:OnSourceLost(SourceStateEventData)
Microsoft.MixedReality.Toolkit.Input.<>c:<.cctor>b__240_1(IMixedRealitySourceStateHandler, BaseEventData)
System.Data.DataColumnChangeEventHandler:Invoke(Object, DataColumnChangeEventArgs)
Microsoft.MixedReality.Toolkit.BaseEventSystem:HandleEvent(BaseEventData, EventFunction`1)
Microsoft.MixedReality.Toolkit.Input.MixedRealityInputSystem:DispatchEventToGlobalListeners(BaseInputEventData, EventFunction`1)
Microsoft.MixedReality.Toolkit.Input.MixedRealityInputSystem:HandleEvent(BaseEventData, EventFunction`1)
Microsoft.MixedReality.Toolkit.Input.MixedRealityInputSystem:RaiseSourceLost(IMixedRealityInputSource, IMixedRealityController)
Microsoft.MixedReality.Toolkit.WindowsMixedReality.Input.WindowsMixedRealityDeviceManager:RemoveController(InteractionSource)
Microsoft.MixedReality.Toolkit.WindowsMixedReality.Input.WindowsMixedRealityDeviceManager:InteractionManager_InteractionSourceLost(InteractionSourceLostEventArgs)
UnityEngine.XR.WSA.Input.InteractionManager:OnSourceEvent(EventType, IntPtr, InteractionSourcePressType)

[SuriyaSwami13]

Do we have an update if this is resolved? Or any other workaround?