Pointer-Interactor

[Ram1508]

@AMollis

In MRTKv2,OnPointerClicked(MixedRealityPointerEventData eventData) called when pointer is clicked, and one can get the position and rotation of pointer using eventData.

It is replaced by interactors in mrtk3, but can you tell me how to get the position and rotation of where the interactor is clicked?

[marlenaklein-msft]

Hi @Ram1508, thanks for reaching out.

If you are working with the interactable, there are events firstSelectEntered/selectEntered or lastSelectExited/selectExited which are fired when an interactor starts or stops selecting an interactable. The args include the interactor responsible.

If you are working the interactor, it has a selectEntered event, which you can add a listener to.

Getting the position and rotation may depend on which interactor you are using. The MRTKRayInteractor and GazeInteractor inherit from XRRayInteractor, which provides some functions to get information about the raycast hit. You can use TryGetHitInfo to get the position of the ray hit and normal of the surface the ray hit or TryGetCurrent3DRaycastHit. The XRI docs should provide more info about this.

Hope that this helps!

[Ram1508]

Thankyou @marlenaklein-msft for the reply.
I tried out the above steps you mentioned and below is the screenshot of that,

I am not able to receive events(postions) from the TryGetHitInfo()/TryGetCurrent3DRaycastHit().

Prerequistes:

In a scene i have added a gameobject with component 'XRRayInteractor' which has the refrence for rayinteractor[in the above screen shot]

rayinteractor.selectenter.addlistener(OnFirstselectEntered) -> not able to recieve position or normal from onfirstselectentered.

For example I tried to hit a plane with layered as 'SpatialMapping'

[marlenaklein-msft]

@Ram1508, ah sorry I misinterpreted this, the selectEntered event is specific to an Interactor selecting an Interactable. If you are working with a plane for spatial mapping I would recommend looking at the MRTK3 TapToPlace sample scene and script. I believe this should provide a similar example. Specifically the RegisterPlacementAction and SolverUpdate methods.

[hemasree99s]

@marlenaklein-msft, How should we get the Pose where a user has air tapped on the real world without the use of solvers?

[hemasree99s]

@AMollis @marlenaklein-msft @mrtk-bld , Any update on the above?

[AMollis]

@hemasree99s, you should be able to do this by querying information from Unity's interactors.

Take a look at a MRTK3's TapToPlace, and how it queries for the list of interactors.

Then take a look at how TapToPlace listens for selects (or "clicks").

Using the SelectEntered or SelectEntering events on the interactor, you will receive a SelectEnterEventArgs object. This object has the interactable that was selected and interactor that did the selecting. From this, you can query the attach point via the GetAttachPoseOnSelect method.

For the select events to work on the spatial mesh, you'll have to add an interactable component to your spatial mesh prefab (see your ARMeshManager for a reference to your spatial mesh prefab). If you don't want to add an interactable component to the spatial mesh, you'll then need another way to detect interactor's select actions and execute your own ray cast, much like MRTK3's TapToPlace component (see this and this). Interactors will only interact with interables. So if you want an interactor's intersection with a non-interactable, you'll need to do you own ray casting.

As you know MRTK3 is based on XRI. I would also recommend taking a look at Unity's XRI samples given your scenario.

[hemasree99s]

@AMollis @marlenaklein-msft
1.) I tried to add an Interactable to ARMeshManager but subscribed events didn't get fired.

2.)When I try to hit a plane with spatial mapping layer, the event subscribed with actionController.selectAction.action.performed (For eg - StopPlacementViaPerformedAction in TapToPlace ) is getting fired. Hence, I need to explicitly do a raycast inorder to find the raycast hit point.

3.)Anyways to receive the actions subscribed with selectInteractor.selectEntered (For eg- selectInteractor.selectEnteredAddListener(StopPlacementViaSelect) in TapToPlace ) for a non-interactable so that i can get the attach point via the GetAttachPoseOnSelect method.

[hemasree99s]

@AMollis @marlenaklein-msft
Any update on the above request?

[AMollis]

@hemasree99s can you please describe your scenario. What are you trying to do?

1. A sample project showing your reproduction would help here. The physics layer might be filtered out.
2. Yes, that is expected. This is exactly how the TapToPlace solver works.
3. XRI events only work with interactables. If you want to work with non-interactables, you need to do your own ray-casting. I'm curious however, what interaction scenario would require non-interactables? Whats the reason you can't add an interactable component to the object?

If you have a sample project, I can take a quick look to see what's going wrong. Do you have something to share?

[AMollis]

Also, if you have XRI questions or need help with XRI concepts. I encourage you to post in the Unity forums as well, https://forum.unity.com/forums/xr-interaction-toolkit-and-input.519/. There are experts there that might be able to help with your particular scenario.

[hemasree99s]

@AMollis, Thanks for the timely help.