UI Pointer ignores Order by Layer

[vcbnm]

Environment

• Source of VRTK: Github
• Version of VRTK: d88c9ea
• Version of Unity3D: 2018.1.4f1
• Hardware used: HTC Vive Pro
• SDK used: SteamVR v1.2.3 (Unity Assets Store)

Steps to reproduce

1. Copy the contents of scene Example 007 and place it in a new scene and save it.
2. Duplicate the gameobject Canvas at least two times in UnityUITestStation/UICanvas.
3. Starting from the canvas highest in the hierarchy, set the Order in Layer value of the canvas component of the original and duplicate canvas objects to respectively 100, 50 and 10.
4. Select the canvas objects with Order in Layer value 50 and 10 and move them along their local Z axis so they'll appear behind the original canvas with layer 100. Keep a distance of about half a unit between them.
5. Make sure that Graphic raycaster on canvas is set to Blocking Objects: All and Blocking Mask: Everything.
6. Play the scene and aim the UI Pointer towards the Scrolly buttons on the canvas with order in layer 100. Make sure that the scrollviews align.

Expected behavior

The UI Pointer would hit the front most canvas, allowing clicks on the buttons. The pointer renderer hits the same canvas. Placing the canvas with 100 behind the others would still force the UI Pointer to pick that one over the others according to the solution specified in #982 and #1609.

Current behavior

The UI Pointer hits the elements of the last canvas instead of the canvas closest to the controller.
The pointer renderer hits the front most canvas. The canvas with layer 100 always renders on top, even when put behind the other canvas but is not hit by the UI Pointer.

[fuboss]

Yeah! Totally vote for this

[Kinggrass]

Bug still exist, child reorder or reorder in general does not fix it.
The behaviour should be the same as the VRTK_UICanvas component is not applied. So like Unity is handling the order.

To Reproduce:
Open [007 - Interactions] InteractionHelpers and duplicate UICanvas set the Z-Position of UICanvas (1) to 0.235
Try to interact in Play Mode with UICanvas (wich is in front). you can only interact with the UI behind it.
I would suggest it is a order problem what is determined the "nearest".

This is also the case if more than 2 duplicates are made and are set behind in row.

I also wrote @bddckr in the slack channel an he only pointed me at #982 and his solution at the end which is not working. And is claiming that it works for others just fine.

[LivelyLiz]

Issue still remains

The canvas is in front of the other (not just drawn in front), Blocking Mask is in set to everything, Blocking Objects is set to all, it has a fitting box collider, the sorting layers are the same and sorting order is too. Changing the sorting order also does not fix anything.

It clearly is a Unity issue as it appears without VRTK, but this makes it totally useless to have multiple canvases in world space if they cannot be used as intended.

[wdulot]

I had the same problem and solved it this way:

• I turned on Blocking Objects = All on the canvas Graphics Raycaster
• I added a custom tracer and a custom cursor to my Straight Pointer Renderers. These custom tracer and cursor have no collider and rigidbody. (there seem to be a raycasting problem with the tracer and cursor created by default)

[LivelyLiz]

@wdulot Would you share a minimal working example or scene setup? I tried your solution but the issue still remains.
I just used a default sphere object for the cursor and a capsule for the tracer and removed the colliders from it, so only the Mesh Filter and Mesh renderer components remained. Then I set the reference in the Straight Pointer Renderer for Custom Tracer and Custom Cursor to these two objects.

Edit: Removing the Sphere Collider from the generated ObjectInteractor object kind of worked, but is no suitable solution because then the pointer can't be used for interaction with other objects. Also, it is autogenerated.

[wdulot]

@finartist I only use pointers to interact with UI so this works for me.
I made a quick test and indeed the problem comes back when I turn on "Interact with objects" as it generates other colliders...

[LivelyLiz]

@wdulot Thank you. Your solution might not help in my case but might help others :) At least, it is a new fact to consider while investigating.

[LivelyLiz]

Strange things are going on. I added a debug log to CheckRaycasts in the VRTK_VRInputModule which essentialy goes through all of the raycast results and logs their names to the console. What is the strange news? Apparently, not all canvases get raycasted per process call and that's why the pointer goes right through it.

[LivelyLiz]

It seems, that the problem lies in the ray used to test the graphic elements in the GraphicsRaycaster (or here in the VRTK_UIGraphicsRaycaster protected Raycast method). For some reason, the ignored canvases have a ray direction of (0.0, 0.0, 0.0) which results in a distance of -infinity or infinity. The ray direction used is eventData.pointerCurrentRaycast.worldNormal.

[LivelyLiz]

So I could trace the problem down down and make it work for me. I will describe briefly, where you need to add lines of code and why the problem occurs in the first place. I think it is not a great solution and might break in some cases. Still, you might want to try this until there is a better solution.

Why does it happen?
The ray used for testing in the VRTK_UIGraphicsRaycaster Raycast call is constructed from eventData.pointerCurrentRaycast which is actually a RaycastResult but is used as a replacement to get the world space position of the controllers into the EventSystem calls. The problem is that this value is changed in the SetNearestRaycast but still used for any further calculations in every other canvas. It's easy to see that a distance calculation doesn't make any sense then.
Second problem, the Unity EventSystem sorts the graphic elements by depth first and then by distance. For a 3D environment, we want to sort it by distance, at least if the distance varies for a noticable amount. Here we could run into problems for intersecting canvases or canvases, that are very close to each other.

How to repair it

• in VRTK_UIGraphicsRaycaster look for SetNearestRaycast and comment eventData.pointerCurrentRaycast = nearestRaycast.Value; in line 59
  This will prevent the ray.direction from being a zero vector.
• in VRTK_VRInputModule go to CheckRaycasts after the line 56, add the following

raycasts.Sort((res1, res2) =>
{
    // use depth, if the canvas is the same and therefore distance is small
    // you might need to tweak the 0.001 for your case
    if (Mathf.Abs(res1.distance - res2.distance) < 0.001)
    {
        return res2.depth.CompareTo(res1.depth);
    }
    return res1.distance.CompareTo(res2.distance);
});

if(raycasts.Count > 0)
{
    pointer.pointerEventData.pointerCurrentRaycast = raycasts[0];
}

With this, you still get the current raycast target because the nearest one is the first in the list.

I still have issues with images that should block underlying elements but at least it does not shoot through canvases anymore. I haven't tested it throughoutly yet so there might still be issues with it.

[rockwalrus]

@finartist's fix worked for me, without doing the custom sort.

[thestonefox]

Closing this as it's related to an older version of VRTK (v3.3) the master branch is now on VRTK v4. Feel free to request to reopen if the issue is still present.