Do we need a simple on/off occlusion solution for a credible "AR min-bar"?

[thetuvix]

We discussed at the January F2F that the hit-testing sample in immersive-web/webxr#493 always prefers hitting the virtual object vs. world geometry due to not having a WebXR occlusion solution, meaning virtual content will always end up on top of walls anyway. However, it doesn't sound like AR apps will produce intuitive experiences for users if things always end up available through walls.

Does this suggest that perhaps a simple on/off occlusion solution is needed to meet the credible "AR min-bar" for WebXR?

Originally posted by @thetuvix in immersive-web/webxr#493

[toji]

Neither of the prominent mobile AR platforms (ARCore/Kit) have any form of occlusion at this point, which I think pretty firmly indicates that it's not a min-bar requirement.

[thetuvix]

To what degree do ARCore/ARKit apps receive wall planes during practical user operation on those platforms? An app receiving wall planes on those platforms could do their own occlusion - however, we've scoped out giving apps that geometry data for 1.0. Could a more scoped version of that just enable occlusion?

[toji]

The planes exposed by those platforms are geared towards object placement and not suitable for occlusion. I guess you could technically use them as occluders but the effect would not be very stable or well fit and it would probably look worse than simply not occluding at all.

[blairmacintyre]

I think that sample code is broken. If we are going to provide an example of how to combine virtual and real hit test results, the sample code should do both hit tests, and then return the closer one.

Visual occlusion is orthogonal to this issue.

For that, I am ambivalent; current versions of ARCore/ARKit return planes that are inadequate for occlusion, but it's easy to make the argument that while inadequate, they are better than nothing.

I suppose a bigger question is "how would this be used" once we are (potentially) exposing this information to developers.

Assuming a developer has access to this geometry, it is very likely they will use it for occlusion: it's unlike a generic app will inspect the geometry to see if it's a dense mesh (like HMDs are returning) or a sparse set of planes.

If the UA does not provide the mesh to the app, where is this "simple on/off occlusion" being done?

[bricetebbs]

I agree that we could use better framing around how we expect people to use HitTest. I filed #hitest/34 about this a while ago.

The point above about the occluders not being stable is also a good one. In the experiments I've done, the initial estimates of the plane geometry are pretty lousy or bloated but they can improve over time. We don't really have any way to surface this yet.