Use of Mirrors Broken on Oculus Quest (Discussion: a feature request for a mirror component in A-Frame)

[PlumCantaloupe]

This might be a an Oculus device issue, or maybe an issue with the XR renderer/camera implementation in three.js ( mrdoob/three.js#19666 ) as it works in Firefox Desktop with the Quest in Link mode but does not work in either the Oculus Browser or Firefox Reality in standalone mode. However, I figured this area is likely the best area to discuss it. I am not exactly sure when this stopped working, though I suspect it was the jump to the WebXR API, but mirrors appear to broken in the two ways.

• This mirror only appears in one eye (seems to be right eye consistently). This just started happening on the most recent Oculus Browser update. Doesn't appear to happen in Firefox Reality.
• When using another camera to render the mirror's texture (RenderTarget) the main camera breaks and will no longer reveal appropriate world matrices and will no longer allow elements to be parented to it (further information on this issue here: Attaching object to camera does not work in Oculus Quest #4626 ).

Hopefully someone more familiar with the underlying systems might see the trivial errors I have made, or perhaps some workarounds as I am using mirrors in my VR research :D

These are the warnings (no errors) I am seeing in the console on the Oculus Browser:

I have created two example scenarios to showcase:

• This one I have been fiddling with, building off other works (three.js and aalavandhaann ):
  ---> https://glitch.com/~aframe-mirror-and-avatar
• And a simpler one that is a more direct implementation of three.js's reflector component:
  ---> https://glitch.com/~aframe-threejs-reflector - not sure why the head texture is black here but I don't think it is related.

Though I understand that the use of secondary cameras for special effects is more of a fringe case it could be indicative of a larger issue. Also, as important as embodiment is for connecting to avatars, and the use of virtual mirrors to create a greater sense of embodiment, I would love to see and help support a "reflector" being an included component in A-Frame if others agree.

• A-Frame Version: 1.0.4
• Platform / Device: Oculus Quest using Oculus Browser or Firefox Reality (Oculus Link works with Firefox Desktop)
• Reproducible Code Snippet or URL: https://glitch.com/~aframe-mirror-and-avatar and https://glitch.com/~aframe-threejs-reflector

[dmarcos]

A mirror component would be probably better as a 3rd party / community one. We can reconsider if there's more demand.

[PlumCantaloupe]

Okay, thank you. Will keep poking at this and see if I can gauge interest somehow.

Is it possible that this issue is browser / WebXR spec. related @Artyom17?

I created a pure three.js example and noted it here (appears to have similarly strange issues): mrdoob/three.js#19666 (comment)

Glitch example: https://glitch.com/~pure-threejs-mirrortest

[dmarcos]

I would have to look into it. Not an answer to your question but a possible alternative would be to have a copy of the avatar model in front of the user and apply headset and controllers pose to them. Simpler and also better performance than rendering the scene multiple times.

[PlumCantaloupe]

Thanks for the “Mario64” suggestion @dmarcos :) I might have to do something like that if I am still stuck later in the summer.

I think my next attempt will be looking at the mirror code itself and comparing to how shadow-mapping cams render depth. Maybe there is something there that could help ...

[dmarcos]

I think there's nothing actionable on A-Frame side. If there's any fix to be done has to happen on the THREE side. Closing but we can reopen if necessary.

[PlumCantaloupe]

Just commenting here to connect all the dots that this has been resolved on the THREE end ( mrdoob/three.js#19759 ); but is still on issue on A-frame oddly ( #4806 (comment) ).

I will see tonight if I can build A-Frame with a non-super-three version of THREE to see if this will fix it to narrow down where this happening ...

[PlumCantaloupe]

For any stragglers, finally found a workaround here: #4806 (comment)

[diarmidmackenzie]

An implementation of mirrors for A-Frame is available here:
https://github.com/diarmidmackenzie/aframe-multi-camera

Specifically:
https://diarmidmackenzie.github.io/aframe-multi-camera/examples/mirror-example.html

Currently they work great on desktop, but they are heavily pixellated in VR (on Oculus Quest 2, at least), and I haven't yet understood why, but some ideas for further investigation.