babylon.js demos have broken XR rendering

[jdm]

./mach run http://paulrouget.com/test/bbjs/basic/ --pref dom.webxr.enabled --pref dom.webxr.glwindow
The scene renders fine in non-immersive mode, and looks totally messed up in immersive mode. Also moving the camera in immersive mode leaves trails on the screen.

[jdm]

It looks like it's not doing the stereoscopic rendering, and the frames aren't being cleared.

[jdm]

Current theory - the base WebGL canvas passes stencil: true when creating the context, but the code to create an XR webgl layer does not:

var engine = new BABYLON.Engine(canvas, true, { preserveDrawingBuffer: false, stencil: true });

    WebXRManagedOutputCanvas.prototype.initializeXRLayerAsync = function (xrSession) {
        var _this = this;
        // support canvases without makeXRCompatible
        if (!this.canvasContext.makeXRCompatible) {
            this.xrLayer = new XRWebGLLayer(xrSession, this.canvasContext);
            return Promise.resolve(true);
        }
        return this.canvasContext.makeXRCompatible().then(function () {
            _this.xrLayer = new XRWebGLLayer(xrSession, _this.canvasContext);
        });
    };

dictionary XRWebGLLayerInit {
  boolean antialias = true;
  boolean depth = true;
  boolean stencil = false;
  boolean alpha = true;
  // double framebufferScaleFactor = 1.0;
};

[jdm]

Another theory - preserveDrawingBuffer: false only clears the default webgl framebuffer (and there are khronos conformance tests that verify this), but the XR webgl layer is a special opaque framebuffer and won't get cleared.

[asajeffrey]

In servo+surfman, the opaque framebuffer gets cleared each time the buffers are swapped (which is a bug because it ignores preserveDrawingBuffer but that should still Do The Right Thing for webxr).

[jdm]

Other possible issues with frame clearing:

The pixel ownership test, the scissor test, dithering, and the buffer writemasks affect the operation of glClear. The scissor box bounds the cleared region. Alpha function, blend function, logical operation, stenciling, texture mapping, and depth-buffering are ignored by glClear. 

• https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glClear.xml

The current clearing implementation only resets the clear color/depth/stencil values, but it sounds like we need to be more thorough.

[jdm]

https://joshmatthews.net/bbjsrender.mov

Something about the default camera position in immersive mode ends up starting off with a fully black render; I need to hold down the arrow keys for a few seconds to orient it to show anything else. As the video shows, there's something really weird going on in the rendering.

[jdm]

I can avoid the blackness by commenting out https://github.com/paulrouget/paulrouget.github.com/blob/109f67050461b5eb122d2604e0cbc9562f05b3d0/test/bbjs/basic/index.js#L55. Based on the way the view of the scene appears to move in a spherical way when I press the arrow keys, whereas in three.js demos it shifts in a very linear way, I suspect at least part of the problem is some matrix math setting up the camera view from the XR pose information isn't working correctly.

[jdm]

WebXRCamera.prototype.updateFromXRSessionManager is the bit of code that controls that.

[jdm]

I'm having a hard time sorting out what I can trust. Firefox runs https://www.babylonjs-playground.com/frame.html#JA1ND3#161 in immersive mode just fine with the webxr emulator installed, but it totally fails to render http://paulrouget.com/test/bbjs/mansion/, which suggests that there's some change that's been made which might be important to know about.

[jdm]

Similarly http://paulrouget.com/test/bbjs/basic/ in Firefox doesn't render correctly - the background appears, but nothing else, and there are some very odd glitches at the very start that disappear quickly.

I saved the default project from https://www.babylonjs-playground.com/frame.html and added the scene.createDefaultXRExperienceAsync(); bit to get the XR integration. This ran immersively in Firefox, but gave me a black screen in Servo. When I changed its babylon.js to use the one from http://paulrouget.com/test/bbjs/shared/babylon.max.js then I got a sphere that rendered in stereo and didn't display the terrible rendering issues from the basic demo, but there was no background. I'm not really sure what to make of that.

[jdm]

For future reference, the three.js XR frame update code is https://github.com/mrdoob/three.js/blob/db5bfed9b0220c235ac168556405732cf13f76d8/src/renderers/webvr/WebXRManager.js. They do some matrix inversion that the equivalent code in babylon does not, but I don't know if that matters. I tried replicating it in babylon and it didn't appear to make a difference.

[paulrouget]

I'm not really sure what to make of that.

Do you use the preview version: https://preview.babylonjs.com/babylon.max.js

[paulrouget]

This is what the basic demo looks like with the device: https://gfycat.com/thoughtfulfrankindianskimmer

[jdm]

asajeffrey/surfman-chains#5 makes a huge difference to the rendering quality of http://paulrouget.com/test/bbjs/basic/ in the glwindow backend. Interestingly, there are clearly now alternating frames between the expected rendering and some other view when moving the camera around. I'm not exactly sure what to make of it.

[jdm]

http://paulrouget.com/test/bbjs/mansion/ is also way more usable with the surfman-chains PR applied, with the caveat that the camera issues are even more readily apparent.

[paulrouget]

This is massive improvement!!! No trail anymore.

[jdm]

https://joshmatthews.net/bbjscamera.mov is the macOS camera rendering issues that I'm talking about.

[jdm]

I have verified by dumping the results of read_pixels in the glwindow backend that the framebuffer contains an upside down view of the same scene from a different position every few frames. I really have no idea what to make of this.

[jdm]

When I use https://servo.org/hl-home/bbjs/basic/ and set https://github.com/servo/webxr/blob/0ae9b56ef7e646c54382f75fe79012e0f889363d/webxr/glwindow/mod.rs#L118 to 0, I get meaningful rendering in Servo without any of the previous camera weirdness. Loading the same url in Firefox with the XR emulator present does not give me meaningful rendering results, so I'm not sure what to make of that.

[jdm]

Nevermind, I think I get the same rendering results with Firefox as Servo when I close the emulator panel and the rendering updates. We should add some translation controls to glwindow to make it easier to explore the babylon scenes, though.

[jdm]

This is fixed.