Redesign OpenXR rendering integration

[jdm]

Based on the changes that @pcwalton is making in https://github.com/pcwalton/rust-offscreen-rendering-context/tree/io-surface-22 and https://github.com/pcwalton/servo/tree/texture-sharing-iosurface, here is the proposed new design:

• the webxr openxr backend creates a NativeSurface, and the webgl thread creates a NativeSurface - these are the two main surfaces that are used for double-buffering.
• When it's time to render, the webxr openxr backend will extract a dxgi ShareHandle from the single surface it possesses, then perform a GPU blit from that d3d texture to the appropriate OpenXR texture
• The WebGL/WebXR integration is responsible for passing a new surface that contains a complete frame to the webxr device's frame rendering method, and obtaining the previous frame's surface from that device so it can be used in the webgl thread
• An XRWebGLLayer's framebuffer will not be backed by an actual framebuffer object (at least in EGL). Instead it will be backed by a NativeSurface and will be associated with the currently active framebuffer (0).
• The webxr crate will need to depend upon rust-offscreen-rendering-context to use its NativeSurface types

[asajeffrey]

This may require changes to the webxr API, so it may impact all devices, not just OpenXR.

[asajeffrey]

IRC chat with @pcwalton https://mozilla.logbot.info/servo/20190910#c16605658

[asajeffrey]

In the middle of doing this, there are a few changes to the outline design:

• Both surfaces are created by the webgl thread, as part of a swap chain.
• WebXR devices will get passed a surfman Surface rather than a native surface; most devices will then use surfman to get a GL texture out of the surface which they will render using GL; openxr will do it's own thing.
• XRWebGLLayers are backed by a swap chain rather than a surface.

The current state of play is https://github.com/asajeffrey/servo/tree/surfman-webxr

[asajeffrey]

It's going to be a lot easier to do this if webxr depends not just on surfman, but also the SwapChains implementation, that is currently part of servo+surfman. I ripped out swap chains from servo and made it a crate, so the dependencies are now surfman -> swap-chains -> webxr -> servo. The current draft is https://github.com/asajeffrey/swap-chains

[jdm]

With https://github.com/jdm/rust-offscreen-rendering-context/tree/d3d-share and https://github.com/jdm/webxr/tree/no-gl-readback used on top of servo/servo#24482, I have successfully rendered the webvr rollercoaster in immersive mode in the hololens emulator. The only downside so far is that the image is upside down.

[Manishearth]

Nice!

[Manishearth]

Why is RORC still necessary?

[Manishearth]

Wait never mind I see what you did there

[jdm]

With the latest commits in those branches, along with building the tip of https://github.com/google/angle/ for UWP and fixing a couple broken constants in the Servo UWP embedding as a result, I have successfully blitted the webxr framebuffer to the openxr textures with the correct orientation and no intermediary steps!

[jdm]

With #79 and pcwalton/surfman#32 we have non-readback immersive mode in openxr.

[jdm]

We achieved the redesign.