[HoloLens] implement immersive mode

[paulrouget]

See https://docs.microsoft.com/en-us/windows/mixed-reality/app-model#creating-an-immersive-view

[paulrouget]

https://docs.microsoft.com/en-us/windows/mixed-reality/getting-a-holographicspace

[jdm]

https://docs.microsoft.com/en-us/windows/mixed-reality/app-model#switching-from-the-2d-xaml-view-to-the-immersive-view may be useful, but it may also be too XAML-focused.

[paulrouget]

I've managed to get a XAML app that spawns an immersive view, in winrt, with Servo running in a SwapChainPanel: https://github.com/paulrouget/HLServo2

Now trying to create a OpenGL context from a CoreWindow (not foreseeing any issue).

[paulrouget]

I'm at a point where the HoloLens code is ready to render immersive mode.

I haven't look at webxr yet.

@asajeffrey I haven't looked at the recent work that has been done to hook a different GL context for the immersive mode. Can you point me to the relevant parts of Servo that I should be looking at? And maybe summarize how you think I should implement that? The GL context can run in the compositor thread if necessary, but it doesn't have to. All in the same process.

@jdm I can't find the link to the openxr rust binding you shared. Can you share again?

@Manishearth I'm planning to implement a openxr "driver" for the webxr implementation. Should I just use servo master? Where do I need to implement this driver?

FYI, I'm using https://manishearth.net/sand/webgl-to-webvr/webxr.html , and enabled dom.webxr.enabled. Is that all I need to do?

[Manishearth]

FYI, I'm using https://manishearth.net/sand/webgl-to-webvr/webxr.html , and enabled dom.webxr.enabled. Is that all I need to do?

No, that's out of date. Clone https://github.com/manishearth/webgl-to-webvr and check out the xr branch

Yeah, you should just use servo master. Patch a new backend into https://github.com/servo/webxr/.

[jdm]

@paulrouget My changes are at https://github.com/servo/rust-webvr/tree/openxr.

[asajeffrey]

@paulrouget The main things are

a) Add a new backend to servo/webxr.

b) In the hololens library, register the Hololens back end.

For the GL context, I ended up sharing the context between both the immersive display and webrender, which worked fine but it might have been better to create two contexts that share textures.

I'm about to port magicleap from the old rust-webvr to the new shiny webxr API. I should have a better idea about how things shake out once I've done that. At the moment webxr just has the test backends (headless and glwindow).

[asajeffrey]

Oh, and as part of the magicleap port, I'm making the necessary changes to libsimpleservo, which you may want for a libhlservo.

[asajeffrey]

The WIP magicleap port... asajeffrey/webxr@6ff2fea

[jdm]

Rebased @paulrouget's existing work against master with the latest upstream fixes (including #23857):

• https://github.com/jdm/webxr/tree/hololens
• https://github.com/jdm/servo/tree/xr

We now panic with a GL error at https://github.com/jdm/webxr/blob/da077788fb0d1188ca2da474ce0ffa5e1d8a4f2f/webxr/hololens/mod.rs#L138 .

[jdm]

The panic occurs after these log messages have appeared:

GL ERROR: Invalid surface
BrowserPage::RecoverFromLostDevice()
BrowserPage:StopRenderLoop()
GL ERROR: Invalid surface

[jdm]

Those GL errors go away if I add MakeCurrentXR() inside BrowserPage::FlushXr() before the swap occurs, but the panic remains.

[asajeffrey]

Hmm... That code has lots of GlWindow in it, which is the glwindow test device. Shouldn't there be some UWP code in there somewhere?

[jdm]

I believe the idea was to get the display working in a second ANGLE-based window in the app.

[asajeffrey]

Not sure how that is going to get at, e.g., the pose information, device viewports, and all the rest of the stuff that's needed for immersive AR.

[paulrouget]

For now, I'm just trying to get the webxr content to render in another UWP Angle surface. Next is to use a shared texture between a offscreen GL buffer and D3D.

[paulrouget]

Those GL errors go away if I add MakeCurrentXR() inside BrowserPage::FlushXr() before the swap occurs, but the panic remains.

The way make_current and swap_buffer are called doesn't look right.

1. make_current is called for XR
2. make_current is called for WR
3. swap_buffer is called for WR
4. swap_buffer is called for XR

[paulrouget]

@asajeffrey I'm trying to reproduce what we do for the glutin port for now. The idea is that we have 2 surfaces, one with WR, one with XR.

[asajeffrey]

@paulrouget Hmm, I suspect what's going on is that an rAF is beginning, but isn't ending promptly, so WR gets scheduled in between, I'm not sure what to do about this, perhaps move the call to make_current from requestAnimaionFrame to renderAnimationFrame?

[jdm]

We have a working immersive mode now.