Support exiting immersive mode

[jdm]

When I'm immersive mode on the HL I can open up the windows menu and press the home button to take me out of immersive mode, but Servo no longer appears to be usable when I do this. We should figure out how to support this properly.

@paulrouget Do you have time to look into this?

[jdm]

@asajeffrey is there a JS API that exits a webxr session? I wonder what happens if a page invokes that on a timer, since we don't have any input support yet.

[asajeffrey]

@jdm the relevant JS API is session.end() https://github.com/servo/servo/blob/master/components/script/dom/xrsession.rs#L395

[paulrouget]

@paulrouget Do you have time to look into this?

Yes.

[paulrouget]

@jdm I get an exception going to immersive mode:

Exception thrown: read access violation.

>	simpleservo.dll!js::InternalBarrierMethods<JSObject *>::postBarrier(JSObject * * vp, JSObject * prev, JSObject * next) Line 265	C++
 	simpleservo.dll!mozjs_sys::jsgc::{{impl}}::post_barrier(mozjs_sys::generated::root::JSObject * * v, mozjs_sys::generated::root::JSObject * prev, mozjs_sys::generated::root::JSObject * next) Line 103	Unknown
 	simpleservo.dll!mozjs_sys::jsgc::Heap<mut mozjs_sys::generated::root::JSObject*>::set<mut mozjs_sys::generated::root::JSObject*>(mozjs_sys::generated::root::JSObject * self) Line 180	Unknown
 	simpleservo.dll!mozjs_sys::jsgc::Heap<mut mozjs_sys::generated::root::JSObject*>::boxed<mut mozjs_sys::generated::root::JSObject*>() Line 169	Unknown
 	simpleservo.dll!script::dom::globalscope::GlobalScope::add_consumed_rejection(mozjs_sys::generated::root::JS::Handle<mut mozjs_sys::generated::root::JSObject*> self) Line 295	Unknown
 	simpleservo.dll!script::script_runtime::notify_about_rejected_promises::{{closure}}(closure) Line 337	Unknown
 	simpleservo.dll!script::script_runtime::notify_about_rejected_promises::{{impl}}::run_once<closure>(script::script_runtime::notify_about_rejected_promises::unhandled_rejection_event<closure> self) Line 25	Unknown
 	simpleservo.dll!script::task::{{impl}}::run_once<script::script_runtime::notify_about_rejected_promises::unhandled_rejection_event<closure>>(script::task::CancellableTask<script::script_runtime::notify_about_rejected_promises::unhandled_rejection_event<closure>> self) Line 115	Unknown
 	simpleservo.dll!script::task::{{impl}}::run_box<script::task::CancellableTask<script::script_runtime::notify_about_rejected_promises::unhandled_rejection_event<closure>>>(script::task::CancellableTask<script::script_runtime::notify_about_rejected_promises::unhandled_rejection_event<closure>> * self) Line 57	Unknown
 	simpleservo.dll!script::script_thread::ScriptThread::handle_msg_from_script(script::script_thread::MainThreadScriptMsg self) Line 1923	Unknown
 	simpleservo.dll!script::script_thread::{{impl}}::handle_msgs::{{closure}}(closure) Line 1528	Unknown
 	simpleservo.dll!script::script_thread::ScriptThread::profile_event<closure,core::option::Option<bool>>(script::script_runtime::ScriptThreadEventCategory self, core::option::Option<msg::constellation_msg::PipelineId> category, closure pipeline_id) Line 1763	Unknown
 	simpleservo.dll!script::script_thread::ScriptThread::handle_msgs() Line 1521	Unknown
 	simpleservo.dll!script::script_thread::ScriptThread::start() Line 1354	Unknown
 	simpleservo.dll!script::script_thread::{{impl}}::create::{{closure}}::{{closure}}(closure) Line 801	Unknown
 	simpleservo.dll!profile_traits::mem::ProfilerChan::run_with_memory_reporting<closure,fn(profile_traits::mem::ReportsChan) -> script::script_runtime::CommonScriptMsg,script::script_runtime::CommonScriptMsg,crossbeam_channel::channel::Sender<script::script_thread::MainThreadScriptMsg>>(closure self, alloc::string::String f, crossbeam_channel::channel::Sender<script::script_thread::MainThreadScriptMsg> reporter_name, script::script_runtime::CommonScriptMsg(*)(profile_traits::mem::ReportsChan) channel_for_reporter) Line 88	Unknown
 	simpleservo.dll!script::script_thread::{{impl}}::create::{{closure}}(closure) Line 798	Unknown
 	simpleservo.dll!std::sys_common::backtrace::__rust_begin_short_backtrace<closure,()>(closure f) Line 78	Unknown
 	simpleservo.dll!std::thread::{{impl}}::spawn_unchecked::{{closure}}::{{closure}}<closure,()>(closure) Line 471	Unknown
 	simpleservo.dll!std::panic::{{impl}}::call_once<(),closure>(std::panic::AssertUnwindSafe<closure> self) Line 316	Unknown
 	simpleservo.dll!std::panicking::try::do_call<std::panic::AssertUnwindSafe<closure>,()>(unsigned char * data) Line 296	Unknown
 	simpleservo.dll!panic_unwind::__rust_maybe_catch_panic() Line 80	Unknown
 	simpleservo.dll!std::panicking::try<(),std::panic::AssertUnwindSafe<closure>>(std::panic::AssertUnwindSafe<closure> f) Line 275	Unknown
 	simpleservo.dll!std::panic::catch_unwind<std::panic::AssertUnwindSafe<closure>,()>(std::panic::AssertUnwindSafe<closure> f) Line 394	Unknown
 	simpleservo.dll!std::thread::{{impl}}::spawn_unchecked::{{closure}}<closure,()>(closure) Line 469	Unknown
 	simpleservo.dll!core::ops::function::FnOnce::call_once<closure,()>(closure *) Line 235	Unknown
 	simpleservo.dll!alloc::boxed::{{impl}}::call_once<(),FnOnce<()>>() Line 922	Unknown
 	simpleservo.dll!std::sys::windows::thread::{{impl}}::new::thread_start() Line 47	Unknown
 	[External Code]	

[paulrouget]

@jdm which webxr demo do you use for your tests?

[jdm]

@paulrouget That's #24061. I recommend using https://www.joshmatthews.net/webgl-to-webxr/webxr-ar.html.

[paulrouget]

Any idea what I might be missing:

analog\input\xrruntime\dll\runtimelocator.cpp(94)\MixedRealityRuntime.dll!00007FFE279565DD: The system cannot find the file specified.
Msg:[Failed to locate runtime library] CallContext:[\LoaderNegotiate] 
RUST: WARN - webxr_api::registry - XR device error BackendSpecific("ERROR_INSTANCE_LOST")
ALERT: failed undefined

[jdm]

Have you installed the OpenXR package from the windows store? That needs to happen every time the emulator is restarted.

[paulrouget]

Yes I did. Still trying to figure out why it can't find openxr.

[paulrouget]

Never mind, I didn't fully install the runtime.

[paulrouget]

It is usable after leaving the immersive mode. I can navigate to a new page, scroll and click.

The problem appears to be only with the page that initiated the immersive mode.

[paulrouget]

Is requestAnimationFrame paused for 2D webgl content when switching to immersive mode?

[asajeffrey]

The 2d rAF should still be running after you entered immersive mode. This is needed for devices where the browser window is still visible when there's impressive content running, e.g. a desktop browser with an attached headset.

[paulrouget]

I could use some help here. When we leave the immersive mode, servo get stuck here:

https://github.com/servo/webxr/blob/bf4e5a541bb2fcbace61cb1a071959bab10d9c19/webxr-api/registry.rs#L157

Not in the callback, but in retain itself. That doesn't make sense to me. I'm assuming that there's nothing that could be blocking in there.

Any clue?

[jdm]

Yeah, that should be calling https://github.com/servo/webxr/blob/bf4e5a541bb2fcbace61cb1a071959bab10d9c19/webxr-api/session.rs#L250 :/

[paulrouget]

This is where it blocks: https://github.com/Ralith/openxrs/blob/71cea14e8b17eba366c579f28b11d8f135f2a398/openxr/src/session.rs#L386

[paulrouget]

What needs to work:

1. as the IM is running, go to start menu, wait 10 seconds, IM mode should still be animating
2. as the IM is running, calling, in JS, session.end(), should bring the user back to the 2D view that is still animating.
3. as the IM is running, going to the start menu, then dismissing it, should bring back the IM modem and still be animating. To try twice.
4. as the IM is running, going to the start menu, click home, should stop the IM mode. Clicking on the Servo window should bring back the page (not reload servo) and animations should still be running.
5. pinning the window should keep both IM and 2D mode running

Testing 2 demos: bbjs basic, and webvr-ar.

bbjs demo:

1. works
2. works
3. first time works, 2nd time menu gets dismissed automatically and getting a STOPPING event, followed by a full app reload
4. Full app reload
5. N/A

webxr-ar demo:

1. receive SessionStateChanged(STOPPING) for some reason. No blocking.
2. gets back to greyed out window. clicking on the window un-grey the app. RAF not called. works
3. works the first time. 2nd time getting a stopping event. RAF not called.
4. once back, window is interactive (can scroll, click buttons, start IM again) but webgl content not animating (RAF stuck shomehow) works
5. N/A

When using webvr-ar demo, when leaving the immersive mode via js:session.end(), no blocking. Going back to the page, servo is not animating.

[paulrouget]

Not getting the block issue anymore with openxr version 100.1908.0.3002

[paulrouget]

One of the issue with the RAF not being called in the webxr-ar demo was an issue with the demo itself. RAF is properly called but nothing is rendered (keeping in mind that in the case of the bbjs demo, the webgl canvas is properly updated).

[paulrouget]

Other issue with webxr-ar is that 1) viewport is not reset 2) after the IM session, it's still bound to xrLayer.frameBuffer.

[paulrouget]

webxr-ar demo updated: http://paulrouget.com/test/webxr-ar.html

[paulrouget]

With latest master + latest openxr + latest demos, "it works". Will be filing specific issues for remaining problems.