Add test infrastructure for WebVR reftests #22840

asajeffrey · 2019-02-06T20:26:30Z

Adds a dom.webvr.test pref, which puts WebVR content into fullscreen mode, so we can use our shiny new fullscreen reftesting apparatus for writing WebVR reftests.

./mach build -d does not report any errors
./mach test-tidy does not report any errors
These changes fix Support a test WebVR/XR display #22795 and fix Add reftests for WebVR/XR #22804
There are tests for these changes

This change is

highfive · 2019-02-06T20:26:37Z

Heads up! This PR modifies the following files:

@KiChjang: components/script/dom/vrdisplay.rs
@emilio: components/layout/display_list/items.rs, components/style/properties/longhands/box.mako.rs, components/layout/display_list/builder.rs, components/style/properties/properties.mako.rs

highfive · 2019-02-06T20:26:39Z

Warning

These commits modify unsafe code. Please review it carefully!

asajeffrey · 2019-02-06T20:27:32Z

This PR is based on #22825 so should land after it does.

tests/wpt/mozilla/tests/mozilla/webvr/reftests/webvr-baseline.html

asajeffrey · 2019-02-06T21:21:25Z

As @jdm said on IRC (https://mozilla.logbot.info/servo/20190206#c15932019) we should get feedback from @paulrouget about whether we should enter fullscreen + use the browser's rAF for all devices without an external display, or whether we should add another property to the rust-webvr
API.

bors-servo · 2019-02-06T22:05:54Z

☔ The latest upstream changes (presumably #22825) made this pull request unmergeable. Please resolve the merge conflicts.

asajeffrey · 2019-02-06T22:09:12Z

Rebased. #22825 has landed, so this can land too, once @paulrouget has commented.

asajeffrey · 2019-02-06T22:12:29Z

Also cc @kearwood.

bors-servo · 2019-02-07T06:43:02Z

☔ The latest upstream changes (presumably #22773) made this pull request unmergeable. Please resolve the merge conflicts.

asajeffrey · 2019-02-07T15:15:53Z

IRC conversation with @paulrouget at https://mozilla.logbot.info/servo/20190207#c15934602

TL;DR: We should update rust-webvr to have three kinds of displays: external, internal-using-webrender, and internal-without-webrender. That last kind is used by oculusvr and googlevr.

… responsible for presenting

asajeffrey · 2019-02-07T18:16:15Z

@paul: Added a flag to rust-webvr over in servo/rust-webvr#49, and used it here to decide whether or not to do presentation inside servo.

components/webvr/webvr_thread.rs

components/script/dom/vrdisplay.rs

paulrouget · 2019-02-08T09:12:34Z

components/script/dom/vrdisplay.rs

        let js_sender = self.global().script_chan();
        let address = Trusted::new(&*self);
        let near_init = self.depth_near.get();
        let far_init = self.depth_far.get();
        let pipeline_id = self.global().pipeline_id();

+        // For displays with no external display, we use the regular window
+        // rAF, so we don't need to spin up a dedicated thread.


How hard would it be to make the TestDisplay sync_poses behave in a way that we would not need to have this specific code path? I'm asking because the test will cover less code by re-using the window's rAF.

IRC conversation: https://mozilla.logbot.info/servo/20190208#c15940619

components/script/dom/vrdisplay.rs

asajeffrey

Addressed most of the comments, just got sync_poses to deal with.

bors-servo · 2019-02-10T23:36:11Z

☔ The latest upstream changes (presumably #22861) made this pull request unmergeable. Please resolve the merge conflicts.

paulrouget · 2019-02-11T11:26:44Z

Here are some questions:

How will we handle a canvas that is not part of the DOM?
What if another JS code triggers fullscreen mode?
Beside the fact the code path doesn't exist yet, is there another reason to not bypass WebRender? (WebRender adds a little bit of overhead)
Do you run Servo's compositor in the main thread? If so, can you move it in a separated thread and in immersive mode, run WebVR code in the main thread?

This general approach feels a bit like a work-around to me. And ML and FxR share a similar setup, so whatever we end-up doing for ML, we will do the same for FxR.

I think your approach might be incompatible with 2 parts of a longer term plan:

use WebGL texture directly and pause Servo's compositor
move RAF/VR calls to a WebVR process/thread, with shared texture (SurfaceTexture)

I'll be happy to r+ this as long as that we agree that it might be a temporary approach until we manage to make WebGL draw directly on a shared surface texture.

MortimerGoro · 2019-02-11T14:44:31Z

I talked with Paul this morning. Adding some feedback/questions:

Using fullscreen approach may no be WebXR compliant. The API allows to use a canvas not added to the DOM, also offscreencanvas. What happens in those situations? We may hack it to add the canvas to the DOM but that would change the parentNode properties, or trigger other events which may not be spec compliant (Dom Listeners, fullscreenchanges). cc @kearwood and @Manishearth may have an opinion about this)
I'm concerned about the extra blit required by waiting for the Webrender output. IMO the goal should be to reduce the latency of the render path as much as possible. Right now we have 1 extra blit in the oculusvr/google vr platforms: WebGL texture <-> SDK Eye Buffer FBO. By using Webrender output I think we'd have 2 blits: WebGL texture <-> WR Surface <-> SDK Eye Buffer FBO. The ideal situation should be 0 extra blits. For example on Oculus, we can do direct render to a SurfaceTexture swapChain, making WebGL render directly to a Surface provided by the SDK.

We should update rust-webvr to have three kinds of displays: external, internal-using-webrender, and internal-without-webrender. That last kind is used by oculusvr and googlevr.

I agree IF we can't avoid the Magic Leap main thread limitations. Other platforms should not be affected by this. For Firefox Reality backends I think the best render path would be to use the render target provided by the SDK (e.g. the SurfaceTexture swapChain from Oculus) and make WebGL render directly to that Surface, bypassing all the extra blits required on other situations.

asajeffrey · 2019-02-11T17:00:11Z

The goal of minimizing the overhead of each rAF loop seems like a good one to me. AFAICT there's two sources of overhead:

Unecessary blit'ing. I'm confused as to why using WR requires an extra blit for devices like MagicLeap. WR is rendering into the surface provided by the embedder, so there isn't a separate WR Surface <-> SDK Eye Buffer blit.
WR. There is overhead from running WR, e.g. WR has to wait for layout to complete before rendering, even in cases like fullscreen or immersive mode where the DOM isn't contributing to the scene..
Context switching between threads. (Run WebVR content without context switching? #22855)

I'm not sure what the trade-off is between these, e.g. how much of a bottleneck there is in waiting for WR.

MortimerGoro · 2019-02-15T14:47:18Z

Unecessary blit'ing. I'm confused as to why using WR requires an extra blit for devices like MagicLeap. WR is rendering into the surface provided by the embedder, so there isn't a separate WR Surface <-> SDK Eye Buffer blit.`

The ideal path would be that WebGL renders to a surface provided by the VR SDK (e.g a swapChain on Oculus). That would be 0 blits.

Adding WR would add a extra blit in that situation., WebGL renders to a texture, and the the texture is blit on the WR surface.

For the 2 blits WebGL texture <-> WR Surface <-> SDK Eye Buffer I was thinking on platforms that don't allow creating swapChain to an Android Surface, and the "WebView" lives in another process from the Main app that renders the VR UI (e.g. GeckoView lives in another process in Firefox Reality)

paulrouget · 2019-11-04T14:25:32Z

Should we keep this PR open?

asajeffrey · 2019-11-04T14:56:04Z

This PR can close, but there may still be an issue around WebXR ref tests.@Manishearth, are there any WebXR ref tests, or are they all tests of the JS APIs?

Manishearth · 2019-11-04T15:06:33Z

JS APIs only. A Webxr reftest would be interesting.

highfive assigned avadacatavra Feb 6, 2019

highfive added the S-awaiting-review There is new code that needs to be reviewed. label Feb 6, 2019

asajeffrey added A-testing A-webvr S-blocked-on-external Something, somewhere else, needs to happen before this PR can be merged. and removed S-awaiting-review There is new code that needs to be reviewed. labels Feb 6, 2019

jdm approved these changes Feb 6, 2019

View reviewed changes

tests/wpt/mozilla/tests/mozilla/webvr/reftests/webvr-baseline.html Outdated Show resolved Hide resolved

highfive added the S-awaiting-review There is new code that needs to be reviewed. label Feb 6, 2019

highfive added the S-needs-rebase There are merge conflict errors. label Feb 6, 2019

asajeffrey force-pushed the script-webvr-reftest branch from ede4822 to 336769d Compare February 6, 2019 22:07

asajeffrey assigned paulrouget and unassigned avadacatavra Feb 6, 2019

Alan Jeffrey added 2 commits February 7, 2019 11:08

Implemented a test WebVR device that uses fullscreen mode

75a940c

Added a WebVR reftest

da78317

asajeffrey force-pushed the script-webvr-reftest branch from 336769d to da78317 Compare February 7, 2019 17:09

Use a new VRDisplay flag to indicate if the display or the browser is…

260cc08

… responsible for presenting

asajeffrey mentioned this pull request Feb 7, 2019

Add flag for devices that expect the browser to present the VR content servo/rust-webvr#49

Merged

asajeffrey removed the S-needs-rebase There are merge conflict errors. label Feb 7, 2019

Updated to rust-webvr 0.10

9cd138d

asajeffrey added the S-needs-squash Some (or all) of the commits in the PR should be combined. label Feb 7, 2019

asajeffrey removed the S-blocked-on-external Something, somewhere else, needs to happen before this PR can be merged. label Feb 7, 2019

Pass test-tidy

ba3b16d

paulrouget reviewed Feb 8, 2019

View reviewed changes

Responding to review comments

ba022a5

asajeffrey commented Feb 8, 2019

View reviewed changes

highfive added the S-needs-rebase There are merge conflict errors. label Feb 10, 2019

paulrouget closed this Nov 5, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add test infrastructure for WebVR reftests #22840

Add test infrastructure for WebVR reftests #22840

asajeffrey commented Feb 6, 2019 •

edited by atouchet

highfive commented Feb 6, 2019

highfive commented Feb 6, 2019

asajeffrey commented Feb 6, 2019

asajeffrey commented Feb 6, 2019

bors-servo commented Feb 6, 2019

asajeffrey commented Feb 6, 2019

asajeffrey commented Feb 6, 2019

bors-servo commented Feb 7, 2019

asajeffrey commented Feb 7, 2019

asajeffrey commented Feb 7, 2019

paulrouget Feb 8, 2019

asajeffrey Feb 8, 2019

asajeffrey left a comment

bors-servo commented Feb 10, 2019

paulrouget commented Feb 11, 2019

MortimerGoro commented Feb 11, 2019 •

edited

asajeffrey commented Feb 11, 2019

MortimerGoro commented Feb 15, 2019 •

edited by paulrouget

paulrouget commented Nov 4, 2019

asajeffrey commented Nov 4, 2019

Manishearth commented Nov 4, 2019

Add test infrastructure for WebVR reftests #22840

Add test infrastructure for WebVR reftests #22840

Conversation

asajeffrey commented Feb 6, 2019 • edited by atouchet

highfive commented Feb 6, 2019

highfive commented Feb 6, 2019

asajeffrey commented Feb 6, 2019

asajeffrey commented Feb 6, 2019

bors-servo commented Feb 6, 2019

asajeffrey commented Feb 6, 2019

asajeffrey commented Feb 6, 2019

bors-servo commented Feb 7, 2019

asajeffrey commented Feb 7, 2019

asajeffrey commented Feb 7, 2019

paulrouget Feb 8, 2019

Choose a reason for hiding this comment

asajeffrey Feb 8, 2019

Choose a reason for hiding this comment

asajeffrey left a comment

Choose a reason for hiding this comment

bors-servo commented Feb 10, 2019

paulrouget commented Feb 11, 2019

MortimerGoro commented Feb 11, 2019 • edited

asajeffrey commented Feb 11, 2019

MortimerGoro commented Feb 15, 2019 • edited by paulrouget

paulrouget commented Nov 4, 2019

asajeffrey commented Nov 4, 2019

Manishearth commented Nov 4, 2019

asajeffrey commented Feb 6, 2019 •

edited by atouchet

MortimerGoro commented Feb 11, 2019 •

edited

MortimerGoro commented Feb 15, 2019 •

edited by paulrouget