Synchronize WebGL contexts with page rendering #21841

jdm · 2018-09-29T03:21:45Z

The code that deals with a integrating a shared GL context with the display list that Webrender receives doesn't work right now. Webrender has an API that allows locking the context, but the period between the end of layout (when script starts running again) and when WR actually processes the display list means that the GL context can be modified so that what WR sees is not what was supposed to be composited. This change marks GL contexts for WebGL canvases as locked as soon as they are processed by layout, causing any future WebGL commands that affect the to context to be buffered and replayed once WebRender unlocks the GL context.

./mach build -d does not report any errors
./mach test-tidy does not report any errors
These changes fix #21838
These changes do not require tests because it's nondeterministic behaviour.

This change is

highfive · 2018-09-29T03:21:48Z

Heads up! This PR modifies the following files:

@asajeffrey: components/script/dom/webglrenderingcontext.rs, components/script/dom/node.rs, components/script/dom/webgl2renderingcontext.rs, components/script/dom/htmlcanvaselement.rs
@KiChjang: components/script/dom/webglrenderingcontext.rs, components/script/dom/node.rs, components/script/dom/webgl2renderingcontext.rs, components/script/dom/htmlcanvaselement.rs
@emilio: components/layout/construct.rs, components/layout/context.rs

highfive · 2018-09-29T03:21:49Z

Warning

These commits modify unsafe code. Please review it carefully!
These commits modify layout and script code, but no tests are modified. Please consider adding a test!

jdm · 2018-09-29T03:22:13Z

@bors-servo try=wpt

bors-servo · 2018-09-29T03:22:17Z

⌛ Trying commit f693335 with merge c0c8602...


        Auto merge of #21841 - jdm:webgl-sync, r=<try>

Synchronize WebGL contexts with page rendering The code that deals with a integrating a shared GL context with the display list that Webrender receives doesn't work right now. Webrender has an API that allows locking the context, but the period between the end of layout (when script starts running again) and when WR actually processes the display list means that the GL context can be modified so that what WR sees is not what was supposed to be composited. This change marks GL contexts for WebGL canvases as locked as soon as they are processed by layout, causing any future WebGL commands that affect the to context to be buffered and replayed once WebRender unlocks the GL context. --- - [x] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [x] These changes fix #21838 - [x] These changes do not require tests because it's nondeterministic behaviour.  --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/21841)

jdm · 2018-09-29T03:29:11Z

r? @emilio
Let me know if this is more than you want to deal with and I can find somebody else.

bors-servo · 2018-09-29T03:47:38Z

💔 Test failed - linux-rel-wpt

emilio

np, I think I can review this :)

Some comment below, I think it looks reasonable, though I think I need to re-page the locking / unlocking scheme is supposed to work to see if this is sound. We can probably take advantage that we're going to see us in person on Monday to double-check this :)

Thanks!

emilio · 2018-09-29T21:15:17Z

components/canvas/webgl_thread.rs

@@ -183,6 +212,13 @@ impl<VR: WebVRRenderHandler + 'static> WebGLThread<VR> {
        self.webvr_compositor.as_mut().unwrap().handle(command, texture.map(|t| (t.texture_id, t.size)));
    }

+    /// Handles a request to prevent a GL context from being modified until further notice.
+    fn handle_prevent_modification(&mut self, context_id: WebGLContextId) {
+        if !self.buffered_contexts.contains_key(&context_id) {


nit: This can use entry instead to avoid two hash table lookups.

emilio · 2018-09-29T21:15:17Z

components/canvas/webgl_thread.rs

+        let target_id = match msg {
+            WebGLMsg::CreateContext(..) |
+            WebGLMsg::Lock(..) |
+            WebGLMsg::Unlock(..) |


We should probably assert that we don't receive lock messages while it's unlocked, or similar. Otherwise the code below in handle_unlock won't work. Though I think this is a pre-existing issue.

emilio · 2018-09-29T21:15:17Z

components/canvas/webgl_thread.rs

+        {
+            let data = Self::make_current_if_needed(context_id, &self.contexts, &mut self.bound_context_id)
+                            .expect("WebGLContext not found in a WebGLMsg::Unlock message");
+            let info = self.cached_context_info.get_mut(&context_id).unwrap();


nit: while you're touching this, using the indexing operator may give a nicer error message if it fails.

emilio · 2018-09-29T21:15:17Z

components/canvas/webgl_thread.rs

@@ -64,6 +65,9 @@ pub struct WebGLThread<VR: WebVRRenderHandler + 'static> {
    webvr_compositor: Option<VR>,
    /// Texture ids and sizes used in DOM to texture outputs.
    dom_outputs: FnvHashMap<webrender_api::PipelineId, DOMToTextureData>,
+    /// Tracking GL contexts that must not be modified but can still be
+    /// targeted by WebGL commands.


This should document that this map only has keys for contexts that are currently locked, and for which a PreventModification message has been sent.

emilio · 2018-09-29T21:15:17Z

components/layout_thread/lib.rs

+            guards.clone(),
+            true,
+            data.reflow_goal.needs_display(),
+            &map);


nit: paren on the next line.

emilio · 2018-09-29T21:15:17Z

components/layout_thread/lib.rs

+                guards,
+                false,
+                goal.needs_display(),
+                &snapshots


nit: I think rustfmt would add a trailing comma here.

emilio · 2018-09-29T21:15:17Z

components/canvas/webgl_thread.rs

@@ -183,6 +212,13 @@ impl<VR: WebVRRenderHandler + 'static> WebGLThread<VR> {
        self.webvr_compositor.as_mut().unwrap().handle(command, texture.map(|t| (t.texture_id, t.size)));
    }

+    /// Handles a request to prevent a GL context from being modified until further notice.
+    fn handle_prevent_modification(&mut self, context_id: WebGLContextId) {


Should this assert that the context is locked? Otherwise there's no guarantee that the context would be unlocked, right?

emilio · 2018-10-01T10:12:44Z

Another thing to check is whether WebRender is guaranteed to give us an unlock() message. I'd expect it to not do so if the image is culled (because the element is off-screen for example).

MortimerGoro · 2018-10-02T14:27:17Z

As an optimization we could start the msg buffering only when a Draw operation arrives (e.g. glDrawElements, glDrawArrays or glClear). It should be safe to run other methods.

bors-servo · 2018-10-09T11:15:47Z

☔ The latest upstream changes (presumably #21543) made this pull request unmergeable. Please resolve the merge conflicts.

jdm · 2018-10-23T17:26:45Z

The concern about webrender only calling lock/unlock when the canvas is rendered onscreen is real. Firefox uses a queue of shared textures (instead of one shared context) to ensure that it can continue performing webgl operations while webrender will always see consistent texture contents. To avoid completely rewriting our webgl setup, I'm going to try delaying all webgl command execution until the webrender image handler lock is called; the webgl thread can buffer all commands and group them by frames. For a canvas with preserveDrawingBuffer: false, we can discard all buffered frames except the most recent complete one, otherwise we will need to execute all of them.

jdm · 2018-11-09T19:18:16Z

This runs into problems where JS will block on compositing. I'm going to try using textures like Gecko instead of a single offscreen framebuffer so we can just tell webrender "this is the frame; draw it" and keep working on a separate texture.

MortimerGoro · 2018-11-09T19:37:44Z

That's a good idea. We are also going to need render to texture (SurfaceTexture) for WebVR integration with FxR.


        Use local offscreen_gl_context.


        Remove offscreen_gl_context dependency from canvas_traits and script.


        webgl: Swap the current draw buffer every time a webgl canvas is mark…

…ed dirty. Refactors the WebGL thread and canvas rendering context to reflect the fact that WebRender image keys are immutable. Any readback that is necessary occurs right before the draw buffer is swapped.

jdm · 2018-11-22T18:23:58Z

Huh. In a release build on my pixel 2, there is way more flickering with these changes than without them. That's... unexpected.

jdm · 2018-11-23T22:07:26Z

This is definitely a huge improvement in debug builds, at least.

jdm · 2018-11-26T17:00:50Z

@bors-servo try=wpt

bors-servo · 2018-11-26T17:00:56Z

⌛ Trying commit 3c229f1 with merge 40dec74...


        Auto merge of #21841 - jdm:webgl-sync, r=<try>

Synchronize WebGL contexts with page rendering The code that deals with a integrating a shared GL context with the display list that Webrender receives doesn't work right now. Webrender has an API that allows locking the context, but the period between the end of layout (when script starts running again) and when WR actually processes the display list means that the GL context can be modified so that what WR sees is not what was supposed to be composited. This change marks GL contexts for WebGL canvases as locked as soon as they are processed by layout, causing any future WebGL commands that affect the to context to be buffered and replayed once WebRender unlocks the GL context. --- - [x] `./mach build -d` does not report any errors - [x] `./mach test-tidy` does not report any errors - [x] These changes fix #21838 - [x] These changes do not require tests because it's nondeterministic behaviour.  --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/21841)

bors-servo · 2018-11-26T17:01:14Z

💔 Test failed - linux-rel-css

jdm · 2018-11-26T17:50:05Z

Current theory about the pixel 2 brokenness:

WR locks the external image and fetches the texture id for the complete frame
the canvas thread processes the lock message and returns the current texture id for the complete frame
the canvas thread receives a message to present the current frame and swaps the textures, resetting the previously complete texture to black
WR draws the texture id that has now been reset to black

More specifically, the locking mechanism currently implemented is broken by the switch to use multiple textures. The swapping code should only run when the texture is unlocked; otherwise we should just reset the current frame and skip displaying it, I suppose.

jdm · 2018-11-29T20:20:48Z

The flickering is not helped by dealing with the issue described in the previous comment. It is helped by commenting out the call to reset_draw_buffer_contents in servo/surfman#131.

bors-servo · 2018-12-31T19:39:39Z

☔ The latest upstream changes (presumably #22385) made this pull request unmergeable. Please resolve the merge conflicts.

jdm · 2019-11-07T13:59:59Z

No longer necessary with the introduction of #24482!

highfive assigned avadacatavra Sep 29, 2018

highfive added the S-awaiting-review label Sep 29, 2018

highfive assigned emilio and unassigned avadacatavra Sep 29, 2018

highfive added the S-tests-failed label Sep 29, 2018

emilio reviewed Sep 29, 2018

View changes

highfive added the S-needs-rebase label Oct 9, 2018

This was referenced Nov 21, 2018

Support multiple textures servo/surfman#130

Closed

Support swappable textures as the backing for the draw buffer. servo/surfman#131

Closed

jdm force-pushed the jdm:webgl-sync branch from f693335 to cf78efe Nov 22, 2018

highfive removed the S-tests-failed label Nov 22, 2018

jdm added 3 commits Nov 14, 2018

Use local offscreen_gl_context.

e3fd037

Remove offscreen_gl_context dependency from canvas_traits and script.

486cacf

jdm force-pushed the jdm:webgl-sync branch from cf78efe to 3c229f1 Nov 22, 2018

jdm added the S-blocked-on-external label Nov 22, 2018

jdm removed the S-needs-rebase label Nov 22, 2018

highfive added the S-tests-failed label Nov 26, 2018

jdm added this to In progress in WebGL content Nov 30, 2018

highfive added the S-needs-rebase label Dec 31, 2018

jdm added this to In progress in High priority WebGL work Jan 31, 2019

jdm mentioned this pull request Feb 20, 2019

WebGL cloth demo does not display anything on magic leap debug build #22748

Closed

jdm closed this Nov 7, 2019

jdm deleted the jdm:webgl-sync branch Nov 7, 2019

jdm moved this from In progress to Done in WebGL content Nov 20, 2019

jdm moved this from In progress to Done in High priority WebGL work Nov 20, 2019

servo / servo

Synchronize WebGL contexts with page rendering #21841

Synchronize WebGL contexts with page rendering #21841

jdm commented Sep 29, 2018 •

edited by SimonSapin

highfive commented Sep 29, 2018

highfive commented Sep 29, 2018

jdm commented Sep 29, 2018

bors-servo commented Sep 29, 2018

jdm commented Sep 29, 2018

bors-servo commented Sep 29, 2018

emilio left a comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

emilio commented Oct 1, 2018

MortimerGoro commented Oct 2, 2018

bors-servo commented Oct 9, 2018

jdm commented Oct 23, 2018

jdm commented Nov 9, 2018

MortimerGoro commented Nov 9, 2018

jdm commented Nov 22, 2018

jdm commented Nov 23, 2018

jdm commented Nov 26, 2018

bors-servo commented Nov 26, 2018

bors-servo commented Nov 26, 2018

jdm commented Nov 26, 2018

jdm commented Nov 29, 2018

bors-servo commented Dec 31, 2018

jdm commented Nov 7, 2019

servo / servo

Join GitHub today

Synchronize WebGL contexts with page rendering #21841

Synchronize WebGL contexts with page rendering #21841

Conversation

jdm commented Sep 29, 2018 • edited by SimonSapin

highfive commented Sep 29, 2018

highfive commented Sep 29, 2018

jdm commented Sep 29, 2018

bors-servo commented Sep 29, 2018

jdm commented Sep 29, 2018

bors-servo commented Sep 29, 2018

emilio left a comment

This comment has been minimized.

emilio Sep 29, 2018

This comment has been minimized.

emilio Sep 29, 2018

This comment has been minimized.

emilio Sep 29, 2018

This comment has been minimized.

emilio Sep 29, 2018

This comment has been minimized.

emilio Sep 29, 2018

This comment has been minimized.

emilio Sep 29, 2018

This comment has been minimized.

emilio Sep 29, 2018

emilio commented Oct 1, 2018

MortimerGoro commented Oct 2, 2018

bors-servo commented Oct 9, 2018

jdm commented Oct 23, 2018

jdm commented Nov 9, 2018

MortimerGoro commented Nov 9, 2018

jdm commented Nov 22, 2018

jdm commented Nov 23, 2018

jdm commented Nov 26, 2018

bors-servo commented Nov 26, 2018

bors-servo commented Nov 26, 2018

jdm commented Nov 26, 2018

jdm commented Nov 29, 2018

bors-servo commented Dec 31, 2018

jdm commented Nov 7, 2019

jdm commented Sep 29, 2018 •

edited by SimonSapin