Media player rendering with GL textures #23483

ceyusa · 2019-05-30T14:38:53Z

These patches pass the application's OpenGL raw context and the its native display address to the media player, in order to create an internal wrapped context, thus it will generate video frames as textures.

For now only EGL from glutin-based app and android are in place, though tested only in Linux glutin app.

This PR also renders the generated frame textures by Servo/Media and renders them by using a thread that connects Webrenderer's ExternalImageHandler and each instantiated player. By now, these patches, disable the WebGL rendering. We need to provide a ExternalImageHandler demuxer.

This PR depends on servo/media#270

./mach build -d does not report any errors
./mach test-tidy does not report any errors
This PR fixes #22300 and fixes #22920

In order to test it you must launch servo as

./mach run -- --pref media.glvideo.enabled [...]

This change is

highfive · 2019-05-30T14:38:59Z

Heads up! This PR modifies the following files:

@asajeffrey: components/constellation/pipeline.rs, components/constellation/constellation.rs, components/script/dom/htmlmediaelement.rs, components/script/script_thread.rs, components/script/dom/window.rs
@cbrewster: components/constellation/pipeline.rs, components/constellation/constellation.rs
@paulrouget: components/constellation/pipeline.rs, ports/glutin/context.rs, components/compositing/Cargo.toml, ports/glutin/headless_window.rs, components/constellation/constellation.rs and 4 more
@KiChjang: components/script/dom/htmlmediaelement.rs, components/script/script_thread.rs, components/script/dom/window.rs, components/script_traits/Cargo.toml, components/script_traits/lib.rs

highfive · 2019-05-30T14:39:02Z

Warning

These commits modify unsafe code. Please review it carefully!
These commits modify script code, but no tests are modified. Please consider adding a test!

bors-servo · 2019-05-31T04:48:31Z

☔ The latest upstream changes (presumably #23485) made this pull request unmergeable. Please resolve the merge conflicts.

bors-servo · 2019-06-04T22:55:18Z

☔ The latest upstream changes (presumably #23506) made this pull request unmergeable. Please resolve the merge conflicts.

bors-servo · 2019-06-05T15:33:44Z

☔ The latest upstream changes (presumably #23517) made this pull request unmergeable. Please resolve the merge conflicts.

jdm · 2019-06-05T15:37:24Z

Is this waiting for review right now?

ceyusa · 2019-06-07T07:43:44Z

Is this waiting for review right now?

Is still a working progress :(

It would be great a glance to check if the sketched ideas make senss.

bors-servo · 2019-06-09T20:42:38Z

☔ The latest upstream changes (presumably #23154) made this pull request unmergeable. Please resolve the merge conflicts.

bors-servo · 2019-06-12T00:04:08Z

☔ The latest upstream changes (presumably #23468) made this pull request unmergeable. Please resolve the merge conflicts.

ceyusa · 2019-06-16T16:09:11Z

Ready for review :)

jdm · 2019-07-04T22:07:02Z

@bors-servo r+

bors-servo · 2019-07-04T22:07:02Z

📌 Commit 5d30716 has been approved by jdm

bors-servo · 2019-07-04T22:22:51Z

⌛ Testing commit 5d30716 with merge 320ec43...


        Auto merge of #23483 - ceyusa:player-context, r=jdm

Media player rendering with GL textures These patches pass the application's OpenGL raw context and the its native display address to the media player, in order to create an internal wrapped context, thus it will generate video frames as textures. For now only EGL from glutin-based app and android are in place, though tested only in Linux glutin app. This PR also renders the generated frame textures by Servo/Media and renders them by using a thread that connects Webrenderer's ExternalImageHandler and each instantiated player. **By now, these patches, disable the WebGL rendering**. We need to provide a ExternalImageHandler demuxer. This PR depends on servo/media#270 - [X] `./mach build -d` does not report any errors - [X] `./mach test-tidy` does not report any errors - This PR fixes #22300 and fixes #22920 In order to test it you must launch servo as `./mach run -- --pref media.glvideo.enabled [...]`  --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/23483)

bors-servo · 2019-07-05T00:11:39Z

💔 Test failed - status-taskcluster

CYBAI · 2019-07-05T02:07:58Z

In magicleap build:

error[E0560]: struct `simpleservo::InitOptions` has no field named `gl_context_points`
   --> ports/libmlservo/src/lib.rs:160:9
    |
160 |         gl_context_points: Some(gl.gl_context),
    |         ^^^^^^^^^^^^^^^^^ help: a field with a similar name exists: `gl_context_pointer`

error[E0560]: struct `simpleservo::InitOptions` has no field named `display`
   --> ports/libmlservo/src/lib.rs:161:9
    |
161 |         display: Some(gl.display),
    |         ^^^^^^^ `simpleservo::InitOptions` does not have this field
    |
    = note: available fields are: `args`, `url`, `coordinates`, `density`, `vr_init` ... and 3 others

error: aborting due to 2 previous errors


        Fix libsimpleservo build


        Fix Windows build


        Fix magic leap build.

jdm · 2019-07-05T03:28:25Z

@bors-servo r+

bors-servo · 2019-07-05T03:28:26Z

📌 Commit 4fe1291 has been approved by jdm

bors-servo · 2019-07-05T03:28:30Z

⌛ Testing commit 4fe1291 with merge 0dc17af...


        Auto merge of #23483 - ceyusa:player-context, r=jdm

Media player rendering with GL textures These patches pass the application's OpenGL raw context and the its native display address to the media player, in order to create an internal wrapped context, thus it will generate video frames as textures. For now only EGL from glutin-based app and android are in place, though tested only in Linux glutin app. This PR also renders the generated frame textures by Servo/Media and renders them by using a thread that connects Webrenderer's ExternalImageHandler and each instantiated player. **By now, these patches, disable the WebGL rendering**. We need to provide a ExternalImageHandler demuxer. This PR depends on servo/media#270 - [X] `./mach build -d` does not report any errors - [X] `./mach test-tidy` does not report any errors - This PR fixes #22300 and fixes #22920 In order to test it you must launch servo as `./mach run -- --pref media.glvideo.enabled [...]`  --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/23483)

bors-servo · 2019-07-05T05:30:42Z

☀️ Test successful - linux-rel-css, linux-rel-wpt, status-taskcluster
Approved by: jdm
Pushing 0dc17af to master...

highfive assigned emilio May 30, 2019

highfive added the S-awaiting-review label May 30, 2019

highfive added the S-needs-rebase label May 31, 2019

ceyusa force-pushed the ceyusa:player-context branch 2 times, most recently from 3e97f8a to 3e803e6 Jun 3, 2019

ceyusa force-pushed the ceyusa:player-context branch from 3e803e6 to 3142627 Jun 5, 2019

ceyusa force-pushed the ceyusa:player-context branch from 3142627 to 33b54b7 Jun 7, 2019

ceyusa force-pushed the ceyusa:player-context branch 2 times, most recently from d14bc2a to 2c3017a Jun 10, 2019

ceyusa force-pushed the ceyusa:player-context branch from 2c3017a to f35b530 Jun 15, 2019

ceyusa changed the title ~~[WIP] Pass GL application parameters down to media player~~ Media player rendering with GL textures Jun 15, 2019

ceyusa force-pushed the ceyusa:player-context branch 3 times, most recently from 848f777 to 72e0941 Jun 16, 2019

ceyusa force-pushed the ceyusa:player-context branch 2 times, most recently from 23e9746 to 4748553 Jun 16, 2019

ceyusa removed the S-needs-rebase label Jun 17, 2019

ceyusa force-pushed the ceyusa:player-context branch 3 times, most recently from 1726eef to a5949a9 Jun 18, 2019

jdm force-pushed the ceyusa:player-context branch from 23e4812 to d9425b3 Jul 4, 2019

highfive removed the S-tests-failed label Jul 4, 2019

highfive added S-awaiting-merge and removed S-awaiting-review labels Jul 4, 2019

highfive added S-tests-failed and removed S-awaiting-merge labels Jul 5, 2019

ferjm and others added 3 commits Jul 4, 2019

Fix libsimpleservo build

728fdff

Fix Windows build

f3e237e

Fix magic leap build.

Loading status checks…

4fe1291

jdm force-pushed the ceyusa:player-context branch from 5d30716 to 4fe1291 Jul 5, 2019

highfive added S-awaiting-review and removed S-tests-failed labels Jul 5, 2019

highfive added S-awaiting-merge and removed S-awaiting-review labels Jul 5, 2019

bors-servo merged commit 4fe1291 into servo:master Jul 5, 2019
3 checks passed

3 checks passed

Taskcluster (pull_request) TaskGroup: success
Details

continuous-integration/appveyor/pr AppVeyor build succeeded
Details

homu Test successful
Details

highfive removed the S-awaiting-merge label Jul 5, 2019

This was referenced Jul 5, 2019

Continue message port #23637

Merged

Implement MessagePort and MessageChannel #16622

Closed

ceyusa deleted the ceyusa:player-context branch Aug 5, 2019

servo / servo

Media player rendering with GL textures #23483

Media player rendering with GL textures #23483

ceyusa commented May 30, 2019 •

edited by atouchet

highfive commented May 30, 2019

highfive commented May 30, 2019

bors-servo commented May 31, 2019

bors-servo commented Jun 4, 2019

bors-servo commented Jun 5, 2019

jdm commented Jun 5, 2019

ceyusa commented Jun 7, 2019

bors-servo commented Jun 9, 2019

bors-servo commented Jun 12, 2019

ceyusa commented Jun 16, 2019

jdm commented Jul 4, 2019

bors-servo commented Jul 4, 2019

bors-servo commented Jul 4, 2019

bors-servo commented Jul 5, 2019

CYBAI commented Jul 5, 2019

jdm commented Jul 5, 2019

bors-servo commented Jul 5, 2019

bors-servo commented Jul 5, 2019

bors-servo commented Jul 5, 2019

servo / servo

Join GitHub today

Media player rendering with GL textures #23483

Media player rendering with GL textures #23483

Conversation

ceyusa commented May 30, 2019 • edited by atouchet

highfive commented May 30, 2019

highfive commented May 30, 2019

bors-servo commented May 31, 2019

bors-servo commented Jun 4, 2019

bors-servo commented Jun 5, 2019

jdm commented Jun 5, 2019

ceyusa commented Jun 7, 2019

bors-servo commented Jun 9, 2019

bors-servo commented Jun 12, 2019

ceyusa commented Jun 16, 2019

jdm commented Jul 4, 2019

bors-servo commented Jul 4, 2019

bors-servo commented Jul 4, 2019

bors-servo commented Jul 5, 2019

CYBAI commented Jul 5, 2019

jdm commented Jul 5, 2019

bors-servo commented Jul 5, 2019

bors-servo commented Jul 5, 2019

bors-servo commented Jul 5, 2019

ceyusa commented May 30, 2019 •

edited by atouchet