Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Media player rendering with GL textures #23483

Merged
merged 31 commits into from Jul 5, 2019
Merged

Conversation

@ceyusa
Copy link
Collaborator

@ceyusa ceyusa commented May 30, 2019

These patches pass the application's OpenGL raw context and the its native display address to the media player, in order to create an internal wrapped context, thus it will generate video frames as textures.

For now only EGL from glutin-based app and android are in place, though tested only in Linux glutin app.

This PR also renders the generated frame textures by Servo/Media and renders them by using a thread that connects Webrenderer's ExternalImageHandler and each instantiated player. By now, these patches, disable the WebGL rendering. We need to provide a ExternalImageHandler demuxer.

This PR depends on servo/media#270

  • ./mach build -d does not report any errors
  • ./mach test-tidy does not report any errors
  • This PR fixes #22300 and fixes #22920

In order to test it you must launch servo as

./mach run -- --pref media.glvideo.enabled [...]


This change is Reviewable

@highfive
Copy link

@highfive highfive commented May 30, 2019

Heads up! This PR modifies the following files:

  • @asajeffrey: components/constellation/pipeline.rs, components/constellation/constellation.rs, components/script/dom/htmlmediaelement.rs, components/script/script_thread.rs, components/script/dom/window.rs
  • @cbrewster: components/constellation/pipeline.rs, components/constellation/constellation.rs
  • @paulrouget: components/constellation/pipeline.rs, ports/glutin/context.rs, components/compositing/Cargo.toml, ports/glutin/headless_window.rs, components/constellation/constellation.rs and 4 more
  • @KiChjang: components/script/dom/htmlmediaelement.rs, components/script/script_thread.rs, components/script/dom/window.rs, components/script_traits/Cargo.toml, components/script_traits/lib.rs

@highfive
Copy link

@highfive highfive commented May 30, 2019

warning Warning warning

  • These commits modify unsafe code. Please review it carefully!
  • These commits modify script code, but no tests are modified. Please consider adding a test!

@bors-servo
Copy link
Contributor

@bors-servo bors-servo commented May 31, 2019

The latest upstream changes (presumably #23485) made this pull request unmergeable. Please resolve the merge conflicts.

@ceyusa ceyusa force-pushed the player-context branch 2 times, most recently from 3e97f8a to 3e803e6 Jun 4, 2019
@bors-servo
Copy link
Contributor

@bors-servo bors-servo commented Jun 4, 2019

The latest upstream changes (presumably #23506) made this pull request unmergeable. Please resolve the merge conflicts.

@ceyusa ceyusa force-pushed the player-context branch from 3e803e6 to 3142627 Jun 5, 2019
@bors-servo
Copy link
Contributor

@bors-servo bors-servo commented Jun 5, 2019

The latest upstream changes (presumably #23517) made this pull request unmergeable. Please resolve the merge conflicts.

@jdm
Copy link
Member

@jdm jdm commented Jun 5, 2019

Is this waiting for review right now?

@ceyusa ceyusa force-pushed the player-context branch from 3142627 to 33b54b7 Jun 7, 2019
@ceyusa
Copy link
Collaborator Author

@ceyusa ceyusa commented Jun 7, 2019

Is this waiting for review right now?

Is still a working progress :(

It would be great a glance to check if the sketched ideas make senss.

@bors-servo
Copy link
Contributor

@bors-servo bors-servo commented Jun 9, 2019

The latest upstream changes (presumably #23154) made this pull request unmergeable. Please resolve the merge conflicts.

@ceyusa ceyusa force-pushed the player-context branch 2 times, most recently from d14bc2a to 2c3017a Jun 10, 2019
@bors-servo
Copy link
Contributor

@bors-servo bors-servo commented Jun 12, 2019

The latest upstream changes (presumably #23468) made this pull request unmergeable. Please resolve the merge conflicts.

@ceyusa ceyusa force-pushed the player-context branch from 2c3017a to f35b530 Jun 15, 2019
@ceyusa ceyusa changed the title [WIP] Pass GL application parameters down to media player Media player rendering with GL textures Jun 15, 2019
@ceyusa ceyusa force-pushed the player-context branch 3 times, most recently from 848f777 to 72e0941 Jun 16, 2019
@ceyusa
Copy link
Collaborator Author

@ceyusa ceyusa commented Jun 16, 2019

Ready for review :)

@ceyusa ceyusa force-pushed the player-context branch 2 times, most recently from 23e9746 to 4748553 Jun 17, 2019
@ceyusa ceyusa force-pushed the player-context branch 3 times, most recently from 1726eef to a5949a9 Jun 18, 2019
@jdm
Copy link
Member

@jdm jdm commented Jul 4, 2019

@bors-servo
Copy link
Contributor

@bors-servo bors-servo commented Jul 4, 2019

📌 Commit 5d30716 has been approved by jdm

@bors-servo
Copy link
Contributor

@bors-servo bors-servo commented Jul 4, 2019

Testing commit 5d30716 with merge 320ec43...

bors-servo added a commit that referenced this issue Jul 4, 2019
Media player rendering with GL textures

These patches pass the application's OpenGL raw context  and the its native display address to the media player, in order to create an internal wrapped context, thus it will generate video frames as textures.

For now only EGL from glutin-based app and android are in place, though tested only in Linux glutin app.

This PR also renders the generated frame textures by Servo/Media and renders them by using a thread that connects Webrenderer's ExternalImageHandler and each instantiated player. **By now, these patches, disable the WebGL rendering**. We need to provide a ExternalImageHandler demuxer.

This PR depends on servo/media#270

- [X]  `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
- This PR fixes #22300 and fixes #22920

In order to test it you must launch servo as

`./mach run -- --pref media.glvideo.enabled [...]`

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/23483)
<!-- Reviewable:end -->
@bors-servo
Copy link
Contributor

@bors-servo bors-servo commented Jul 5, 2019

💔 Test failed - status-taskcluster

@CYBAI
Copy link
Member

@CYBAI CYBAI commented Jul 5, 2019

In magicleap build:

error[E0560]: struct `simpleservo::InitOptions` has no field named `gl_context_points`
   --> ports/libmlservo/src/lib.rs:160:9
    |
160 |         gl_context_points: Some(gl.gl_context),
    |         ^^^^^^^^^^^^^^^^^ help: a field with a similar name exists: `gl_context_pointer`

error[E0560]: struct `simpleservo::InitOptions` has no field named `display`
   --> ports/libmlservo/src/lib.rs:161:9
    |
161 |         display: Some(gl.display),
    |         ^^^^^^^ `simpleservo::InitOptions` does not have this field
    |
    = note: available fields are: `args`, `url`, `coordinates`, `density`, `vr_init` ... and 3 others

error: aborting due to 2 previous errors

@jdm
Copy link
Member

@jdm jdm commented Jul 5, 2019

@bors-servo
Copy link
Contributor

@bors-servo bors-servo commented Jul 5, 2019

📌 Commit 4fe1291 has been approved by jdm

@bors-servo
Copy link
Contributor

@bors-servo bors-servo commented Jul 5, 2019

Testing commit 4fe1291 with merge 0dc17af...

bors-servo added a commit that referenced this issue Jul 5, 2019
Media player rendering with GL textures

These patches pass the application's OpenGL raw context  and the its native display address to the media player, in order to create an internal wrapped context, thus it will generate video frames as textures.

For now only EGL from glutin-based app and android are in place, though tested only in Linux glutin app.

This PR also renders the generated frame textures by Servo/Media and renders them by using a thread that connects Webrenderer's ExternalImageHandler and each instantiated player. **By now, these patches, disable the WebGL rendering**. We need to provide a ExternalImageHandler demuxer.

This PR depends on servo/media#270

- [X]  `./mach build -d` does not report any errors
- [X] `./mach test-tidy` does not report any errors
- This PR fixes #22300 and fixes #22920

In order to test it you must launch servo as

`./mach run -- --pref media.glvideo.enabled [...]`

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/23483)
<!-- Reviewable:end -->
@bors-servo
Copy link
Contributor

@bors-servo bors-servo commented Jul 5, 2019

☀️ Test successful - linux-rel-css, linux-rel-wpt, status-taskcluster
Approved by: jdm
Pushing 0dc17af to master...

@bors-servo bors-servo merged commit 4fe1291 into servo:master Jul 5, 2019
3 checks passed
3 checks passed
Taskcluster (pull_request) TaskGroup: success
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
@bors-servo
homu Test successful
Details
@bors-servo bors-servo mentioned this pull request Jul 5, 2019
5 tasks
@ceyusa ceyusa deleted the player-context branch Aug 5, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

7 participants