Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Media player rendering with GL textures #23483

Open
wants to merge 19 commits into
base: master
from

Conversation

Projects
None yet
6 participants
@ceyusa
Copy link
Collaborator

commented May 30, 2019

These patches pass the application's OpenGL raw context and the its native display address to the media player, in order to create an internal wrapped context, thus it will generate video frames as textures.

For now only EGL from glutin-based app and android are in place, though tested only in Linux glutin app.

This PR also renders the generated frame textures by Servo/Media and renders them by using a thread that connects Webrenderer's ExternalImageHandler and each instantiated player. By now, these patches, disable the WebGL rendering. We need to provide a ExternalImageHandler demuxer.

This PR depends on servo/media#270

  • ./mach build -d does not report any errors
  • ./mach test-tidy does not report any errors
  • This PR fixes #22300 and #22920

In order to test it you must launch servo as

./mach run -- --pref media.glvideo.enabled [...]


This change is Reviewable

@highfive

This comment has been minimized.

Copy link

commented May 30, 2019

Heads up! This PR modifies the following files:

  • @asajeffrey: components/constellation/pipeline.rs, components/constellation/constellation.rs, components/script/dom/htmlmediaelement.rs, components/script/script_thread.rs, components/script/dom/window.rs
  • @cbrewster: components/constellation/pipeline.rs, components/constellation/constellation.rs
  • @paulrouget: components/constellation/pipeline.rs, ports/glutin/context.rs, components/compositing/Cargo.toml, ports/glutin/headless_window.rs, components/constellation/constellation.rs and 4 more
  • @KiChjang: components/script/dom/htmlmediaelement.rs, components/script/script_thread.rs, components/script/dom/window.rs, components/script_traits/Cargo.toml, components/script_traits/lib.rs
@highfive

This comment has been minimized.

Copy link

commented May 30, 2019

warning Warning warning

  • These commits modify unsafe code. Please review it carefully!
  • These commits modify script code, but no tests are modified. Please consider adding a test!
@bors-servo

This comment has been minimized.

Copy link
Contributor

commented May 31, 2019

☔️ The latest upstream changes (presumably #23485) made this pull request unmergeable. Please resolve the merge conflicts.

@ceyusa ceyusa force-pushed the ceyusa:player-context branch 2 times, most recently from 3e97f8a to 3e803e6 Jun 3, 2019

@bors-servo

This comment has been minimized.

Copy link
Contributor

commented Jun 4, 2019

☔️ The latest upstream changes (presumably #23506) made this pull request unmergeable. Please resolve the merge conflicts.

@ceyusa ceyusa force-pushed the ceyusa:player-context branch from 3e803e6 to 3142627 Jun 5, 2019

@bors-servo

This comment has been minimized.

Copy link
Contributor

commented Jun 5, 2019

☔️ The latest upstream changes (presumably #23517) made this pull request unmergeable. Please resolve the merge conflicts.

@jdm

This comment has been minimized.

Copy link
Member

commented Jun 5, 2019

Is this waiting for review right now?

@ceyusa ceyusa force-pushed the ceyusa:player-context branch from 3142627 to 33b54b7 Jun 7, 2019

@ceyusa

This comment has been minimized.

Copy link
Collaborator Author

commented Jun 7, 2019

Is this waiting for review right now?

Is still a working progress :(

It would be great a glance to check if the sketched ideas make senss.

@bors-servo

This comment has been minimized.

Copy link
Contributor

commented Jun 9, 2019

☔️ The latest upstream changes (presumably #23154) made this pull request unmergeable. Please resolve the merge conflicts.

@ceyusa ceyusa force-pushed the ceyusa:player-context branch 2 times, most recently from d14bc2a to 2c3017a Jun 10, 2019

@bors-servo

This comment has been minimized.

Copy link
Contributor

commented Jun 12, 2019

☔️ The latest upstream changes (presumably #23468) made this pull request unmergeable. Please resolve the merge conflicts.

@ceyusa ceyusa force-pushed the ceyusa:player-context branch from 2c3017a to f35b530 Jun 15, 2019

@ceyusa ceyusa changed the title [WIP] Pass GL application parameters down to media player Media player rendering with GL textures Jun 15, 2019

@ceyusa ceyusa force-pushed the ceyusa:player-context branch 3 times, most recently from 848f777 to 72e0941 Jun 16, 2019

@ceyusa

This comment has been minimized.

Copy link
Collaborator Author

commented Jun 16, 2019

Ready for review :)

@ceyusa ceyusa force-pushed the ceyusa:player-context branch 2 times, most recently from 23e9746 to 4748553 Jun 16, 2019

@ceyusa ceyusa removed the S-needs-rebase label Jun 17, 2019

@ceyusa ceyusa force-pushed the ceyusa:player-context branch 2 times, most recently from 2f96f36 to 1726eef Jun 18, 2019

ceyusa and others added some commits May 13, 2019

Ignore frame if it is a GL texture
And mock what would be if we handle GL textures.
Add media (WindowGLContext) module in canvas_trait
This module adds a structure (WindowGLContext) which holds the
OpenGL parameters that are going to be used by servo-media player
to render video frames using OpenGL.

In order to fill this structure, three new methods were added to
WindowMethods trait. In this patch only the Glutin-based
implementation provides a simple boilerplate.

The WindowGLContext is created in the entry point of libservo, when
the application window is created, and later passed to the
constellation, the pipeline and to the window element in dom, thus
htmlmediaelement has a mean to obtain these parameters via its
window.
Add GLPlayerMsg and glplayer_channel in canvas_traits
GLPlayerMsg enum values are going to be the commands to send to
the glplayer_thread.

glplayer_channel mod is a copy of webgl_channel.

@ceyusa ceyusa force-pushed the ceyusa:player-context branch from 1726eef to a5949a9 Jun 18, 2019

ceyusa added some commits Jun 11, 2019

Add GLPlayerThreads in canvas
This factory will launch a GLPlayerThread from the application main
thread.

And add GLPlayerThread, the multiplexor for media players' video
renderers. This thread will receive commands from htmlmedialement
and webrenderer.

This code is also inspired by webgl_threads and WebGLThread.
Pass GLPlayerThreads to constellation
Create the thread only if the GL context is known.
Add GLPlayerMsgForward enum
These are the messages which are going to be sended/forwarded to
the player.
Implement webrender::ExternalImageHandler for player
Added trait GLPlayerExternalImageApi and its implementation
Implemented webrender::ExternalImageHandler using
GLPlayerExternalImageApi

@ceyusa ceyusa force-pushed the ceyusa:player-context branch from a5949a9 to 15fabbd Jun 19, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.