Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: videoWorklet [worklets-1] #905

Closed
guest271314 opened this issue Jun 21, 2019 · 1 comment

Comments

@guest271314
Copy link

commented Jun 21, 2019

Given a single or list of URL's videoWorklet does not render media but rather, similar to Web Audio OfflineAudioContext startRendering() https://webaudio.github.io/web-audio-api/#OfflineAudioContext

does not render to the audio hardware, but instead renders as quickly as possible

essentially the opposite of paintWorklet, provides the images to main thread or another context (Worker, SharedWorker, ServiceWorker, window, etc.) as an array of ImageBitmap OR MediaStreams.

Consider the capabilities and functionality of HTML <video> element - without having to load an entire HTML document to access the underlying Media Decoder, Media Encoder, Web Media Player C++ code implemented by browsers (at least Firefox and Chromium) in a Worklet.

Similar concepts and art

for the ability to create media such as https://creativecommons.org/about/videos/a-shared-culture and https://mirrors.creativecommons.org/movingimages/webm/ScienceCommonsJesseDylan_240p.webm (a video or image collage) using only the browser - without third-party code.

A rough-draft proof of concept that does not meet the specific requirement though provides a tentative model of how controlling a <video> element in a Worklet could resemble, using JavaScript (Ecmascript specification) Module exported and imported https://plnkr.co/edit/5bvp9xv0ciMYfVzG?p=preview.

The videoWorklet can be stateless though still have the ability to send and receive messages.

Ideally, since we should not have to wait for the video to be played at a frame rate for the user to view, the images encoded in the media file (any container or codec the browser is already capable of parsing) should be able to be extracted faster than real-time (at least faster than playback and capturing a MediaStream from a DOM element and MediaRecorder).

@tabatkins

This comment has been minimized.

Copy link
Member

commented Jun 26, 2019

Ok, so it sounds like you're trying to solve a more general problem, of having the ability to manipulate video streams without needing to instantiate them in a video element.

In that case, it seems like you've already got this well-handled by the existing issues on HTML, and/or the other API proposals you listed. This isn't a Houdini issue; it's not about CSS at all.

(This is why I said that Worklets are not a feature in and of themselves; they're a tool used by other specs that need a specific kind of separate-from-the-main-document JS context.)

@tabatkins tabatkins closed this Jun 26, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.