Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MSE-in-Workers: Centrally define the window/worker HTMLMediaElement<->MSE API communication mechanism #279

Closed
wolenetz opened this issue Jun 18, 2021 · 1 comment
Closed

MSE-in-Workers: Centrally define the window/worker HTMLMediaElement<->MSE API communication mechanism #279

wolenetz opened this issue Jun 18, 2021 · 1 comment

Comments

@wolenetz
Copy link
Member

MSE-in-Workers feature issue is #175.

This issue tracks clarifying and centrally defining the communication mechanism(s) necessary for interoperable implementations MSE-in-Workers.

The initial draft spec (https://github.com/wolenetz/media-source/tree/mse-in-workers-initial-spec) has repeated large blocks of text, rather than concise reusable references, to describe the communication mechanisms in a black-box fashion. Clarity would be greatly improved, I think, if the mechanism and constraints were described in a separate section in detail in the spec.

Example of an instance that would be improved:

Set recent duration and recent live seekable range respectively to be what the duration and live seekable range were recently, with no greater delay than what would be perceived by a message listener in Window if the most recent update to duration or live seekable range had been sent from the DedicatedWorkerGlobalScope to the Window using postMessage(message, options).
Note: This allowance for delay acknowledges that the HTMLMediaElement may not have direct access to the current values of duration and live seekable range when the MediaSource is in a different execution environment. Instead of forcing all implementations to block multiple execution environments until the HTMLMediaElement can reliably retrieve the precise current values of duration and live seekable range, this allowance lets both proceed under assumptions of message passing coherency that are no worse than what is allowed to the application code running in these different execution environments.

Potential improvement:

  • On changes to live seekable range or duration, if in the worker, run New Algorithm X to communicate the change to the HTMLMediaElement in the window context.
  • New Algorithm X would use some Well-defined internal, nonexposed MessageChannel pair established at attachment time to send the change message, with a handler in the window context that would update an internal recent duration and recent live seekable range cache for the media element.
  • Add a note in New Algorithm X describing why this cache is useful and that it can lag behind the actual state in the worker.
  • Add a note in New Algorithm X indicating that implementations MAY use internal communication channels that are faster than MessageChannels for on-demand querying the current live seekable range and duration. (Chrome's experimental implementation includes this optimization with careful locking and cross-thread data sharing abstractions.)

Various other places would need similar, including some that communicate from window HTMLMediaElement to worker MSE.

Currently, the following are known places in the feature draft spec that could use this improvement:

  • Worker to Window: recent live seekable range (for seekable attribute value determination)
  • Worker to Window: recent duration (for seekable attribute value determination)
  • Worker to Window: recent intersection ranges (for HTMLMediaElement's buffered attribute value determination)
  • Worker to Window: track creation, removal, and enabled/selected/showing state changes (for populating window-side mirrors of the worker MSE tracks and updating the HTMLMediaElements track lists).
  • Window to Worker: track removal and enabled/selected/showing state changes involving window-side mirrors of the worker MSE tracks (for updating MSE tracks, tracklists and activeSourceBuffers list changes).
  • Window to Worker: HTMLMediaElement error occurrence
wolenetz added a commit to wolenetz/media-source that referenced this issue Jun 18, 2021
@wolenetz
MSE-in-Workers: Planning w3c#277, w3c#278, w3c#279
fc435e9
@wolenetz
Copy link
Member Author

Steps in the [=attaching to a media element=] will need similar clarification.

wolenetz added a commit to wolenetz/media-source that referenced this issue Jun 30, 2021
@wolenetz
MSE-in-Workers: Planning w3c#277, w3c#278, w3c#279
1a2b401
wolenetz added a commit to wolenetz/media-source that referenced this issue Jul 26, 2021
@wolenetz
MSE-in-Workers: draft feature specification
077e233 
Specifies expansion of MSE API to usage from DedicatedWorker contexts.

This is an MSEv2 feature tracked by issue w3c#175.

See also earlier WICG explainer [1].

This is a squashed commit of 23 original commits, notably:
* Adds `closing` issue and link to w3c#276.
* Adds the `github:` respecConfig key and an issues-summary appendix.
* Removes redundant 'Repository' otherLinks config (addition of 'github'
  to respecConfig has make the manual enumeration in the 'Repository'
  otherLinks portion of respecConfig redundant.)
* Adds subsections for each of the extended HTMLMediaElement's .seekable
  and .buffered attributes.
* Adds more normative detail to the .buffered extension.
* Specifies what .seekable and .buffered do when worker has terminated,
  and references w3c#277 for further discussion.
* Describes cross-context track object aliasing. AudioTrack, VideoTrack,
  TextTrack extensions' sourceBuffer attribute must be null in the
  Window context's alias of the track if the track is an alias to a
  worker-created track from MSE-in-Worker. This is to prevent any
  assumption that cross-context object references are somehow now
  allowed in MSE or HTMLMediaElement+MSE.
* Describes a MessageChannel-based cross-context communication mechanism
  in a new section, including how MessagePorts on that channel are given
  to each side (media element, MediaSource) for communicating state
  needed by algorithms. Updates multiple algorithms and methods to use
  this mechanism where necessary. Resolves w3c#279.
* Specifies how to detect this feature using new
  `canConstructInDedicatedWorker` attribute without requiring the
  detection happening in a DedicatedWorker context.
* Modernizes to use respec's new variable syntax (`|...|` with types
  inlined)) for related lines. Wider modernization is an editorial item
  that is out of scope of this feature.
* Modernizes to use internal slots for the new cross-context
  communication mechanism's channel and ports. Wider modernization is an
  editorial item out of scope of this feature.o

Note, multiple simultaneous changes to track's `selected`, `enabled` or
`hidden`/`showing` states could be in-flight from window to worker, and
from worker to window. A non-normative note might be good to add here to
warn API users of this possibility. See w3c#278.

Note, {Audio,Video,Track}{,List} IDL need to be exposed in
DedicatedWorker. This might need to be directly in the media element
spec. See w3c#280.

Note, a reference to an MSE-in-Worker example/demo would be good in
the merged PR. Such a demo exists already at [2].
[1] https://github.com/wicg/media-source/blob/mse-in-workers-using-handle/mse-in-workers-using-handle-explainer.md
[2] https://wolenetz.github.io/mse-in-workers-demo/mse-in-workers-demo.html
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@wolenetz