-
Notifications
You must be signed in to change notification settings - Fork 14
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Filtering for relevant configurationchange events #82
Comments
Thanks for filing this issue!
You're absolutely right. I'd like us to find a proper way to define this
The same way |
That event would usually fire when the user manually resizes the window, which is a relatively rare event, and therefore not much of a concern. But if video frames are coming in 30 or even 60 times a second, and audio about 50 times a second, continuously throughout the call, then there is a significant opportunity for CPU savings in not firing an event each time.
Throttling by a Web developer would deal with repeated events of a type that is of interest to the developer. My concern and the solution I propose deal with the inverse case - those events which the Web developer intends to simply ignore. In that case, I am concerned with the inefficiency inherent in firing needless events into JS-land. Even a handler that does nothing but a simple string comparison and a quick
If something changes N times a second, how can the UA discern that the app is not interested in exactly N notifications a second? Some apps would be. |
@eladalon1983 in #82 (comment):
Would that subscribe both to blur capability changes and to blur setting changes? What if the event handler is only interested in one of them? @beaufortfrancois in #82 (comment):
Note also that the name of a property might not be enough because the changed aspect of the property could be the capabilities and/or the setting (like the background blur setting could be and remain as
@eladalon1983 in #82 (comment):
Are there any other property than estimated frameRate which could change on every frame? Real configuration changes are usually relatively rare event, so they will not occure multiple times per second.
If nothing but frameRate changes N times a second, maybe UA can throttle frameRate. So webapps get notified on other changes in time (because no throttling is needed by done by UA) and webapps interested in frame rates on every frame can subscribe to receive frames using other means (like video frame callback or transform streams). |
Maybe these deserve different event handlers. Wdyt?
Possibly audio latency, but I've not looked deeply into that yet. Assuming for the sake of argument that it's only frameRate, I'd say that:
(The difference between 2 and 3 is that the latter applies even when we do eventually decide "this is rare enough." Consensus-building is hard enough. The less potential points of contention, the better.)
If the UA employs such a heuristic, it would make certain apps harder to write, and less accurate. For example, an app that plots the frameRate over time would now require more code, and end up less accurate due to JS-processing delays. |
How about introducing a new const stream = await navigator.mediaDevices.getUserMedia({ video: true });
const [track] = stream.getVideoTracks();
// New!
const options = {
capabilities: ['backgroundBlur'],
settings: ['backgroundBlur', 'frameRate'],
};
function configurationChange(changes, observer) {
const change = changes[0];
if ('backgroundBlur' in change.capabilities) {
// Background blur capabilities have changed.
}
if ('backgroundBlur' in change.settings) {
// Background blur setting has changed.
}
if ('frameRate' in change.settings) {
// Frame rate setting has changed.
}
}
const observer = new MediaStreamTrackObserver(configurationChange, options);
observer.observe(track);
// Later...
observer.unobserve(track); It may be overkill though... |
A single event seems better to me if we can stick to it. |
Do you mean single event for capabilities and settings? |
(I support François's proposal.) |
I tend to prefer one event. The event could tell whether capabilities and/or settings are changed.
This seems a bit odd to me. |
I agree that it gives diminished returns, if we agree on quickly exposing which settings/capabilities have changed. My interest in that stems in part from my desire for a mechanism that can be used with my auto-pause mechanism, without causing frames to be needlessly dropped when an irrelevant config-change occurs, which the handler will just unpause-and-return once it sees.
I am open to alternative shapes. |
We should first decide whether auto-pause should be handled at the source or at each track. |
Is there a reason, other than implementation complexity[*], for auto-pause to happen on sources rather than tracks? [*] Normally a valid concern, but here I suspect the complexity is quite low, as pausing could be implemented by dropping frames where applicable. Later, the optimization could be added of not posting frames from the relevant source if all associated tracks are paused. Or wdyt? |
Let's continue this discussion in the auto-pause issue, w3c/mediacapture-screen-share-extensions#4 |
Makes sense. |
There is the potential for many different configuration changes. What if an application is only interested in some? For example, we see this demo code by François, which currently reads:
This assumes that the only possible event is to blur, or else the following would be possible:
A likely future modification of the code would be:
However, depending on what changes and how often, this could mean that the event handler is invoked in vain 99% of the time, and possible multiple times per second, needlessly wasting CPU on processing the event by JS code.
I think a more reasonable shape for the API, would be to expose a subscribe-method.
Wdyt?
(Some prior art here is in w3c/mediacapture-screen-share#80. I propose my change as incremental progress over it. CC @beaufortfrancois, @guidou, @eehakkin and @youennf)
The text was updated successfully, but these errors were encountered: