Unclear how to aggregate windows or how to handle multiple windows/monitors

[henbos]

4. Terminology says multiple windows may be aggregated into a single track, but it does not explain how this is done. Will the background be black? What if they are far apart? What happens if windows move around, will that change the resolution to match covering both windows? Or should the windows always be grouped next to each other, so that their relative position in the track does not correlate to the relative position of the windows on the monitor?

The same questions arise if you think about sharing "monitor" and the user have multiple monitors. Additionally, does the user pick a monitor or do you get all monitors, etc?

What happens if a window is resized?
What happens if resolutions change?
What happens if monitor setups change?
etc.

[henbos]

Do MediaStreamTracks have alpha channels? Should we do a black background, a transparent background, or let the application choose the color? How would the application know what is outside of the interesting area and what is just filled in as "black space"?

[henbos]

What to do if a window is partially outside of the visible area?

[henbos]

What to do if a window is partially outside of the visible area?

6.1 Capturing Logical or Visible Display Surfaces says...

Information that is not currently rendered to the screen SHOULD be obscured in captures unless the application has been specifically authorized to access that content

...it is unclear what "obscured" means. Clipping, painting black, etc?

[alvestrand]

Firefox implements application; Edge and Chrome implement screen and window only.

[henbos]

Let's split this into multiple issues.

[a2sheppy]

I think there ought to be an additional option, combineSurfaces, with the default value of true, where if true, the windows of an application, or the browser tabs if multiple tabs are able to be captured at once, are all drawn into a single track using whatever background is decided upon (it might make sense to have an option to specify transparent or a color to use for the background).

If combineSurfaces is false, on the other hand, each window or tab is captured into its own MediaStreamTrack. This would give greater flexibility, and would allow some really interesting stuff to be done presentationally when there are multiple surfaces being captured.

[eladalon1983]

This issue has not seen any activity in a long time. It appears that, important though it is, it is not important enough to merit action. I am closing this. Feel free to reopen.