Integrate with Window Segmentation API

[kenchris]

Some internal displays may be foldable displays or made up of multiple physical displays - though be considered one display by the OS, like on the Surface Duo. The Window Segments API (should really be called Screen Seg.) allows getting the segments for these displays.

I think it would be nice with a property like hasSegments and a getter to get the segments for a given display.

As on macOS you can use an iPad as external display, you could imagine using some of these devices as external displays as well in the near future, so this doesn't just cover internal/primary displays

https://github.com/webscreens/window-segments

@darktears @zouhir

[michaelwasserman]

Thanks for the feedback; see also #21. Let's seek compatibility between proposals and support forthcoming screen paradigms!
Q: Do we know how dual/foldable/etc. screen hardware will be exposed by OS/WMs? That may inform some API shapes.
Q: Should we plan to discuss plans/compatibility as a TPAC Second Screen CG topic?

IIUC, the Window Segments Enumeration API exposes bounds for each content region within a single window that spans multiple displays/Screens. If a single Screen may contain one or more folds, their bounds or the bounds of the 'screen segments' delineated by those folds should be exposed on Screen (or on Fold?), beyond the proposed window.getWindowSegments() interface. I filed webscreens/window-segments#7

IIUC, the Screen Fold API exposes the angle and orientation of a Fold in a Screen. This seems to indicate that a foldable screen is exposed as a single Screen interface object, but also notes relevance of "devices with a single flexible screen (seamless), and devices with two screens (with seam)". I'd like to better understand how this API might handle (1) folds on a seam between two Screens/displays, (2) off-center folds, and (3) multiple folds in a single Screen. I filed w3c/screen-fold#38

Here are initial high-level thoughts on how these APIs might coordinate with Screen interface extensions and related proposals:

1. If new information/interfaces are exposed through the Screen interface, a multi-screen API should support that too.
   a. If Screen gains fold or HDR/WCG attributes, multi-screen API objects should include those attributes.
   b. If Screen gains fold(s), they should include bounds, or Screen should expose fold-delineated screen segment bounds.
2. If the web should expose folds between multiple separate Screens, a multi-screen API should support that too.
   a. Perhaps the Fold interface should expose its relationship to a single Screen or multiple Screens?
   b. Perhaps the Screen interface should expose multiple Fold interface objects?
   c. Perhaps the Fold interface objects should be vended outside the Screen interface?

I hope proposal owners will consider my questions/feedback, prompts for coordination, and suggested TPAC discussion!

[kenchris]

@anssiko can we get this on the agenda?

[anssiko]

Good discussion topic. Add a comment to w3c/secondscreen-wg#1

[michaelwasserman]

One potential concern for this proposal's shape is the singular Screens.currentScreen. If Windows will regularly span multiple Screens, it should be easy to determine which Screens (plural) the Window is shown on.

[michaelwasserman]

The relationship between these APIs has been explored to some degree with a Form Factors Explained CG report. The Device Posture API picks up where the Screen Fold API left off and Window Segments Enumeration is now a segments attribute of the Visual Viewport API.

It remains to be seen how the OS and WM semantics of emerging and future form factors relate to the concept of a Screen in the web platform. I think there are viable backwards-compatible approaches to supply information about multiple-current-screens, the segments within individual Screens, the global coordinates in such a space where a window should be placed, or even the desired set of screens over which a content fullscreen session might span.

See our API Design Risks doc for more thoughts in this vein, but I believe the current API shape is a responsible incremental step towards supporting web applications making opinionated use of generalized multi-screen environments on the web.

I'm going to close this issue until further specific issues are identified.