Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[css-media-queries] Foldables support and enablement #4141

Open
argyleink opened this issue Jul 22, 2019 · 10 comments

Comments

@argyleink
Copy link

commented Jul 22, 2019

fold-emulator-running

Why

Android Q beta has support for folding the screen in the emulator and a list of features and APIs to support developers in using and adapting to foldable screens, I'd like to see the web have similar offerings in the form of a new @media query or queries.

fold-out-app-continuity

This new device/OS state is not the same as orientation or device width. It's more than that, as a webapp could project content while in a folded state onto the back or side of a device.

Imagine an <aside> literally on a foldable side of a device while reading an article.

What

As I dig into the Android featureset and APIs, the nature of the media query feels akin to how devs work with projectors and orientation.

Designers and developers should be able to leverage or adapt to these foldable device states and either project content onto another half of the device, or just simply adjust to the wider or thinner viewports respectively. There are many options, designs, patterns, etc for how to handle and adjust to foldable phones, all of which are being pioneered or explored right now as the technology unfolds (😉).

From the Android blog:

On this new form factor, your application could be transitioned from one screen to another automatically (eg. when folding / unfolding a foldable phone).

During this transition, your app will receive a configuration change for the new layout (and possibly density in some cases).

The ability to handle this transition shouldn't be limited to native apps, the web should have hooks into handling this as well.

xiaomi_fold_lgd

image-blog-perfecto-web-foldable-phones-recap-1

galaxy-fold-100788841-large

How

These are just my initial aggregated thoughts and assumptions from my research on which media queries would be useful. I hope to expand upon it greatly to cover the matrix of possibilities and unblock engineers on the web.

  1. is foldable screen
  2. is consuming all screens
  3. is folded vertically
  4. is folded horizontally
  5. is on main screen
  6. is front / is back
  7. is on screen (1-n)
  8. can project onto screen (1-n)

I envision these media queries assisting and enhancing:

  • css env vars
  • regions!
  • device width and height media queries
  • portrait and landscape media queries

google_bubbles_android_q_larger

Conclusion

I'm feeling like this media query could be very complex, as it needs to provide syntax for a wide array of foldables. That being said, the types of experiences and adaptations that web pages could do with the display hooks would make for very interesting and meaningful interactions. We should work towards enabling the web to handle the upcoming array of devices.

Links/References
@tabatkins

This comment has been minimized.

Copy link
Member

commented Jul 22, 2019

Yeah, those diagrams showing a 3-panel folder make me very nervous about designing a MQ here. Gives me flashbacks to trying to generalize about a rounded-screen MQ, which proved impossible.

The core question we have to answer here is, what questions should we enable the author to ask, such that they can do something useful with the answer? I'm not seeing very many sorts of questions I could ask that would let me usefully design to both a side-folding phone and a 3-pane tablet. :(

A JS API can be more fine-grained in answers, perhaps.

There might still be some stuff we can usefully extract here, tho. This could use some exploration. Like, maybe we can catch just "the X edge is folded", with a fallback "lol this is complicated good luck" value.

@tabatkins

This comment has been minimized.

Copy link
Member

commented Jul 22, 2019

Oh, and the "main screen" vs "secondary screen" dichotomy seems potentially useful. I'd want to see a worked-out example that would convincingly work on different devices using that first, tho.

@AmeliaBR

This comment has been minimized.

Copy link
Contributor

commented Jul 22, 2019

I agree that this needs to go beyond media queries. You need to be able to target the different parts of the screen for layout in some way. At the very least, there would need to be environment variables defining the areas and their dimensions, so that you could create a CSS grid on your root element that defines the different screen panels as different grid areas.

And it makes sense to approach this in a more general way, and consider the ability to have a single website control two screens for working with dual monitors or a projector. Another device with a secondary screen is the mac "touch bar" screen. I think apps have a way to draw content here, I'm not sure if there is any interest in exposing that screen to websites (or if it is better to let it be used for browser controls).

But, we need to distinguish between second screens/panels that should only be used for sidebar/controls, versus those where it makes sense for the main body context to fragment across & fill up the extra screen. With a folding screen, this could be the difference between a bent screen (like a book, you might want a column of text on each side, paginated to fit the screen) and one completely folded over (you could put the nav on the back, but might want to have the main text on the front screen in a scroller).

It's also worth thinking about how much user-control there should be. I assume there would be a user setting of the form "if my screen is folded, only use the front screen" vs "use both screens". Are there other relevant user preferences? If so, how should these variations be exposed?

Just from this discussion, I think there are two incomplete areas of CSS layout that need to be resolved to make these layouts really useful: CSS regions (fragmented text across grid regions representing panels of different sizes), and for screens.

@fbartho

This comment has been minimized.

Copy link

commented Jul 22, 2019

Is it really crucial that this is a "Foldable" concept? Or is it rather more general to talk about a series of viewports that you might be able to display into, and the geometry of those edges?

Aka if I have 2 panels, can I choose to display a seamless thing across them? or are they distinct screens that aren't close to each other?

@tabatkins

This comment has been minimized.

Copy link
Member

commented Jul 22, 2019

There is a foldable-specific aspect to this; displaying on the folded screen edge is different from displaying anywhere else.

But yes, other than that this is a more generic "i have multiple display surfaces" problem.

@fbartho

This comment has been minimized.

Copy link

commented Jul 22, 2019

Right, so I meant to elaborate my idea, you could have any number of viewports, that would attach to each others in different ways.

Example of macOS display arrangement

Additionally, it's not hard to imagine a multi-display device, that slides one screen behind another (locking at different positions) in some circumstances, rather than "folding". -- As long as the metaphor covers different edges cases like this, then I don't exactly "care" if it uses the word foldable in the name, I just wanted to make sure we didn't design this without thinking about things that aren't strictly folded into place.

@patrickhlauke

This comment has been minimized.

Copy link
Member

commented Jul 22, 2019

If the different folds are meant to work independently, it feels more like you'd definitely want something more akin to separate browser instances/windows that can still cross-communicate, rather than a single page. To me anyway...

@AmeliaBR

This comment has been minimized.

Copy link
Contributor

commented Jul 22, 2019

For the specific issue of avoiding putting content on the fold itself (the seam/spine as the screen wraps around a bend), this seems to be similar to the "safe inset" regions for notched iPhones. Within each screen panel, there will be some parts of the screen that you could paint a background on, but wouldn't want to put important content.

However, the finite set of safe inset margins currently defined for CSS environment variables falls apart if you need margins for many different layout regions.

@tabatkins

This comment has been minimized.

Copy link
Member

commented Jul 22, 2019

Yes, I assume that, in cases like the folded-phone where it's just "small screen with folded edge" vs "large screen, normal edges", we can specify that safe-inset will communicate the size of the fold-region so you can avoid it if that's all you're worried about.


So my thoughts right now are:

  1. Specify that env(safe-area-inset-*) should take folds into account, letting you avoid positioning anything on the fold.
  2. Might be useful to add a fold-side: [top | bottom | left | right | inline-start | ...] MQ, which can match multiple of those values, to communicate which edges are folded if you want to adapt your UI to take advantage of the fold.
  3. Investigate the multiple-display-area issue: how are they exposed to pages on current devices, what information might authors need from that, and is that information amenable to MQs or more suited for a specialized JS api.
@argyleink

This comment has been minimized.

Copy link
Author

commented Jul 23, 2019

should we open a separate issue for safe-area-inset types for foldables? or let it stay within this context of media queries?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.