Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add height-based selection to srcset/sizes #2973

Open
zcorpan opened this issue Aug 29, 2017 · 21 comments
Open

Add height-based selection to srcset/sizes #2973

zcorpan opened this issue Aug 29, 2017 · 21 comments

Comments

@zcorpan
Copy link
Member

@zcorpan zcorpan commented Aug 29, 2017

See https://bugs.chromium.org/p/chromium/issues/detail?id=421909#c19

As part of the the photo page redesign on unsplash.com, we want to constrain images by viewport height. We also want to use img srcset and sizes to deliver responsive images.

In our sizes attribute, it's possible to define the width of the image when constrained by viewport height using media queries—for example, sizes="(min-aspect-ratio: 1/2) 80vh". However, if we want to add vertical padding around the image, there appears to be no way to exclude that padding from the calculated aspect ratio.

If calculations in media queries were possible, we could achieve this using (min-width: calc(100vh - var(--vertical-padding))). For the time being we are having to rely on JavaScript to perform these calculations, with necessary fallbacks.

Here is a full example of the image behaviour we are trying to achieve: http://jsbin.com/melewe/edit?html,css,js,output

This pattern we're pursuing seems to be increasingly common, so it would be great if we could make this easier for authors.

calc() in MQ would be nice, but better still is probably to allow specifying the image heights directly (in srcset and sizes). We excluded this use case originally to reduce complexity, but since this appears to be a recurring issue for web developers, it seems worthwhile to address.

Earlier issue for this: ResponsiveImagesCG/picture-element#86

@zcorpan
Copy link
Member Author

@zcorpan zcorpan commented Aug 29, 2017

Examples of width- and height-constrained images:

Examples of only height-constrained images:

@OliverJAsh
Copy link

@OliverJAsh OliverJAsh commented Aug 29, 2017

Thanks for filing @zcorpan.

I'm not sure I see how height-based selection in srcset/sizes would help with my example. The problem remains of how to define the media query part in sizes:

  • when the image is constrained by viewport height, height is viewport height - vertical padding (easily expressed as width using aspect ratio calculation)
  • otherwise the image width is viewport width - horizontal padding

I need a media query for the part in bold. In my example I'm using JavaScript to calculate this—I really want to be able to express this in plain CSS, which is where calc would come in.

@zcorpan
Copy link
Member Author

@zcorpan zcorpan commented Aug 29, 2017

I was looking more at the unsplash page, which appears to have a width-based layout breakpoint. So for that page, the media condition in sizes would reflect that breakpoint, and the specified size for the narrow layout would be 100vw and the specified size for the wide layout would be height 100vh (or whatever).

The jsbin example appears to be both width and height constrained, with some padding around the image, and the aspect ratio of the image is known. Correct? Maybe calc() in the media condition is enough to make it possible, but I'd also like to explore possibilities to make these things easier (maybe a contain keyword could help?).

@OliverJAsh
Copy link

@OliverJAsh OliverJAsh commented Aug 29, 2017

The JSBin example is what we're moving towards, with width and height constrained images—exactly as you said.

(maybe a contain keyword could help?).

I saw some discussion about this in ResponsiveImagesCG/picture-element#86 and couldn't quite see how it would help in my example. I did originally try to achieve my layout using object-fit: contain, however I don't want to stretch the image to fill the viewport height—I only want to constrain it by the viewport height. (This way there is no extraneous white space.)

but I'd also like to explore possibilities to make these things easier

I have found it extremely difficult to express the layout you see in the JSBin example. My requirements are:

  1. Reserve space for the image whilst it loads.
  2. Contain image (including padding) in viewport (fill width or height, whichever is smallest), whilst only taking up necessary space.
  3. Minimum height
  4. Responsive images

For 1 we can use the padding-bottom trick.

Because all elements are constrained on the X axis by default, we have to express constraints along the Y axis as constraints along the X axis. For 2 we have to define the maximum height as a max-width (calculated using viewport heights and the aspect ratio). For 3 we have to define the minimum height as a min-width (calculated using the aspect ratio).

For 4 we want to repeat the layout described in 2 and 3 for the sizes attribute. My current solution requires JavaScript due to the lack of calc in media queries, which unfortunately means we lose benefits of the preloader, etc.

If you have any suggestions, for how to improve this now or in the future, I would love to hear them.

@OliverJAsh
Copy link

@OliverJAsh OliverJAsh commented Aug 29, 2017

@zcorpan FYI, you can opt-in to the (WIP) new photo page on Unsplash with this link: https://unsplash.com/?xp=new-photos-page:experiment (temporary link only). If you then click through to a photo, you will see the new photo page with the behaviour described above. The layout is identical to the JSBin example I posted.

@zcorpan
Copy link
Member Author

@zcorpan zcorpan commented Aug 29, 2017

Thank you! This is extremely useful info for evaluating solutions. (Note that whatever we come up with here won't be usable immediately, it will have to be implemented and shipped in multiple browsers first, so in a few years or so...)

  1. Reserve space for the image whilst it loads.

This is what ResponsiveImagesCG/picture-element#85 is about, and I think we should fix that together with this issue.

  1. Contain image (including padding) in viewport (fill width or height, whichever is smallest), whilst only taking up necessary space.

OK, so that is what contain means. If we consider the proposal in ResponsiveImagesCG/picture-element#86 (comment) we get something like

sizes="contain calc(100vw - var(--horizontal-padding)) calc(100vh - var(--vertical-padding))"
  1. Minimum height

OK, that adds a height-based breakpoint to sizes, and ability to specify the height would help so you don't need to map it to a width (which is not possible if the aspect ratio is unknown).

sizes="(max-height: 400px) height calc(400px - var(--vertical-padding)),
       contain calc(100vw - var(--horizontal-padding)) calc(100vh - var(--vertical-padding))"

plus h descriptors in srcset so the browser can select a candidate when sizes only gives a height size, and calculate the intrinsic size.

@OliverJAsh
Copy link

@OliverJAsh OliverJAsh commented Aug 29, 2017

@zcorpan Thanks so much for the detailed response. We may be years away from having support for these changes, but it's good to understand the constraints of what we have today, and what is being done about that.

As I understand it, contain in sizes would tell the browser the image is contained, and either one of the specified width or height will be used depending on how the image is constrained. Is this correct? How does the browser know whether the image is constrained by width or height?

  1. Minimum height

OK, that adds a height-based breakpoint to sizes

As the image is contained, I think it also adds a width based breakpoint to sizes? This is why in my example I have to specify both a max-width and max-height media query. Or is this somehow made redundant by the specified height?

I think this looks like it would fit my example perfectly, although it's hard to know for sure without trying it out.

@zcorpan
Copy link
Member Author

@zcorpan zcorpan commented Aug 29, 2017

Ah right, I missed that aspect. It would then be:

sizes="(max-height: 400px) contain calc(100vw - var(--horizontal-padding)) calc(400px - var(--vertical-padding)),
       contain calc(100vw - var(--horizontal-padding)) calc(100vh - var(--vertical-padding))"

I don't think further breakpoints are needed, or rather "contain" handles it already.

The browser would know which dimension to use because it knows the viewport size and the image aspect ratio would be provided by srcset by using both w and h descriptors.

@OliverJAsh
Copy link

@OliverJAsh OliverJAsh commented Aug 29, 2017

If I understand correctly, wouldn't the first entry in sizes need to be:

sizes="(max-height: 400px) contain calc((var(--min-height) * var(--width-as-proportion-of-height)) - var(--horizontal-padding)) calc(400px - var(--vertical-padding)),
       contain calc(100vw - var(--horizontal-padding)) calc(100vh - var(--vertical-padding))"

That is, the contain width is (min height * width as proportion of height) - horizontal padding)?

@zcorpan
Copy link
Member Author

@zcorpan zcorpan commented Aug 30, 2017

I probably managed to confuse myself, sorry. It's difficult to reason about this without testing.

Anyway, I realize that the min-height would need to apply on both sides of the breakpoint, since shrinking the viewport width makes the image smaller on both dimensions. I need to think through how to apply the proposal to make it work as intended.

Alternatively, the proposal is still too difficult to work with, and we should come up with something else to make it simpler.

@zcorpan
Copy link
Member Author

@zcorpan zcorpan commented Aug 30, 2017

I think w3c/csswg-drafts#544 could help.

sizes="contain
       max(100vw - var(--horizontal-padding), var(--min-height) * var(--width-as-proportion-of-height))
       max(100vh - var(--vertical-padding), var(--min-height))"
@eeeps
Copy link
Contributor

@eeeps eeeps commented Sep 26, 2017

@OliverJAsh as for your use case/requirements – here’s the best, simplest thing I could come up with using what’s in browsers now: https://codepen.io/eeeps/pen/VMPJzK

It uses @tigt’s awesome coping-with-the-lack-of-h-descriptors technique.

Problems:

  1. The sizes is not completely accurate right around the constrained-on-width/constrained-on-height boundary, because of the padding. Most of the time, this shouldn't affect resource selection.
  2. There will likely be some jank when the image loads. Again I couldn't work this out, given the padding†.

h descriptors + contain would solve both problems. @zcorpan’s use of max() looks more elegant, but here’s what my not-used-to-max()-yet brain spit out:

<img srcset="https://via.placeholder.com/150x100  150w  100h,
             https://via.placeholder.com/300x200  300w  200h,
             https://via.placeholder.com/600x400  600w  400h,
             https://via.placeholder.com/1200x800 1200w 800h"
  sizes="((min-width: 342px) and (min-height: 292px)) contain calc(100vw - 12rem) calc(100vh - 12rem), 150px"
  src="https://via.placeholder.com/150x100" />

Note: 342px = min-img-width (150px) + padding (12 rem); 292px = min-img-height (100px) + padding (12rem) – consider the magic-ness of these numbers an argument for calc() in MQ.


†: this is the best that I could do. A closer fit to the requirements (it reserves the correct amount of space most of the time), but at some cost of complexity.

@eeeps
Copy link
Contributor

@eeeps eeeps commented Sep 26, 2017

As for use cases, I'll toss out a couple more.

Here's a sideways-scrolling site which is very cool and unsual.

Click on any of the images here to get thrown into a viewport-fit lightbox. Given the ~3,000 “lightbox” repos on Github, I expect this use case is much more common.

@eeeps
Copy link
Contributor

@eeeps eeeps commented Nov 1, 2017

Another use case which only dawned on me in a conversation with @yoavweiss about Hui Jing Chen’s (awesome) talk at You Gotta Love Front End – probably the biggest potential use case of all – images in vertically-sized blocks of vertically-flowing text (like this https://www.chenhuijing.com/slides/yglf-2017/#/8).

@tylersticka
Copy link

@tylersticka tylersticka commented May 30, 2018

I ran into this organically, so I thought I'd contribute another use-case and the steps I attempted to take (in case that's helpful to folks evaluating this issue).

I wanted a big ol' image to lead off this blog post about an art project I've been doing: https://tylersticka.com/journal/drawing-every-day/

The img container is always 100% width, 50vh tall when orientation is portrait, 75vh tall when it is landscape. The img fills the available height and width and uses object-fit: cover.

Initially I wrote the img element like so:

<img src="..." alt="..."
  srcset="
    grid-05-27-960.jpg 960w, 
    grid-05-27-1920.jpg 1920w, 
    grid-05-27-2560.jpg 2560w" 
  sizes="100vw">

But this often resulted in the chosen asset being too small, since the asset will extend beyond the horizontal boundaries of its container.

So then I tried using calc to base the width on the aspect ratio multiplied by the height:

<img src="..." alt="..."
  srcset="
    grid-05-27-960.jpg 960w, 
    grid-05-27-1920.jpg 1920w, 
    grid-05-27-2560.jpg 2560w" 
  sizes="
    (orientation: portrait) calc(16 / 9 * 50vh), 
    calc(16 / 9 * 75vh)">

This kinda seemed to work in Firefox (or at least it was failing gracefully) but it seemed to cause Edge to abandon the srcset.

So, ever the optimist, I tried writing this:

<img src="..." alt="..."
  srcset="
    grid-05-27-960.jpg 538h, 
    grid-05-27-1920.jpg 1076h, 
    grid-05-27-2560.jpg 1435h" 
  sizes="
    (orientation: portrait) 50vh, 
    75vh">

When that failed, I finally Googled the issue, found ResponsiveImagesCG/picture-element#86 and finally arrived here. 🙂

(I ended up just using a sizes value that was comfortably above 100vw. It'll probably download more than the user needs sometimes, but it's better than nothing!)

@OliverJAsh
Copy link

@OliverJAsh OliverJAsh commented May 30, 2018

@tylersticka We had a similar problem on https://unsplash.com for the "hero image" you see on the homepage, which also uses object-fit: cover. We ended up solving it using the picture element and providing multiple sources representing a spectrum of aspect ratios.

We sampled a rough height of the container element at regular width intervals (e.g. from 200px to 2000px, every 200px) and then provided a source for each.

image

image

@tylersticka
Copy link

@tylersticka tylersticka commented May 30, 2018

@OliverJAsh That's a clever solution, thanks for sharing!

A similar technique could work for my example, though it makes my head hurt a little considering my image's visible area is based on orientation (not viewport width) and my nav shifts from the top to the side as well. It doesn't help that my personal site is static, so I'd be preparing those images and writing that markup "by hand!"

(It could also just be my end-of-workday brain being slow…)

@DanMan
Copy link

@DanMan DanMan commented Apr 3, 2019

Anything still happening on this front? I'm also trying to create a lightbox and as is, I find it hard if not impossible to use portrait images on a landscape display in a responsive way. I can make it right in either direction (horizontally or vertically), but not both. Either the image is displayed too small/big, or a bad alternative is chosen. With object-fit, depending on circumstances, chances are that part of the image will be obstructed (instead), which you don't want in a lightbox.

For now I've solved it by giving the image a max-height:86vh (100vh minus margins and stuff), but that's just a workaround. The selected alternative is likely wrong.

@TobiasMalikowski
Copy link

@TobiasMalikowski TobiasMalikowski commented May 22, 2019

Hey guys, also wanted to ask if there are some news on this? We are working on an image service which works together with these native features to deliver the perfect image on the fly (also includes stuff like cropping, focus point for art direction, image compression, and much more). For some edge cases we really need to tell the browser the image height in addition to the width.

@rpeterman-gp
Copy link

@rpeterman-gp rpeterman-gp commented Sep 11, 2019

The solution I came up with was to calculate the resulting width based on the width to height ratio when creating the media, srcSet, and sizes string values for my <source> and <img> elements. (This solution works in the browser or server-side rendered React application.)

//...
// This should get you the max image height, minus any margin and padding
// around the parent container.
const containerHeight = imgElm.parentElement.offsetHeight;
// For determining orientation.
const { info: { width: mainImageWidth, height: mainImageHeight } } = mainImageData;
// Used later to decide wether or not to calculate width in `size` attribute value.
const orientation = mainImageWidth >= mainImageHeight ? 'landscape' : 'portrait';
// For brevity and readability later on.
const isPortrait = orientation === 'portrait';
// Simple media query for orientation if using `object-fit: cover`.
// const media = `(orientation: ${orientation})`;
// Let browser know which images to use and their widths, as usual.
const srcSet = imageSrcs
  .map(({ src, info: { width } }) => `${src} ${width}w`)
  .join(',\n');
const sizes = `${imageSrcs.map(
  ({ info: { width, height } }) =>
    // Set media query for size as you need. I just went with the image width here.
    `(max-width: ${width}px) ${
         // Let browser know the width of image at this media query.
         // For `object-fit: contain`, calculate width for portrait, falling back to just width for landscape.
         isPortrait ? `${(width / height) * width * (height / contentHeight)}px` : `${width}px`
         // For `object-fit: cover`, do the inverse. Will crop vertical excess on portrait,
         // horizontal excess on landscape. Fine for use as background element.
         // isPortrait ? '${width}px' : `${(width / height) * width * (height / contentHeight)}px`
      }`
  )}`;
//...

Basic concept illustrated in javascript. Update/refactor for your programming language and image data model as needed.

@DanMan
Copy link

@DanMan DanMan commented Sep 13, 2019

Of course, with the help of JS you can get done all kinds of things. The request is about a CSS solution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
7 participants