Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lazy loading video element poster attribute #6636

Open
asuh opened this issue Apr 30, 2021 · 38 comments · May be fixed by #8428
Open

Lazy loading video element poster attribute #6636

asuh opened this issue Apr 30, 2021 · 38 comments · May be fixed by #8428
Labels
addition/proposal New features or enhancements topic: media

Comments

@asuh
Copy link

asuh commented Apr 30, 2021

This approach for lazy loading videos is close to perfect.

<video preload="none" src="video.mp4" poster="videoposter.jpg" type="video/mp4">

In a video gallery with many videos, using preload="none" can help speed up page loading with its lazy loading capability. The one missing feature is lazy loading the poster attribute image.

I realize loading="lazy" only works for iframe and img elements, but this one enhancement to the lazy loading capabilities would potentially be a noticeable performance improvement in addition to not preloading the videos.

@annevk annevk added addition/proposal New features or enhancements needs implementer interest Moving the issue forward requires implementers to express interest topic: media labels Apr 30, 2021
@annevk
Copy link
Member

annevk commented Apr 30, 2021

cc @whatwg/media @domfarolino

@foolip
Copy link
Member

foolip commented May 6, 2021

Paging some people involved in #3752: @bengreenstein @domenic @jakearchibald @zcorpan

If we just add a loading="lazy" attribute for <video>, should it affect the poster in exactly the same way as <img src>? Or should it also do something for the video resource if no poster attribute is used?

@jakearchibald
Copy link
Contributor

Gut feeling: posterloading="lazy" to control the loading of the poster, and if we want the loading of video to be impacted by viewport intersection, add another option to preload, eg preload="lazymetadata" & preload="lazy".

Otherwise, combinations like loading="lazy" preload="none" are confusing.

@zcorpan
Copy link
Member

zcorpan commented May 6, 2021

I wonder if it makes more sense to bring all of img's features to video's poster image by something like

<video use-first-child-img-or-picture-as-poster-also-rename-this-attribute>
 <source ...>
 <picture>
   <source ...>
   <img loading="lazy" decode="async" srcset="..." sizes="...">
 </picture>
</video>

Otherwise how do we add srcset, art direction, and type-based resource selection to poster images?

@zcorpan
Copy link
Member

zcorpan commented May 6, 2021

If loading="lazy" appears on a <video> I would expect it to apply to the video, or maybe both the video and the poster. Its behavior is orthogonal to preload in that preload specifies how much to load, and loading specifies when to load (ASAP vs wait until it's in view or almost in view).

@jakearchibald
Copy link
Contributor

I wonder if it makes more sense to bring all of img's features to video's poster image by something like

I like that!

loading behavior is orthogonal to preload…

loading="lazy" preload="none" seems really weird though

@zcorpan
Copy link
Member

zcorpan commented May 6, 2021

It's not so weird if it also applies to poster="...", in my opinion. I guess the loading attribute would have no effect when there's no poster attribute and you use preload="none", sure. Is it a problem? loading="lazy" with src="data:..." is also weird, but isn't disallowed.

@zcorpan
Copy link
Member

zcorpan commented May 12, 2022

I think these are the options to solve this issue:

  • Add a posterloading=lazy attribute to control lazy-loading of the poster image (but not the video).
  • Add a posterelement="#imgid" attribute to point to an <img id=imgid> element to use as the poster image, and support (almost) all of <img>'s features also for poster images. This also solves other feature requests like picture/srcset support for poster images. See The <video> poster attribute should accept multiple sources #4004
  • Don't add any new attribute, but use lazy-loading behavior when preload=none and dimension attributes are specified, as suggested in https://bugs.chromium.org/p/chromium/issues/detail?id=1317849#c6 (personally I don't like this option, as noted in the bug).
  • Add a loading=lazy that controls both the poster image and the video with the same attribute.

For lazy-loading of the video:

  • Also add loading=lazy to control lazy-loading of the video (but not the poster image).
  • Reuse the preload attribute to control lazy-loading of the video (but not the poster image).
  • Add a loading=lazy that controls both the poster image and the video with the same attribute. (This is also in the above list.)

@zcorpan
Copy link
Member

zcorpan commented May 12, 2022

I'm not sure it's worthwhile to specify lazy-loading for video, since preload="none" exists and people can use lazy-loading poster images (when that is supported).

@scottaohara
Copy link
Collaborator

@zcorpan would

I wonder if it makes more sense to bring all of img's features to video's poster image by something like

mean that mean authors would be able to provide alternative text for the poster then? because that'd be something people have wanted for quite some time, and would be most welcome.

@zcorpan
Copy link
Member

zcorpan commented May 13, 2022

@scottaohara maybe, but I think it's better to discuss alt text (or accessible name) for video (and audio) in a new issue.

For now, aria-label or aria-labelledby can be used, right?

@scottaohara
Copy link
Collaborator

We're talking about different things, @zcorpan. Those attributes can be used to provide a name for the 'video' element, but they would not (and should not) be used to provide a description for the poster itself. This graphic is often used (misused) to provide some sort of title card with content that is only available via that image, and is only relevant when that image is displayed, has long been a frustrating topic for some.

Here is a closed HTML AAM issue which then links to other past issues on the subject. (following the links leads to some pretty heated 'discussion')

Understood if you think this should be spun out into a separate issue, but the reason I brought it up is because of your mention of "all of img's features", and if this idea is accepted, then this image (and thus its alternative text) could be exposed.

@asuh
Copy link
Author

asuh commented May 13, 2022

Here's what I'm reading from the various conversations above, also with my preference.

<video
  src="url_of_src" // URL of video source
  poster="url_of_img" // URL of image source
  aria-label="Video element name" // Describes the name of the video element, similar to describing a button label
  alt="Description of video content" // Describes the contents of the video itself, similar to image alt text
  loading="lazy" // Lazy load the video element and poster image
  preload="none" // Don't preload the video element, similar to loading="lazy", but could also help define how to lazy load the video
>

It makes more sense for loading="lazy" to be a consistent attribute name with consistent functionality. Even if the loading attribute duplicates most of preload, I would always be inclined to add loading to all media for better performance. Education can help differentiate it from preload and when/where to use either.

@zcorpan
Copy link
Member

zcorpan commented May 16, 2022

@scottaohara ok. Yeah, I think a separate issue would be best to align on how to expose the poster image to ATs. It might inform how we want to solve this issue, but still lazy loading and accessibility are orthogonal concerns.

@zcorpan
Copy link
Member

zcorpan commented Jun 30, 2022

I think we should add posterloading=lazy to solve this issue, since it's least complex and the other things can still be added separately (lazy-loading the video with loading=lazy, adding alt text for the poster #7954 could be a separate attribute like posteralt, and srcset/picture support for posters I think should be solved together with #6363).

zcorpan added a commit that referenced this issue Oct 25, 2022
Add a new posterloading attribute to the video element which controls lazy-loading of the poster image.

Fixes #6636.
@zcorpan zcorpan linked a pull request Oct 25, 2022 that will close this issue
5 tasks
@zcorpan
Copy link
Member

zcorpan commented Oct 27, 2022

PR: #8428

This issue still has needs implementer interest. @yoavweiss is there interest for Chromium? @annevk for WebKit? @emilio for Gecko?

@annevk
Copy link
Member

annevk commented Oct 31, 2022

posterloading seems like a reasonable addition, yes.

@yoavweiss
Copy link
Contributor

^^ @chrishtr for Chromium interest, as his team is likely to own the implementation work.

@chrishtr
Copy link
Contributor

Confirmed interest from Chromium.

@zcorpan zcorpan removed the needs implementer interest Moving the issue forward requires implementers to express interest label Nov 9, 2022
@zcorpan
Copy link
Member

zcorpan commented Nov 9, 2022

Thanks @annevk @chrishtr !

What's remaining is review of #8428 and the tests.

zcorpan added a commit that referenced this issue Nov 10, 2022
Add a new posterloading attribute to the video element which controls lazy-loading of the poster image.

Fixes #6636.
@emilio
Copy link
Contributor

emilio commented Dec 20, 2022

This seems fine to me as well. In general it'd be great to have it better-defined how <video poster> works (in Gecko the poster is just an anonymous <img> element), but that might be a bit trickier... It could unlock very cool stuff tho, like something like:

<video ...>
  <picture slot="poster">...</picture>
</video>

or pretty much anything (doesn't have to use the slot attribute, could be magic a la <details>, but you get the idea...).

The shadow tree of <video> could be specified as something like:

#shadow-root
  <slot name=poster></slot>
  <ua-defined-controls></ua-defined-controls>

Where the ua-defined-controls contents are, well, UA-defined, and would include captions, the video controls, etc.

I wouldn't oppose to the one-off extra attribute, but that would be perhaps a more flexible / better design over-all... Would there be appetite for such a thing?

@zcorpan
Copy link
Member

zcorpan commented Dec 20, 2022

Thanks @emilio

That would be like the second option in #6636 (comment) but without ID lookup (instead relying on picture or img being a child or descendant of video + some opt-in).

It's more work, but we get more features. Curious to hear what others think.

@johannesodland
Copy link

I wonder how this would look if the source media attribute was reintroduced later as in #6363.

@scottaohara
Copy link
Collaborator

It's more work, but we get more features. Curious to hear what others think.

just chiming in again that the idea of using picture > img within a video would be welcome for providing alternative text for the poster image. #7954

@annevk
Copy link
Member

annevk commented Dec 21, 2022

How common is poster? A bigger redesign might be worth doing if it's popular and also if there are already popular libraries available that address some of the shortcomings.

@zcorpan
Copy link
Member

zcorpan commented Dec 21, 2022

Web Almanac: https://docs.google.com/spreadsheets/d/1T5oVAVmcH3sM6R-WwH4ksr2jFtPhuLXs3-iXXoABb3E/edit#gid=1515756004 - though poster appears multiple times and I'm not sure how to interpret pct_attribute and pct_videos. @eeeps can you help?

Chromium use counter V8HTMLVideoElement_Poster_AttributeGetter is at 2.1123% and V8HTMLVideoElement_Poster_AttributeSetter at 2.0306%, but I assume this doesn't count use in HTML without author JS accessing the getter/setter, and if so, usage is probably higher.

A quick query in httparchive sample_data (10,000 pages) gives 445 pages = 4.45% of pages, and 1743 pages use video, so 25,5% of pages with a video element also use poster.

queries
SELECT COUNT(DISTINCT page) AS num FROM `httparchive.sample_data.response_bodies_desktop_10k` WHERE
REGEXP_CONTAINS(body, r'(?i)(<video\s+[^>]*\sposter\s*=)')

445

SELECT COUNT(DISTINCT page) AS num FROM `httparchive.sample_data.response_bodies_desktop_10k` WHERE
REGEXP_CONTAINS(body, r'(?i)(<video\s+)')

1743

@kevinfarrugia
Copy link

kevinfarrugia commented Mar 2, 2023

Incidentally I was looking at a similar query and the results are similar to what you have in the sample data. Uses the same data set as the Web Almanac 2022, (June 2022) sampled at 10%.

+---------+-------------------+-------------------+-----------------------+-------------+
| client  | pages_with_videos | pages_with_poster | pct_pages_with_poster | total_pages |
+---------+-------------------+-------------------+-----------------------+-------------+
| mobile  |             38847 |             10458 |    0.2692099776044482 |      771291 |
| desktop |             35098 |             10071 |   0.28693942674796286 |      558537 |
+---------+-------------------+-------------------+-----------------------+-------------+

pct_pages_with_poster only includes pages which have at least one video element.

Query
#standardSQL

CREATE TEMPORARY FUNCTION getAttributes(payload STRING)
RETURNS STRUCT<poster INT64> LANGUAGE js AS '''
try {
  var almanac = JSON.parse(payload);

  var obj = {
    poster: 0,
  };

  for (var node of almanac.videos.nodes) {
    if (typeof node.poster != "undefined") {
      obj.poster++;
    }
  }

  return obj;
} catch (e) {
  return {};
}
''';

SELECT
  client,
  COUNTIF(total_video > 0) AS pages_with_videos,
  COUNTIF(attributes.poster > 0) AS pages_with_poster,
  COUNTIF(attributes.poster > 0) / COUNTIF(total_video > 0) AS pct_pages_with_poster,
  COUNT(0) AS total_pages
FROM (
  SELECT
    _TABLE_SUFFIX AS client,
    SAFE_CAST(JSON_EXTRACT_SCALAR(JSON_EXTRACT_SCALAR(payload, '$._almanac'), '$.videos.total') AS INT64) AS total_video,
    getAttributes(JSON_EXTRACT_SCALAR(payload, '$._almanac')) AS attributes,
  FROM
    `httparchive.pages.2022_06_01_*` TABLESAMPLE SYSTEM(10 PERCENT)
)
GROUP BY
  client

@zcorpan
Copy link
Member

zcorpan commented May 8, 2023

I discussed with @chrishtr, he said supporting picture might be too much complexity. But supporting img still gives srcset, alt, and other features of img.

@emilio wrote

<video ...>
  <picture slot="poster">...</picture>
</video>

or pretty much anything (doesn't have to use the slot attribute, could be magic a la <details>, but you get the idea...).

Yeah, I think we shouldn't use slot for an HTML-native feature. We could use a new boolean attribute on video:

<video posterchild>
  <img ...>
  <source ...>
</video>

(Please suggest other names for posterchild, although I also kinda like it 🙂.)

@annevk
Copy link
Member

annevk commented May 19, 2023

Does <video> need an attribute? Is an <img> child not sufficient?

@johannesodland
Copy link

I discussed with @chrishtr, he said supporting picture might be too much complexity. But supporting img still gives srcset, alt, and other features of img.

Given the resent reintroduction of the media attribute on media element sources, do you think it's worthwhile to reconsider the picture element?
I'm concerned there might be an inconsistency if the video element is able to support art-direction for videos but not for the poster image.

@zcorpan
Copy link
Member

zcorpan commented Jun 13, 2023

@annevk it might change behavior for legacy web content that already use img in video intended for old browsers without video support.

@jernoble
Copy link

jernoble commented Oct 3, 2023

Why not a new <poster> element, defined to have the same semantics and behavior of <img>?

Non-video supporting browsers would ignore it. Video-supporting browsers without specific support for it would ignore it as well.

@zcorpan
Copy link
Member

zcorpan commented Oct 4, 2023

But it would only do something when it's a child of a video element?

@jernoble
Copy link

jernoble commented Oct 4, 2023

Sure. Much like adding a poster= attribute on any non-video element doesn't do anything.

@jernoble
Copy link

jernoble commented Oct 4, 2023

(<poster> could also be a <picture> element instead of a <img>. The goal with this straw man was to solve the problem of legacy, non-video supporting UAs.)

@zcorpan
Copy link
Member

zcorpan commented Oct 5, 2023

I don't worry about what happens in non-video supporting UAs, most likely most of the web doesn't work in those browsers for other reasons anyway. I'm more concerned about currently existing web content and the effect for such content in new browsers. However, I haven't looked into how common it is to have an img in a video; if it's very rare maybe we can do what @annevk suggested in #6636 (comment)

@arogers-enthink
Copy link

arogers-enthink commented Oct 14, 2023

Art direction [out-of-the-box] would be a big plus when supporting prefers-reduced-motion.

@scottjehl
Copy link

scottjehl commented May 29, 2024

I like the sound of picture being used inside video to define poster images, since it'd support the variety of conditions that video source elements can now target via their media attributes, and offer srcset and sizes as well. In that case, would the picture element need a child img element or would the picture element act as a controller for the source of the video poster attribute's currentSrc?

Edit:

I suppose a child img would offer the loading=lazy behavior, so it's nice to have there.

That said, if video itself offered loading=lazy, could that attribute determine how other video features behave in addition to poster? For example, could loading=lazy combined with autoplay cause a video to delay its data fetching and autoplay behavior itself until the video element is in (near) the viewport? Lazy autoplay is another feature I'd love to see addressed declaratively and it'd be great if this one attribute we already know could drive it. (related new issue: #10376 )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
addition/proposal New features or enhancements topic: media
Development

Successfully merging a pull request may close this issue.