Add autopause attribute to media elements to allow automatic pausing of media

[keithamus]

@brucelawson has shared an experience where videos are not paused when hidden from view. See #9770 for more discussion around that.

In this twitter thread @zcorpan has suggested we add an autopause attribute which would allow the browser to control the ability to autopause the video, based on if it intersects the viewport.

This is a common feature in other social applications, such as Twitter, which will pause video as you scroll down and it no longer intersects the viewport, so there some strong use cases for such a feature.

To give this some concreteness:

interface HTMLMediaElement : HTMLElement {
  // ... 
    [CEReactions] attribute boolean autopause;
  // ...
}

autopause — Hint that the media resource can be paused and resumed automatically as the element intersects the viewport.

/cc @zcorpan @lukewarlow

[keithamus]

@brucelawson asks in #9770:

Would it be too weird for autopause to be considered true if absent from the markup, and over-rideable with autopause="false" for those who want audio to continue?

I think this would be a breaking change, and so would not be viable as existing documents which rely on video continuing to play while not visible would break.

[romainmenke]

How does this interact with autoplay if the video element doesn't initially intersect with the viewport?

For example :

<video autoplay autopause muted loop src="...">

Would this pause the video initially until it intersects with the viewport, at which point it would (re)start playback if possible? Optionally doing clever preloading of metadata and the first frame.

We apply that pretty much everywhere through a lot of javascript, so having this natively would be nice

[zcorpan]

@romainmenke yeah I was thinking about autoplay also, which currently autoplays as soon as possible. But more useful may be to autoplay when visible. We could add that as a new value for autoplay e.g. autoplay="when-visible" or heuristically do so when specifying both autoplay and autopause.

Is there any case where you would want autoplay-when-visible but not autopause when no longer visible?

[romainmenke]

Is there any case where you would want autoplay-when-visible but not autopause when no longer visible?

We have not encountered any.

[zcorpan]

This feature may need to be disabled when scripting is disabled for privacy reasons (at least for scrolling), same as lazy-loading.

Related: https://xsleaks.dev/docs/attacks/experiments/scroll-to-text-fragment/

[zcorpan]

cc @whatwg/media

[brucelawson]

Thinking further about autopause, this wouldn't necessarily solve my use case. I'm iframing a video from YouTube (but could be vimeo, or soundcloud, or bandcamp, or any number of other similar services) so don't have access to the underlying media element. Could the attribute work on an iframe?

[annevk]

It might be nice to align the terminology with lazyload too.

[jernoble]

Grist for the mill:

WebKit will currently "autopause" silent, autoplaying video once it scrolls out of the viewport or when the web view is otherwise hidden. (Unless overridden by user settings, WebKit won't generally allow audible content to autoplay, but in the case of autoplaying audible videos, WebKit will not "autopause" those.)

Once the page makes an explicit call to play() or pause(), the can autoplay flag is set to false, and the "implicit autopausing" behavior no longer applies.

So, an explicit "autopause" behavior could be implemented by WebKit to have the effect of automatically pausing audible content that still has its "can autoplay flag" set. However, given our policies about autoplaying audible content, this would be a very rare scenario.

[jernoble]

(Continued...)

However, it's important to point out that this will likely not solve the issue raised in #9770; a well-behaved player will start playback explicitly after a user gesture, which will clear the "can autoplay flag". The "autopause" attribute would have to be explicitly defined to ignore the "can autoplay flag", and that seems non-ideal. Sites which explicitly play() can use an IntersectionObserver to explicitly pause() when they lose visibility.

[tabatkins]

I really really like this, especially with the "scrolled off-screen" behavior. Specifically, I think the behavior would be that you auto-pause if you either (a) don't have a layout box (aka display:none or similar), or (b) are fully off-screen.

Auto un-pausing is a thornier issue. I think my preferences are, in descending order:

1. Don't auto-unpause at all. The user can unpause it themselves if they want.
2. Give autopause a value that causes it to remember the play state when it auto-pauses, and resumes that state when it's back visible. So if it was playing, it'll resume playing, but if it was paused it'll still be paused. Like autopause="auto-resume" or something.
3. Do 2 automatically, no opt-in (maybe provide an opt-out?).

Sites which explicitly play() can use an IntersectionObserver to explicitly pause() when they lose visibility.

If we can avoid it, I'd like to not distinguish between the user pressing the built-in play button and them pressing a page-provided play button that calls .play() on the video. I think if the site explicitly asks for autopause we can assume they want it to auto-pause even if they explicitly called .play(), and shouldn't force them to write even more script. Unlike autoplaying, the site choosing to autopause isn't a nuisance behavior we have to guard the user from.

Thinking further about autopause, this wouldn't necessarily solve my use case. I'm iframing a video from YouTube (but could be vimeo, or soundcloud, or bandcamp, or any number of other similar services) so don't have access to the underlying media element.

Hm, I suppose the easiest path here is those sites just looking for a parameter in the URL and setting autopause by themselves. But another alternative could be allowing autopause on iframe and auto-applying it to any media that doesn't explicitly specify a value for the attribute. (That is, let the default value be "auto" or something, rather than "true" or "false", and have it respond to some environmental hints like this.)

This might help explain the WebKit behavior too; they can supply some different environmental hint.

[lukewarlow]

Yeah I think it pauses and there's no unpause behaviour. (Autoplay aside I'm not sure either way on that)

Potentially can apply to the whole document using a meta tag? (Or as an attribute on iframes?). That way you as an author can opt in to it as a default. Having said that I think it'd also be fine to just accept this would require the embed to provide some mechanism to set the value?

[jernoble]

@tabatkins, the complexity in your proposal is one of the reasons I would argue that "autopausing" explicitly play'd content is not ideal.

If autopause causes the "internal play steps" or "internal pause steps" to run, then an autoplay'd element won't autoplay when made visible again, unless we add a "was this playing when the autopause steps were run" flag, which we'd have to clear somehow if play() or pause() were called when the element was invisible, etc., etc.

I think if the site explicitly asks for autopause we can assume they want it to auto-pause even if they explicitly called .play()

I don't think we can make that assumption. For one, the name "autopause" implies the behavior is similar to "autoplay", which effectively becomes a no-op once play() or pause() are called. The naming alone would make this confusing. If we want to add a feature which would force the video to pause when not visible, it should be called "require-visibility" or something; same for the iframe proposal.

...and shouldn't force them to write even more script.

This is an ergonomics argument. If a page could do something like video.onhide = event => event.target.pause(), this doesn't seem unduly burdensome. But IntersectionObservers are complicated and hard. Maybe we should make this easier instead.

I do agree that this is a use case worth solving. "autopause" doesn't seem like the right fit though.

[lukewarlow]

I do think we should have something that doesn't require JS though. Popovers don't, videos don't, so using them together shouldn't imo.
The name itself is open for bikeshedding it shouldn't be a blocker to the underlying mechanism.
pausewhenhidden or something was my idea for a name fwiw.

[padenot]

Some of the behaviour discussed here is optional in the spec: https://html.spec.whatwg.org/multipage/media.html#ready-states:eligible-for-autoplay-2

Gecko implements and ships something like WebKit (and this implements various occlusion detection if it can, not only viewport, e.g. if you minimize your browser window, or hide it with another window of your desktop environment that's opaque, etc.). This only applies to non-audible videos (muted or even with a silent audio track, iirc).

It also resumes playback once it's visible again, if it was previously auto-playing, and it's still allowed to do so. This effectively does the same as you'd do with an animated image, but with all the benefits of videos. Gecko also plays various tricks to save resources of non-visible videos by stopping decoding the video tracks, but this is transparent to the page.

In any case, users can decide if they allow auto-play of audible content, non-audible content, or disallow autoplay altogether, globally with override per origin. We made this follow the general rules we enforce with auto-playing media content. The default is to allow auto-playing of non-audible media.

[brucelawson]

jernoble wrote

the complexity in your proposal is one of the reasons I would argue that "autopausing" explicitly play'd content is not ideal.

From the perspective of a user, if some content pauses when made invisible (because it's scrolled off screen, or a popover is dismissed) it's weird if some pause, and others don't, because of a technical way that they were initially started playing that is opaque to the end user?

[lukewarlow]

From the perspective of a user, if some content pauses when made invisible (because it's scrolled off screen, or a popover is dismissed) it's weird if some pause, and others don't, because of a technical way that they were initially started playing that is opaque to the end user?

Especially if the video player happens to be a custom one such that it's using controls linked to the JS APIs instead of the native browser one? The user shouldn't need to care about that.

If the name autopause is the issue then pauseoffscreen or somethng could be used instead, we can bikeshed a name. The main thing is the underlying capability.

[jernoble]

@brucelawson Said:

From the perspective of a user, if some content pauses when made invisible (because it's scrolled off screen, or a popover is dismissed) it's weird if some pause, and others don't, because of a technical way that they were initially started playing that is opaque to the end user?

Frankly, as a user, I think it's weird that any content doesn't pause when it's dismissed via a dialog or popover. The user doesn't know or care about the distinction between removing an element from the DOM and display:none. But I made that comment in the related bug.

And again, I am not arguing that this isn't a problem worth solving; the "complexity" I'm referring to is around "auto-unpausing". A naive "autopause" specification would mean content played via the "autoplay" attribute would not resume autoplaying after an "autopause" because it's "has autoplay flag" would be cleared.

So, to break down use cases:

1. Dismissing a popover or alert should (imo) have the same behavior as removing that same element from the DOM, and for the same reason. Explicitly marking elements as "autopause" shouldn't be necessary.

2. Autoplaying (starting without a user gesture) audible content is already very difficult, so "autopause" wouldn't have much practical effect there.

3. Autoplaying non-audible content already "autopause"s (at least in WebKit and Firefox) so it wouldn't have much practical use there.

The remaining use case is:

4. Pausing explicitly played content when the user scrolls it out of the viewport. "requirevisibility" would solve this without the ambiguity of "autopause", but I would hate to give sites more power to block user's abilities to play audible content when the browser is backgrounded, so I would rather it only apply during visibility transitions, and not have it mean "this element must be visible for any playback to make occur", so perhaps "requirevisibility" isn't the best name either.

[tabatkins]

Dismissing a popover or alert should (imo) have the same behavior as removing that same element from the DOM, and for the same reason. Explicitly marking elements as "autopause" shouldn't be necessary.

I agree that we probably should do this, and we probably have the opportunity to do so now, before popover usage (containing audio/video) is high enough to cause compat issues.

We probably want that to be controllable, tho, so it's still worth thinking over the problem. (For example, a music-playing page that puts the audio in a dialog that can be summoned and dismissed easily for scrubbing or selecting tracks.) Hooking this into the attribute that also controls pausing when scrolled off-screen probably makes sense, then.

Pausing explicitly played content when the user scrolls it out of the viewport. "requirevisibility" would solve this without the ambiguity of "autopause", but I would hate to give sites more power to block user's abilities to play audible content when the browser is backgrounded, so I would rather it only apply during visibility transitions, and not have it mean "this element must be visible for any playback to make occur", so perhaps "requirevisibility" isn't the best name either.

Hm, you jumped from "scrolled off-screen" to "backgrounded" here - is there a reason you think conflating the two would be useful? No one's asked for auto-pausing when the page is blurred or backgrounded. Focusing purely on the scrolling use-cases would be ideal here, I think, as they're already proven to be desirable (as it's the behavior of Twitter).

[jernoble]

We probably want that to be controllable, tho, so it's still worth thinking over the problem. (For example, a music-playing page that puts the audio in a dialog that can be summoned and dismissed easily for scrubbing or selecting tracks.)

Well, I think it would be, for the same reason as it is during DOM removal: pausing is async, and removeChild();play() should prevent the autopause behavior.

Hm, you jumped from "scrolled off-screen" to "backgrounded" here - is there a reason you think conflating the two would be useful? No one's asked for auto-pausing when the page is blurred or backgrounded. Focusing purely on the scrolling use-cases would be ideal here, I think, as they're already proven to be desirable (as it's the behavior of Twitter).

I think the two concepts are tied together (visibility, from the users pov), and sites will want to pause their content when tabs are backgrounded using this API. TikTok, for example does this already.

[tabatkins]

removeChild();play() should prevent the autopause behavior.

Ah, I didn't realize that! If that works, then I'm fine with it; the use-case I'm concerned about is already running script, and that's very easy to handle.

I think the two concepts are tied together (visibility, from the users pov), and sites will want to pause their content when tabs are backgrounded using this API. TikTok, for example does this already.

As you said, tho, we probably don't want to make this easier. Plus, if a site does want to do this, there's an event they can listen for to achieve it fairly trivially. This is distinct from the scrolling case, where setting up IntersectionObservers is substantially more involved.

[jernoble]

As you said, tho, we probably don't want to make this easier.

Actually, pausing on boundary conditions I don't really have a problem with. E.g., iOS WebKit itself will pause video+audio playback when entering the background. On iOS however, once paused, a user can always restart playback through system-provided controls. What I would like to avoid was making it easy to block that particular ability, by speccing a new attribute that newly requires media elements to be visible for playback. (Such an attribute would be a particular foot-gun for <audio> elements, which without controls are in the DOM but are not "visible".)

Plus, if a site does want to do this, there's an event they can listen for to achieve it fairly trivially. This is distinct from the scrolling case, where setting up IntersectionObservers is substantially more involved.

Right, and then we're back to an ergonomics argument. IntersectionObservers are difficult to use, and hence my video.onhide strawman above. There's also the content-visibility CSS attribute that may have interesting precedent here.

[zcorpan]

pausing is async, and removeChild();play() should prevent the autopause behavior.

I think there's a spec issue that makes this not work currently: #9467

[dalecurtis]

FWIW, Chrome already "auto pauses" video-only/muted sources when they're put into background tabs. autoplay muted initiated playbacks also automatically pause when moved out the viewport. We haven't seen any issues filed due to this, as such it's unclear to me that we need a new attribute.

https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/platform/media/web_media_player_impl.cc;l=3485;drc=dc1f1aa37a5923b7e46eaef9423724d8a5162442
https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/core/html/media/autoplay_policy.cc;l=344;drc=047c7dc4ee1ce908d7fea38ca063fa2f80f92c77

[zcorpan]

@dalecurtis it's easier for users to notice that it's automatically paused when there's audio. And maybe in many cases, users want the audio to continue playing even when they've scrolled the video off screen or switched to another tab or application (e.g. podcast, music video, news). Since videos with audio don't automatically pause in browsers today, it seems risky to start doing so without any opt-in.