Autoplay Policy Detection API

[alastor0325]

Request for Mozilla Position on an Emerging Web Specification

• Specification Title: Autoplay Policy Detection API
• Specification or proposal URL: https://w3c.github.io/autoplay/
• Caniuse.com URL (optional):
• Bugzilla URL (optional): https://bugzilla.mozilla.org/show_bug.cgi?id=1773551
• Mozillians who can provide input (optional): alwu

Other information

This is a new API which allows websites to detect whether they are allowed to play autoplay media. We would like to be the first browser starting implementing this. We already have a similar experimental feature before.

[bgrins]

Do Blink and WebKit have positions on this?

[bgrins]

A quick search on blink-dev for Autoplay Policy turned up https://groups.google.com/a/chromium.org/g/blink-dev/c/5Y1BqbGauEs/m/rRocaytKBgAJ which is and mentioned Web Audio - could you give a little bit of history/context to this and related features for the uninitiated?

And if we don't have a previous signal from webkit-dev then you could ask at https://github.com/WebKit/standards-positions/.

[alastor0325]

Not yet, but Chrome, Safari and Edge were all involved in this spec discussion during the past Media WG meetings. The link you mentioned above is a different thing, which is about their implementation of blocking web audio.

In short, every browsers has their own way to block autoplay (media element & web audio), but it's a pain for web developers to know whether they can play something or not. Current situation for them is that they have to make some hacky workaround, trying to start the media first and see if the play promise gets rejected or not. This spec aims to give web developers a clear picture about whether they are allowed to play autoplay media without using any other hacky workarounds.

My plan is to ask their attitude about this API in the TPAC this year, but if other browsers have same repo like this, I would go to file issues for them as well.

[bgrins]

My plan is to ask their attitude about this API in the TPAC this year, but if other browsers have same repo like this, I would go to file issues for them as well.

From https://wiki.mozilla.org/ExposureGuidelines#How_do_we_know_what_other_browser_engines_think.3F - there's not a defined process to preflight position requests from Blink (other than waiting for an "intent to"), so a discussion within the working group makes sense to me. For WebKit - yeah, please file an issue there.

[martinthomson]

I've been interested in the intersection between user consent and engagement gestures (clicks, keypresses) for some time. The permissions API was the place where we did a lot of that work. Take the notifications permission, for which browsers recently made the consent dialog conditional on a user gesture. That state or condition is not visible through any API surface and it is still inconsistent between browsers (and sites, thanks to Google's implementation quirks), which makes it pretty hard for sites to build to reliably.

I would like to see more effort concentrated on reconciling things in this area so that they are more consistent. This appears to be a point solution. @alastor0325, can you speak about what the Media WG has done about that sort of thing?

there's not a defined process to preflight position requests from Blink

If you have contacts at Google or Microsoft (or Brave or DDG or Vivaldi or Opera or...), who you can ask directly, within the working group or outside, that might work.

[Explorer09]

Hello. I'm just an ordinary user of Firefox and may I give a comment on this?

The introduction of the spec mentioned this use case:

For instance, if a user agent only blocks audible autoplay, then web developers can replace audible media with inaudible media to keep media playing, instead of showing a blocked media which looks like a still image to users. If the user agent does not allow any autoplay media, then web developers could stop loading media resources and related tasks to save the bandwidth and CPU usage for users.

I think the use case is weak for justifying the need to tell the web server about the autoplay policy. I think a better way for achieving the task is to let web pages present both the media and the alternate content (still image) and let the browser decide which resource to load.

EDIT: I have one more thought. If the user wishes to save bandwidth by downloading and playing inaudible video only, There can be an API or a protocol to humbly request the server to provide inaudible stream (in the case of livestream video), or to go back and use audible stream. This API can be more useful than exposing the autoplay policy, which is supposed to be client-side information only, to a web server.

[alastor0325]

Sorry for my late reply.

I would like to see more effort concentrated on reconciling things in this area so that they are more consistent. This appears to be a point solution. @alastor0325, can you speak about what the Media WG has done about that sort of thing?

The only issue related with Permission API is this one and so far we don't have other discussion related with that. The purpose of the autoplay policy detection is to provide an abstract layer no matter what kind of blocking autoplay mechanism is implemented by user agent, which should be helpful to address the pain point of not knowing whether those internal status affects autoplay or not.

I think the use case is weak for justifying the need to tell the web server about the autoplay policy. I think a better way for achieving the task is to let web pages present both the media and the alternate content (still image) and let the browser decide which resource to load.

HTML spec already has defined poster on the video element, which is used to display an image when video is not able to play.

---

In addition, in last TPAC, Safari expressed their interest in this spec as well. At the time I was presenting this spec, no chrome and edge guys were there so I couldn't know their position. But one of the spec reviewers is from Chrome, at least he didn't express negative or objection during the review process.

[Explorer09]

@alastor0325
Sorry for my ignorance, but is there anyone concerned about the potential fingerprinting when the web page can query the autoplay policy per element, instead of just yes and no on the "can-autoplay" question?

Specifically, why is there a separate allowed-muted value and not just allowed and disallowed? The media element should have been told if it is audible already, and whether the user would play the media with sound, or play it muted, is not a concern for a web page.

And if the user wants to save bandwidth by requesting an inaudible stream only, there should be a dedicated API for doing so. The autoplay policy API seems (for me) to be a wrong solution to the problem.

Detecting whether a media can autoplay ahead of time has merits, but I would wish a browser that, for privacy reasons, not tell too specifically about whether a media can autoplay muted. In other words, just return allowed or disallowed. In case a web page trying to guess if the policy is allowed-muted, the browser may pretend it's not (but autoplay everything muted to the user anyway; in other words, lie on the policy details).

[alastor0325]

Sorry for my ignorance, but is there anyone concerned about the potential fingerprinting when the web page can query the autoplay policy per element, instead of just yes and no on the "can-autoplay" question?

Yes, this issue is for discussing the fingerprinting possibility, feel free to leave your suggestion there as well.

Only providing allowed and disallowed for the media element isn't enough which can't really solve the problems which web authors encounter. Especially blocking only audible autoplay has become a common thing among different browsers, if web authors just want to know whether autoplay is allowed or not for a media element, they can simply call media.play() then they can know the result by checking the play promise. So only answering yes/no seems not too useful for this API design.

[Explorer09]

@alastor0325 Unfortunately in the issue you referenced, you only discussed about the allowed vs. disallowed difference in the context of fingerprinting. But I have a question specifically on the allowed-muted value. The benefit of reporting allowed-muted separate from allowed is small, and for the use case like:

(https://w3c.github.io/autoplay/#introduction) For instance, if a user agent only blocks audible autoplay, then web developers can replace audible media with inaudible media to keep media playing

it seems to me like a wrong tool for the job (the web browser can play the audible media muted, without any additional work of web developers).

And for the use case of livestream media where this autoplay API might benefit, as I mentioned in a previous comment, it would be better to just have the browser request a "no-audio" stream (as a dedicated request API), similar to how a video resolution change request can be done, and there would be no need to expose the client side setting (the autoplay policy) right away.

[uazo]

So only answering yes/no seems not too useful for this API design.

I don't think that is strictly true.
For web developers, knowing exactly why the promise goes wrong possibly grants an advantage in terms of user experience, since it is possible to show the reason for the error, whereas now the only possibility is introspection of the exception.

[alastor0325]

it seems to me like a wrong tool for the job (the web browser can play the audible media muted, without any additional work of web developers).

Sorry I don't agree this. Currently it did require extra work to allow muted media to autoplay depending on what kind of blocking autoplay mechanism a user agent is using.

And for the use case of livestream media where this autoplay API might benefit, as I mentioned in a previous comment, it would be better to just have the browser request a "no-audio" stream (as a dedicated request API), similar to how a video resolution change request can be done, and there would be no need to expose the client side setting (the autoplay policy) right away.

I don't think browser should do any request to server, this is completely different from the scenario of changing the video resolution. Those are all website's implementation, not API level design.

I don't think that is strictly true. For web developers, knowing exactly why the promise goes wrong possibly grants an advantage in terms of user experience, since it is possible to show the reason for the error, whereas now the only possibility is introspection of the exception.

I don't understand your point. Handling rejected promise is independent with this API and why we need to put them together to discuss? In addition, another benefit of this API is also allowing website to understand if they can play autoplay media before they actually create a real instance, which was a request we received from web developers during the spawning process of this API.

[alastor0325]

In addition, for finger printing discussion, feel free to discuss that in the spec issue or this Firefox issue. Thanks.

[alastor0325]

This spec got good feedback from TAG and Apple also expressed support on their position. So I'm going to close this bug and file the Intent-to-Ship later in order to target this API on Fx112+.

[zcorpan]

We should still record our position and add a dashboard entry.

[zcorpan]

Proposed position: positive
Proposed rationale: This API provides a convenient and synchronous answer to whether a particular kind of media can autoplay, which is something web developers are currently detecting either with various hacks or through UA detection. (example, more examples). This doesn't expose new information for fingerprinting (see w3c/autoplay#24) as media elements already expose through events which media will autoplay by trying to autoplay.

[martinthomson]

Tagged. I initially had concerns about the API being synchronous, but it seems like information like that needs to be available synchronously anyway.

I think that we can avoid a dashboard entry for this. Though it is a spec in its own right, it's a trivial one.