Interoperability concerns by lack of a common denominator in transport

[cynthia]

This is a follow-up from the TAG review request.

I haven't done a particularly good job at getting back to the group with the feedback we had, I apologize for that. The API side of the spec is sound, and I don't see anything that would be a serious issue worth opening issues for - but the elephant in the room is interoperability.

Setting aside the fact that this is already being implemented and shipping, with each implementation possibly shipping using some form of proprietary transport - we think that the spec not defining a common denominator transport mechanism in a normative section is problematic and would hurt interoperability, and potentially jeopardize the usefulness of the API.

If implementations ship with incompatible transports, we will be stuck in a situation where content authors have to rely on sniffing and whitelisting, either in the form of code or documentation, which is undesirable - and this encourages tight coupling between the remote playback device and the user agent.

From what I see, with the spec as of today: (Correct me if I am wrong)

• Android TV as a remote playback device will most likely not work from Safari, and Firefox will most like not be able to stream to an Apple TV.
• Users would have to randomly try different browsers (although intuitively, compatible pairing for most cases would be rather obvious) until they find something that works.
• On the same computer, you would see the remote playback device on some browsers and not on others due to this.

This doesn't seem like the kind of usability we should be promoting. I've brought this up with the team, and @w3ctag does not consider this specification ready for implementation for these reasons.

I have been made aware of the open screen protocol, would it be possible to accelerate the development of that and include that as a common denominator requirement for interoperability before we end up in another sniffing/whitelisting situation?

[dbaron]

The TAG review request is w3ctag/design-reviews#145 (which I'm saying so there's a link both ways).

[mounirlamouri]

Thanks for the feedback @cynthia. The open screen protocol that @mfoltzgoogle is working on is indeed one of the solution we are considering for this interoperability issue.

This said, the Remote Playback API completely abstracts the protocol to the web developer contrary to the Presentation API. That means that websites will not have to do sniffing and will only be told that there is or is not available devices depending on the user's setup. Because of this, we believe that the Remote Playback API will offer a good interoperability story for the platform. In other words, if the browser supports Chrome Cast, and the user has a one in its network, the API will reflect that the device is available but the page doesn't need to know the Cast protocol. If the user switches to a browser that only handles AirPlay, the API on this browser will report no available device. The interop issue will mostly be for the user as changing browser might provide a different experience depending on their hardware setup.

[avayvod]

@cynthia based on the dicsussion at the last spec review meting it doesn't appear you've see @mounirlamouri's response above.

Does it address your concerns about the protocol?

Also, w.r.t the failure to play remotely, the spec defines throwing DOMException of different types in different situations (like NotSupportedError, NotFoundError, NotAllowedError and so on).

[cynthia]

@avayvod Yes, you are correct - for some reason that reply slipped through my attention. (I blame a large backlog of mails, but that's mostly an excuse. Apologies to @mounirlamouri .)

First of all, I understand the rationale for the abstraction - I had a quick chat with @anssiko about the background when I was first tasked to look into this. Unfortunately, it doesn't address the concerns - but more on that below.

If the user switches to a browser that only handles AirPlay, the API on this browser will report no available device.

This is the part that I'm worried about. While the Safari-Apple TV and Chrome-Android pairing would be relatively obvious to the user after a bit of trial and error (and the tradeoff of not having to yak shave a transport protocol into the specification for this quirk makes sense) - this requires users to switch browsers for remote playback compatibility, which doesn't quite feel like something an open web standard should be doing.

e.g. For a user with a Apple TV that uses Chrome as their main browser, it would not show up as a playback device. The application running in Chrome wouldn't have any way of knowing that the user has an Apple TV which is incompatible with Chrome, so it wouldn't even have the contextual information needed to notify the user to switch to say, Safari. Users would intuitively need to know when to switch to what browser, based on experience. For a tech-savvy user, this wouldn't be a problem - I'm not sure how well this would work for the average user though. And as noted above, there are obvious pairings - but it's also not entirely obvious what browsers from vendors with no companion device offerings (e.g. Mozilla) would support, even to a tech-savvy user.

My understanding is that websites which plan to use this will need to explicitly make the users aware of the browser-playback device pairing in some form of documentation as their best bet, possibly with a list of working pairs of browsers and devices. This isn't great, both from the content developer or user's perspective.

It is understandable that utilizing existing transport mechanisms were the most logical way forward, and the TAG can't really stop implementations from shipping. The ideal way to deal with this is to implement a cross-implementation compatible transport, but I understand that is a long term goal.

This issue was raised in hope to find a better way to deal with the problem at hand and deliver a better experience to the users. I'd be happy to sit down and throw around ideas to at least make the user experience aspect better if that is the most pragmatic way forward.

(I'm pretty sure this comment could have been made a bit more concise.)

[mounirlamouri]

@cynthia you are right that this is not ideal but one goal of this API was to offer a way for the web page to trigger the current "fling" feature that browsers have implemented by default (Chrome and Firefox offer Cast on Android, Safari offers Air Play). A user trying to "fling" a video shouldn't have to care whether it is using the API or the built-in feature. The concerns of not being able to use an AirPlay device from Android is larger than this API and anyone with such a configuration will be aware of the issue. Therefore, in our opinion, it is very unlikely that a user will have an unexpected/bad experience with this API.

[mounirlamouri]

I don't think the issue can be realistically acted upon apart from getting the Open Screen Protocol supported widely and have device makers and user agent use it. This way, if UA_1 supports OSP, it will be able to connect to all devices implementing the spec and if UA_2 also supports OSP, switching from an UA to another would not affect the user's ability to connect to their devices.

Given that OSP is a new Working Group item, I believe we are on track to get there, or at least, we putting the pieces together. I am going to close this issue but please feel free to re-open if that doesn't sound reasonable.

[mfoltzgoogle]

We believe the OSP spec has added the features necessary to support interoperable Remote Playback. See https://w3c.github.io/openscreenprotocol/#remote-playback-protocol for the details.