Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RemotePlaybackState enum can become misleading when changing media.src #125

Open
jernoble opened this issue Jul 17, 2019 · 9 comments

Comments

@jernoble
Copy link

commented Jul 17, 2019

The steps for prompt() allow the promise returned to be rejected with NotSupportedErr if the media element cannot support remote playback. This allows, e.g., UAs which support MSE to indicate that a media element backed by a MediaSource cannot be played back remotely, and further allows sites to switch to file-based playback. But a page that changes the media's src/srcObject from a file-based URL to a MediaSource would only get a "disconnect" event, and would be unable to tell whether the user explicitly disconnected the remote playback session or whether the new src is unsupported.

This could be mitigated by adding a new RemotePlaybackState enum, such as (straw man) "unsupported", with a matching "unsupported" and onunsupported message and event handler.

Sites who start loading a media element with a file-based URL, successfully initiate remote playback with prompt(), and then switch the src/srcObject to a MediaSource would (straw man) receive a "unsupported" message which would allow them to switch back to a file-based URL.

@jernoble

This comment has been minimized.

Copy link
Author

commented Jul 17, 2019

This came up in the context of a recent bug report where YouTube.com is playing a video playlist in Safari while playing back wirelessly to an AppleTV, so this is a real world problem affecting a well-visited media site.

@mfoltzgoogle

This comment has been minimized.

Copy link
Contributor

commented Aug 14, 2019

CC @mounirlamouri for his thoughts.

This proposal seems reasonable to me.

Thinking through how resumption would work: if the video gets an unsupported event and switches back to a supported (file-based) URL, it could call prompt() again and the browser could handle that as a special case and resume playback on the same device (if it's still available).

We could add some additional resume() API if that isn't sufficient, but I don't see a need currently.

@mounirlamouri

This comment has been minimized.

Copy link
Member

commented Aug 16, 2019

Instead of adding a new state, could we add information to the disconnect event that explains why the disconnection happened?

@jernoble

This comment has been minimized.

Copy link
Author

commented Aug 20, 2019

@mounir said "Instead of adding a new state, could we add information to the disconnect event that explains why the disconnection happened?"

I'm not sure how that simplifies things. "disconnected" doesn't appear to imply that the element will attempt to reconnect when the src changes. Also, in the "Browser initiated remote playback" case, you'd either get a "disconnected" event without a matching "connecting" one, or no events at all, depending on your interpretation of the "Browser initiated remote playback" section.

@mounirlamouri

This comment has been minimized.

Copy link
Member

commented Aug 21, 2019

My concern is that by introducing a new state, it makes things more confusing. I see the RemotePlaybackState as a connection state and unsupported isn't much of a connection state. Can you go from unsupported to disconnected? or to connecting? Maybe there is a very simple graph that could show how to go from each of these states but after a quick look, it doesn't seem "simple".

I'm not sure how that simplifies things. "disconnected" doesn't appear to imply that the element will attempt to reconnect when the src changes.

Is it what we want though? If we tell the page what happened, shouldn't we let the page decide?

@jernoble

This comment has been minimized.

Copy link
Author

commented Aug 21, 2019

My concern is that by introducing a new state, it makes things more confusing. I see the RemotePlaybackState as a connection state and unsupported isn't much of a connection state. Can you go from unsupported to disconnected? or to connecting?

To answer your question with another question, can you go from disconnected to disconnected? I don't think that makes sense, but that's what would happen if you used the default media controls to try to initiate remote playback, or if the system/UA tried to initiate Remote Playback. You'd either get a disconnected event without a matching connecting one, or you'd get nothing at all, which is worse.

So, can you go from unsupported to disconnected? Sure, if the user explicitly selects local playback as a result of prompt(), or in a similar interface provided by the UA, or if the remote device becomes unavailable. Can you go from unsupported to connecting? Sure, if the video.src changes to a supported media type, or if the user selects a different endpoint that does support the current media type. I think all that state transition is clear and makes sense.

Is it what we want though? If we tell the page what happened, shouldn't we let the page decide?

As I said above, there's a hole in that flow, namely around UA-driven remote playback. I also think it's weird that every time a page gets a "disconnected" event, they have to check some status code to try and tell if the disconnection is recoverable. Another is that a) video.src = null; video.load(); video.src = 'foo.mp4', b) video.src = 'foo.mp4' and c) video.srcObject = mediaSource; video.src = 'foo.mp4' all have different behaviors. If you start with a connected remote playback session, b) will result in a connected remote playback session, c) will result in a disconnected remote playback session, and a) is not really clear about whether it will or not. Introducing a unsupported state would mean all three would result in the same behavior: "foo.mp4" would start playing remotely.

@mounirlamouri

This comment has been minimized.

Copy link
Member

commented Aug 23, 2019

I guess there is one key implementation point: I would assume that when receiving data that isn't compatible, the UA would disconnect from the remote screen and wouldn't attempt to auto-reconnect but instead leave it to the website to decide. You seem to suggest that the UA should keep a connection alive and attempt reconnecting as soon as possible. Is that correct?

@jernoble

This comment has been minimized.

Copy link
Author

commented Aug 27, 2019

This is currently the platform behavior of AirPlay on iOS yes. I'm not saying that the spec should require this behavior, but the spec should allow this behavior to occur.

That said, the only specced mechanism for a website to trigger remote playback is the prompt() function. It doesn't seem like reasonable behavior to present a "remote device picker" to the user, change the element's source, and present the same "picker" again in order to establish a connection. If UAs were given the freedom to allow a re-connect to a previously chosen remote device upon source change, I'm guessing most would do so.

@tidoust

This comment has been minimized.

Copy link
Contributor

commented Oct 10, 2019

This was discussed during the Second Screen WG F2F in Fukuoka.

Main options raised during that meeting:

  1. Add something like a connected-but-unavailable state. To get from this state to connected, application needs to change src.
  2. Use disconnected state and provide a reason. To get back to connected, there should be a reconnection mechanism that does not prompt the user.
  3. Stay with connected and fire a new event, noting that applications that don't listen to the new event will assume that everything's going well.

Also the point was made that supporting source switching may actually be optional.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.