Skip to content
This repository has been archived by the owner on Jun 1, 2022. It is now read-only.

Remote control API

Jump to bottom
Tobias Hieta edited this page Oct 21, 2015 · 1 revision

User Experience

The areas marked "MUST" below have a degree of leeway when implemented on clients which have restricted or very different UI conventions (e.g. Roku).

  • Controller MUST display a player selection button during navigation and during media playback.
  • Button displays a selection dialog with all players:
    • If the controller is a player, it MUST list itself first.
    • The dialog MUST have a refresh button which refreshes the list of players by all means available.
  • Selecting a player results in:
    • The player selection button highlights to show a remote player is selected.
    • All play operations default to the selected remote player.
    • The selected remote player does NOT persist across app launches.
    • Failure to connect to the player (i.e. subscribe) or at any time, if sending a command fails, the controller MUST notify the user via a platform appropriate toast and consider the player disconnected. This implies button status and if the controller is showing the Now Playing screen, it should just back to showing local state.
  • When controlling a player, the controller MUST display a "Now Playing" button that changes its icon as appropriate for the type of media the player is playing. If the player is not playing any media, then this button is grayed out. When clicked, the controller MUST navigate to an appropriate control screen for the media type. This screen MAY display a "Now Playing" button if appropriate. When not controlling a player, the "Now Playing" button MUST lead to a local media screen as appropriate (e.g. local music).
  • The Now Playing button leads to a media-appropriate playback screen (e.g. music artwork + seek bar + buttons).
    • The remote screen SHOULD resemble the local playback screen as much as possible, or be identical (e.g. for music).
    • For remote video, the screen should show background art in place of actual playing video.
    • For photo galleries, the screen should show the same images as the player, and offer standard back/forward buttons.
    • As noted above, the screen MUST have a player selection button.
    • If multiple type of media are active (slideshow + music) the controller should show the foreground media (slideshow).
  • When in navigation mode, changing the selected player changes which player is being controlled.
  • When on the Now Playing screen, changing the selected player moves the media playback between players.
  • For example, if controller is showing a video being played remotely, and user selects local player, playback MUST be stopped on the remote player, and resumed on the local one.
  • If music is playing on the local player, and a remote player is selected, music playback MUST stop on the local player and be picked up on the remote player.
  • When playback is stopped on the player (e.g. a local user at the player stops a video and goes back to navigation), and the controller is on the Now Playing screen, it remains there, but shows the stopped state.

The following UI flow should be employed when running into these error conditions:

  1. Error on starting remote playback: Present user with a dialog indicating a failure, and offering the choice to play the media back locally. "Your video was not able to start, would you like to play on this device?"
  2. Error occurs during playback: If the application is active, and the controller isn't playing other media, the controller should issue a similar error to above and offer continuing the playback locally.

Player Advertising

  • Players MUST advertise as supporting the ‘plex’ protocol and the plex/media-player content-type.
  • Players MUST advertise their client identifier in order to allow controllers to uniquely identify them.

The response is line-terminated with CRLF:

HELLO * HTTP/1.0
Content-Type: plex/media-player
Name: Smoke
Port: 32467
Product: Plex for Windows
Protocol: plex
Protocol-Version: 1
Protocol-Capabilities: timeline,playback,navigation,mirror,playqueues
Resource-Identifier: 02a06dff-8014-472a-9aec-261a847a7b47
Version: 1.5.0.0

In order to test availability, all Plex resources which are running MUST respond to the /resources endpoint with a Media Container containing one Device child element for each functionality the device provides (e.g. player, server). This endpoint is implemented also by myPlex, which will return registered players and servers. In addition, the media server can return multiple players in the case of proxying, for example, AirPlay and DLNA players.

For example:

<MediaContainer>
  <Server title="Office" machineIdentifier="x" platform="MacOSX" platformVersion="10.9.0" /> <!-- All existing root attributes -->
  <Player title="Smoke" machineIdentifier="abc123" product="Plex for Windows" version="1.6.0.4" platform="Windows" platformVersion="6.3.9600" protocolVersion="1" protocolCapabilities="timeline,playback" deviceClass="pc" />
  <Player title="Living Room Apple TV" ... deviceProtocol="airplay" />
</MediaContainer>

The deviceClass parmeter is one of: phone, tablet, stb, tv, pc, cloud. The deviceProtocol attribute defaults to speaking "plex" protocol.

If the resources endpoint is exposing resources that are not local (e.g. myPlex, or the media server talking about non-proxied players), then each resouce element has a list of connections.

<MediaContainer>
    <Server ...>
        <Connection protocol="http" address="x.x.x.x" port="p" class="wifi">
        <Connection protocol="http" address="x.x.x.y" port="p" class="ethernet">
    </Server>
</MediaContainer>

Add new headers to GDM player advertising:

  • Protocol-Version (default is 0, 1 means you support this new API).
  • Protocol-Capabilities comma-separated values for parts of the protocol supported. Example header value: timeline,playback,navigation,mirror,playqueues (all five supported).
    • Players MUST implement the timeline and playback sections.
    • They MAY choose to not implement navigation.
    • Supporting playqueue means the player will manage moving the window of the playqueue as tracks progress as well as responding to requests refreshPlayQueue from a controller. If this capability is missing the controller might want to put the queue in read-only mode when it's being casted since there is no way to notify the player about updates.
  • Server returns new attributes in /clients (and new /resources) endpoint:
    • protocolVersion="x"
    • protocolCapabilities="x,y,z"

Player Discovery

There are three sources of player information for a controller:

  1. GDM discovery.
  2. The PMS /clients endpoint.
  3. The upcoming myPlex /resources endpoint, which will return players and servers.
  4. The upcoming PMS /resources endpoint, which will return server and proxied player.

Controllers should manage player discovery in an asynchronous fashion, so that presenting the player selection dialog is instantaneous from a UX perspective. In all likelihood code can be generally shared with existing server management code.

Controllers should ask all known servers for resources, as different servers could be proxying different players for various reasons. Players connectivity testing should be performed identically to server testing (where multiple connections might exist).

General Guidelines

All commands sent to a player MUST include a X-Plex-Client-Identifier header identifying the controller.

All messages sent to a controller (i.e. timeline requests) MUST include a X-Plex-Client-Identifier header identifying the player.

Controllers MUST send commands directly to any player that implements the ‘plex’ protocol. Servers proxying commands to a non-native player device (e.g. DLNA/AirPlay devices) may masquerade as ‘plex’ players by advertising as players and proxying requests from controllers. Consequently, all requests sent to a player MUST include an X-Plex-Target-Client-Identifier header, identifying the player the controller believes it is talking to. This allows a proxy player to multiplex requests to N players.

If the controller sees that the target identifier is the same as the identifier (all cases but proxied player), it's free to omit the header.

If present, players should verify that X-Plex-Target-Client-Identifier matches its own identifier and return HTTP error 404 if it doesn't.

Command Responses

Commands return an HTTP response code of 200 OK and an empty body if they succeed, and an HTTP error code if they fail. Besides the general 400 (badly formed request), general application-level errors are returned as HTTP 500 errors, along with code/status in an XML Response. Specific error codes will be defined as we go.

<Response code=”xxx” status=”This is a message”>

New Remote Control Commands

  • /player/timeline/subscribe?protocol=PPP&port=XXX&commandID=Y
    • Required Headers: X-Plex-Client-Identifier, X-Plex-Device-Name
    • Controller sends subscribe every 30 seconds.
    • Protocol is http or https.
    • Player Subscription times out after 90 seconds.
    • Player MUST respond to subscribe by immediately sending timeline data to the new observer. If the player isn't playing, this is used to pass initial state (e.g. volume levels) for each type.
    • If player is idle, it MUST at least pass back the minimal state information as shown above.
    • Players MUST accept and process commands received from unsubscribed controllers, even if they lack a commandID.
    • If the controller is currently subscribed to a remote player, the controller MUST disconnect from them when it receives a subscribe.
    • If the controller currently has subscriptions from controllers, when they subscribe to a remote player, the controller MUST disconnnect all subscriptions and send a final timeline with disconnected="1" in the container.
  • /player/timeline/unsubscribe
    • Required Headers: X-Plex-Client-Identifier, X-Plex-Device-Name
    • Takes effect immediately.
    • Removed by client identifier.
  • /player/timeline/poll?wait=0/1
    • Required Headers: X-Plex-Client-Identifier, X-Plex-Device-Name
    • This is an alternative to the subscribe command for controllers that cannot use persistent connections to receive updates from the player.
    • Player MUST reply to poll with the same response as is POSTed via timeline as defined below.
    • If wait=0 (or not specified), the player MUST respond immediately.
    • If wait=1, the controller MUST hold the request and respond to the player when the next timeline request would be sent to an actual subscriber.
    • Client MUST pass the X-Plex-Client-Identifier header with this request to allow the player to track these more ephemeral subscribers.
    • Note that commandID must be sent to this endpoint too, this allows debouncing with commands sent.
  • /player/playback/setParameters?volume=[0, 100]&shuffle=0/1&repeat=0/1/2
  • /player/playback/setStreams?audioStreamID=X&subtitleStreamID=Y&videoStreamID=Z
  • /player/playback/seekTo?offset=XXX - Offset is measured in milliseconds.
  • /player/playback/skipTo?key=X - Playback skips to item with matching key.
  • /player/playback/refreshPlayQueue?playQueueID=XXX - Tells the player that a playQueue needs to be refreshed. This is usually done by the controller when it has modified a playQueue that is being casted. The player should make sure to refresh the playQueue if it's currently displayed or playing. The playQueueID is also supplied to differenciate between multiple PlayQueues if available. When a PlayQueue is refreshed the player MUST update all controllers with a timeline request described below.
  • /player/navigation/home
  • /player/navigation/music - Navigate to the player’s music playback view, if music is currently playing.
  • /player/mirror/details - Can be called from the remote to show the pre-play screen for a specific item. The function takes the following args:
    • key is the relative path of the item to show
    • machineIdentifier is the machineIdentifier of the server that the item is located.
    • address is the ip address of the target server
    • port is the port used by the target server
    • protocol is a scheme: http or https. No protocol argument means http.
    • token is the access token the player should use to access the server. (NB - the controller must obtain a transient token from the server).

All of the playback commands take a mandatory type argument, to specify which media type to apply the command to, (except for playMedia).

Modified Commands

  • /player/playback/playMedia now accepts key, offset, machineIdentifier, address, port, protocol, token, and containerKey parameters instead of the previous path parameter. This is consistent with the /:/timeline parameters, and provides players with all information required to establish a connection.
    • key is a relative path that provides metadata for the item to play.
    • offset is an integer representing the number of milliseconds at which to start playing, with zero representing the beginning
    • machineIdentifier is the machine identifier of the target server.
    • address is the ip address of the target server
    • port is the port used by the target server
    • protocol is a scheme: http or https. No protocol argument means http.
    • token is the access token the player should use to access the server. (NB - the controller must obtain a transient token from the server).
    • If the video is coming from a myPlex queue, the machineIdentifier must be node, and address must be node.plexapp.com.
    • containerKey is a relative path that provides the media container containing a set of items for context around the key. All players MUST play continuously through the container from key.
    • mediaIndex is an optional parameter which must be used when the user has explicitly selected a particular media item to be played. It corresponds to the ordinal offset into the set of media items offered by the server. If this parameter is omitted, or the value falls outside that of the known set of media items, the player should simply try to select whatever media item is thought to be "best" by running it's own Media Decision Engine.
  • /player/playback/stepBack seeks back 15 seconds, or the expected platform value.
  • /player/playback/stepForward seeks forward 30 seconds, or the expected platform value.

Existing Commands

  • /player/application/setText?field=<field>&text=<text> - can be sent when textFieldFocused has been received. The field specifies the field which is getting text set.

These playback commands take an optional “type” argument (music, photo, video), in case there are multiple things happening (e.g. music in the background, photo slideshow in the foreground).

  • /player/playback/pause
  • /player/playback/play
  • /player/playback/skipNext
  • /player/playback/skipPrevious
  • /player/playback/stop

The difference between pause and stop is nuanced enough it's worth spelling it out, from the perspective of the player. A pause command:

  • Keeps the play queue intact.
  • If the player is showing full-screen playback, it still shows it.
  • Keeps the view offset (e.g. middle of a track).

On the other hand, a stop command:

  • Keeps the play queue intact.
  • If the player is showing full-screen playback, it exits back to nav.
  • View offset and play queue head is reset.

No change in the following commands:

  • /player/navigation/moveRight
  • /player/navigation/moveLeft
  • /player/navigation/moveDown
  • /player/navigation/moveUp
  • /player/navigation/select
  • /player/navigation/back

Command IDs

  • All commands to the player MUST have a commandID=X parameter.
  • Controllers MUST increment this ID every command sent.
  • Players MUST retain local transient state recording the last completed commandID for each subscription.
  • Players MUST send the last known commandID=X back with each timeline request for each subscription.
  • The commandID parameter in the subscribe message establishes a new command id baseline for the specific controller. The player MUST reset its local last known command id to that value when a subscribe message is received.
  • This allows players to easily de-bounce.

The controller uses the command IDs to do the following:

  1. After sending PlayMedia, the controller ignores timelines older than the last PlayMedia commandID.
  2. After sending a command that affects UI, the controller ignores the relevant subsets of the timelines that are older than that command. This makes things that naively resonate like scrubbing or volume control very smooth.

Timeline Request

  • Sent to all subscribed controllers at the following path: /:/timeline as a POST with XML, representing state for all media activity on the player.
<MediaContainer location="y" commandID="z" textFieldFocused="t">
  <Timeline type="video" state="stopped" controllable="..." />
  <Timeline type="music" state="paused" time="42442" duration="235225" machineIdentifier="yyy" address="x.x.x.x" port="32400" protocol="http" token="k" key="x" containerKey="y" volume="75" controllable="..." />
  <Timeline type="photo" state="playing" machineIdentifier="zzz" address="x.x.x.x" port="32400" protocol="http" token="k" key="x" controllable="..." />
</MediaContainer>

The X-Plex-Client-Identifier header should be checked to ensure the timeline is coming from the expected player. To support both polling and regular subscribers, this header MUST be on the the POSTs to timeline subscribers and a response header on the poll responses. In order to allow polling subscribers access to this header, you need to include the Access-Control-Expose-Headers: X-Plex-Client-Identifier header in the response to the poll GET request. More info on supporting CORS.

The following data is sent:

  • state is the state of the media playback. Valid values are stopped, paused, playing, buffering and error.
  • duration is the duration of the media, in milliseconds.
  • time is the current playback position, in milliseconds.
  • ratingKey is the corresponding rating key attribute in the media.
  • key is the corresponding key attributre in the media.
  • containerKey is the key for the container, if appropriate.
  • audioStreamID videoStreamID subtitleStreamID to pass selected streams.
  • machineIdentifier, address, port, protocol, token to identify server.
  • type=photo|video|music to identify for which media types the notification applies (because player might be playing music in the background and displaying photos full screen).
  • mediaIndex to identify the index of the <Media> currently playing, starting at 0. If not present, implies a single media.
  • partIndex to identify the index of the part curently playing, starting at 0. If not present, an index of 0 is assumed, with a single part.
  • partCount to indicate how many total parts are present. Omitting implies a single part.
  • A state of stopped can be optionally sent between items (e.g. tracks within an album). If this state is being sent, send continuing="1" to specify that there is something else to display.
  • controllable=[volume,shuffle,repeat,audioStream,videoStream,subtitleStream,skipPrevious,skipNext,seekTo,stepBack,stepForward,stop,playPause] to identify controllable parameters/valid actions. This is sent for each media type.
  • seekRange=X-Y is optionally sent to specify the specific subrange that is seekable, in milliseconds
  • volume=[0,100], shuffle=0/1, repeat=0/1/2 (where 1 = repeat one, 2 = repeat all). If parameter is not sent, it doesn't apply (perhaps to current media, e.g. volume to photos) and shouldn't be displayed.
  • location=[navigation,fullScreenVideo,fullScreenPhoto,fullScreenMusic] to identify where the app is currently.
  • textFieldFocused=[field] if the player has focused a text field and can receive input. The field specifies the specific text field which which is focused. If there is no field name to be obtained, use generic "field".
  • textFieldContent=[content] If there is a text field focused, contains the contents of the field.
  • textFieldSecure=1 Sent if the focused text field is secure (e.g. password).
  • playQueueID=XXX Sent if the player is currently playing a playQueue.
  • playQueueItemID=XXX Sent if the player is currently playing a playQueue.
  • playQueueVersion=XXX This should be set to the current playQueueVersion if the the player is modifying a playQueue, should be used by controllers to know when to update the playQueue. Players MUST send the playQueueItemID in the same timeline.
  • Player MUST send timeline request to controllers every time the state changes, or at a minimum, every time the play offset increments by a second. This allows the client to display a smooth play time without keeping its own timer.
  • Player SHOULD use HTTP/1.1 with persistent connections when talking to controllers to avoid overhead.

Sometimes errors occur during playback. In this case, a timeline will be sent to subscribers with state="error", and then appropriately set code and status attributes which parallel error responses.

In the case of a player wanting to disconnect a controller, they must send back a last timeline with disconnected="1" in the media container. A controller receiving this MUST not issue another timed subscribe.

Guidance for Remote Control of Play Queues

Play queues are a media container on the server that contains a window of items around the currently played item. They are created from a library item and can be added to, rearranged and navigated.

A play queue aware client will use a play queue as the container of items it's playing through. Not all content can be added to server play queues so clients are expected to have a local fallback equivalent that offers a useful subset of the functionality.

The play queue must be refetched as playback continues - since the returned container is a limited sized window.

A client that uses play queues has new responsibilities when acting as a remote controller or player. It cannot assume that the corresponding player or controller is aware of play queues.

Controller

If the controller is using a remote play queue, they should pass the play queue URL as the containerKey for playMedia.

Since the corresponding player may not recognise the container as a windowed collection, the URL should be postfixed with window=200 to provide a large window of the play queue when the oblivious player asks. 'own=1' should also be provided so that the server attributes the play queue to the correct client.

If the play queue is a local fallback then the last queued content should be sent as the containerKey. In trivial cases this will match the play queue, but will obviously not match a hand crafted play queue and will only play the last item. Local play queues are only needed for channels, legacy and synced servers.

The client will typically show UI for the active play queue and allow reordering, deletion and addition to it, as well as skipping directly to an item in the list. The controller is responsible for checking the connected player has the playqueues capability before allowing such actions.

When such an action is performed. The controller must inform the player with the refreshPlayQueue command.

If the attached player is capable of using play queues then it may also provide UI for manipulating the queue. The controller should notice any changes in the returned timeline's playQueueVersion and refetch the controller's play queue when this changes.

Player

The player needs to advertise its capability for play queues. This is done via the playqueues capability. This indicates it will refresh its play queue when asked and is capable of continuous playback of a collection of videos.

When passed a containerKey of the form playQueues\{id}?window=200&own=1 then a play queue aware player should fetch that play queue. The player also needs to deal with non play queue keys - both for legacy controllers and for when the controller has had a local play queue and resorted to sending the last added item.

A player that uses play queues is responsible for refetching when informed that in needs to. The refreshPlayQueue command will be sent by a controller in such a situation.

If the player displays UI for manipulating the play queue and the user modifies it then the player must return the new version in the timeline back to the controller.

Clone this wiki locally