Unable to resume playing primary HLS audio stream after failover

[makp9k]

Hello,

[REQUIRED] Content description

I use ExoPlayer to play an hls audio stream. My master playlist consists of primary and backup playlists, where the primary one sometimes restarts and begins producing segments starting from 0. But media sequences are in sync. After player switches from backup variant back to the primary, it is unable to continue playing the stream.

I checked the same stream with the hls.js player (http://github.com/video-dev/hls.js/) and it works fine there. There are also no issues with this stream on iOS.

Here are the playlist snapshots when switching variants:
from backup https://gist.github.com/makp9k/db98ed83c2efe264f97ae13faf22a221
to primary https://gist.github.com/makp9k/300f59087ab84b7e90d78d87200dfdd6

After some debugging there is a following picture in my head (please correct me if I'm wrong):
The DefaultHlsPlaylistTracker handles the primary playlist update and tries to find an overlapping segment in order to determine the start time. Because the last snapshot to compare with is not the currently playing playlist but rather the old primary playlist, it's not possible to find an overlap any more. In this case the primary playlist is aligned with the backup by setting it's startTime to primarySnapshotStartTimeUs
https://github.com/google/ExoPlayer/blob/release-v2/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/playlist/DefaultHlsPlaylistTracker.java#L408

// No segments overlap, we assume the new playlist start coincides with the primary playlist.
return primarySnapshotStartTimeUs;

And then the HlsChunkSource rejects this new playlist, because it's duration is shorter than the currently playing backup's
https://github.com/google/ExoPlayer/blob/release-v2/library/hls/src/main/java/com/google/android/exoplayer2/source/hls/HlsChunkSource.java#L464

// If the playlist is too old to contain the chunk, we need to refresh it.
return mediaPlaylist.mediaSequence + mediaPlaylist.segments.size();

It looks like the javascript library does the fallback a bit differently:
https://github.com/video-dev/hls.js/blob/master/src/controller/stream-controller.js#L305

 /* we are switching level on live playlist, but we don't have any PTS info for that quality level ...
    try to load frag matching with next SN.
    even if SN are not synchronized between playlists, loading this frag will help us
    compute playlist sliding and find the right one after in case it was not the right consecutive one */

What are the possible solutions to this issue? Is it possible for ExoPlayer to handle this setup the same way as the JS player? I suppose that adding EXT-X-PROGRAM-DATE-TIME tag should help, but I'm not able to modify those playlists.

[REQUIRED] Link to test content

I will send a master playlist link to dev.exoplayer@gmail.com.

[REQUIRED] Version of ExoPlayer being used

Tested with 2.8.2 and the 2.10.0

[REQUIRED] Device(s) and version(s) of Android being used

Samsung S6 / Android 7.0
Honor 9 Lite / Android 8.0

[marcbaechinger]

@AquilesCanta May I ask you to look into this and verify the given hypothesis?

[AquilesCanta]

A few questions before I look into this:

After player switches from backup variant back to the primary, it is unable to continue playing the stream.

What's happening exactly? Does it get stuck in the BUFFERING state? Or does it fail with a player error?

where the primary one sometimes restarts and begins producing segments starting from 0. But media sequences are in sync

Which media sequences are in sync? Back up and primary? If so, wouldn't that mean if primary resets, then back up should reset too?

Besides the master playlist link, can you provide specific reproduction steps? How often do the servers reset?

[makp9k]

Thank you for quick reply!

What's happening exactly? Does it get stuck in the BUFFERING state? Or does it fail with a player error?

The player gets stuck in the BUFFERING state, periodically trying to load the primary playlist.

Which media sequences are in sync? Back up and primary? If so, wouldn't that mean if primary resets, then back up should reset too?

I'm sorry for not being clear. What I meant was that if primary resets, the playlist it produces will start with a single segment "segment00000000.ts" while the back up playlist will have much more segments (like from "segment00069559.ts" to "segment00069648.ts"). But the media sequences will be in sync, so that it is possible to find an overlapping segment (based on #EXT-X-MEDIA-SEQUENCE + some offset) in both playlists.
For example, the snapshot of the primary playlist starts with a #EXT-X-MEDIA-SEQUENCE:1118846 and contains 7 elements, and the snapshot of the backup playlist starts with #EXT-X-MEDIA-SEQUENCE:1118761 and contains 90 elements. If I understand correctly, this should tell us, that the 0th segment in the primary playlist is the same one as the 85th in the backup.

Besides the master playlist link, can you provide specific reproduction steps? How often do the servers reset?

I used ExoPlayer’s demo app, where I added master playlist to the media.exolist.json. For testing purposes the primary server I sent to dev.exoplayer@gmail.com is down for 2 minutes every 2 minutes.

I will send you the fiddler's session archive where you can see the requests being made by the player. In that particular dump we can see that player was playing backup playlist, then switched back to the primary and stucked trying to reload it. After 2 minutes, as the primary server went down again, the player switched to the backup and continued playing the stream.

Thank you for your time.

[AquilesCanta]

• I think the content is malformed and we won't fix in ExoPlayer in the near future. The spec says that variants should provide the same content.
• In order to fix in the library, we would have to download a segment to peek the timestamps and then re-select a segment to actually consume content. The added complexity is considerable.
• In order to fix in the app, you can inject your own playlist tracker implementation so as to put program date time tags in the provided playlists. You can also implement program date time injection in the playlist parser, but that would require a stateful parser, which may have unexpected side effects.

[makp9k]

Thank you for your help! Interesting is that other players are able to consume this playlist without any issues. The problem I'm currently facing is that I don't quite understand where to get the program date time tag values from or how to simulate them. I tried to modify the playlist tracker so that it will align currently playing playlist and the loaded one to their ends, but this results in some strange stutter effect. The only solution that helped was to modify the HlsChunkSource#getChunkMediaSequence by removing switchingTrack condition, but I'm not sure whether this is a good way, considering that usually we have no access to this class.

[AquilesCanta]

Actually, forget about the program date time, you just need to implement your own playlist tracking logic (tracking logic means assigning the correct start time to the playlists) using the knowledge you already possess about the playlists. The tracking logic in the default playlist tracker can be found here. By implementing an equivalent method to use the sequence number or filename to assign a start time that matches, you should be able to achieve correct sync, assuming that the PTS in the segments match.

[fwjavox]

I think the content is malformed and we won't fix in ExoPlayer in the near future. The spec says that variants should provide the same content.

As far as I understand, "same content" means samething like same tv show or radio program, not the same binary data. Otherwise it would contradict one of the next points in the list that says: "Content that appears in a Media Playlist of one Variant Stream but not in another MUST appear [...]".

In order to fix in the library, we would have to download a segment to peek the timestamps and then re-select a segment to actually consume content. The added complexity is considerable.

The fix might be a lot simpler than that. In RFC section 6.3.2 it says that clients must use the relative position of the segment in the playlist to navigate across variants. Small example: The player is playing the third last segment of the primary backup and now fails to download the second last one. No matter how many segments there are in the backup playlist, the second last one in there will be the right one. For VOD playlist that is not really an issue but for live playlists it is. Players like hls.js and the native Apple HLS Player are doing it that way and it works.

By implementing an equivalent method to use the sequence number or filename to assign a start time [...].

That would violate the point "A client MUST NOT assume that segments with the same Media Sequence Number in different Variant Streams or Renditions have the same position in the presentation" from section 6.3.2 (see above).

Would you mind to reopen the issue for further discussion, please?

[AquilesCanta]

Otherwise it would contradict one of the next points in the list that says ...

That quote omits a very relevant piece (in bold):

Content that appears in a Media Playlist of one Variant Stream but
not in another MUST appear either at the beginning or at the end
of the Media Playlist file and MUST NOT be longer than the target
duration.

If my understanding is correct, then the length difference of both playlists shouldn't be > 10 seconds (i.e. the target duration), while in the provided sample, the difference is considerably bigger than that.

In RFC section 6.3.2 it says that clients must use the relative position of the segment in the playlist to navigate across variants.

That's what ExoPlayer does. The difference with your suggestion is that ExoPlayer uses the start of the playlist instead of the end of the playlist for relative positioning. In the samples you provided, this works well because playlists are aligned at the live edge (right?). But by changing the way we align playlists, we could actually break playlists that align at the start. Either case is legal, according to the HLS spec section quoted above.

Would you mind to reopen the issue for further discussion, please?

I'll discuss with the team, I suppose that it's in our interest to match the behavior of other media libraries. However, i don't think we will resort to fixes more complex than changing the alignment end.

Aside: can you prove the sample stream is legal, considering the piece of the spec quoted above?

[fwjavox]

However, i don't think we will resort to fixes more complex than changing the alignment end.

That would probably fix the problem.

Content [..] MUST NOT be longer than the target duration.

It's a bit unclear to me if this statement targets the segments (i.e. each segment separately) or the sum of the segments. I will think about it.