Best way to have multiple devices play a single UDP/MPEG-TS stream in sync?

[boulaycote]

I am using ExoPlayer 1.5.12, and willing to move to newer versions.

The video is a low bitrate (10mbits) 4K MP4. I am streaming a UDP/MPEG-TS stream. Every frame is a keyframe to prevent bleed and blocking. To make sure devices are synced, whenever one starts buffering, I reset the player making sure I have the latest data.

Here are the issues I am facing now:

• Resetting ExoPlayer causes a delay and creates a freeze frame of course until it starts again.
• Resetting a player (meaning releasing the old one) sometimes causes an ANR.
• Quality of the stream is lacking mostly because of the "every frame is a keyframe" restriction.

The plan is to use this in displays where no user can interact with the app. Meaning an ANR or a crash is problematic.

Thoughts?

[ojw28]

@andrewlewis should be able to provide some high level advice on syncing ExoPlayer instances across devices, although it's not something we support directly and so beyond that is something you'll have to figure out for yourself.

Regarding the content itself:

• I wouldn't class 10Mbit/s as low bit-rate.
• It's slightly unclear whether you're playing MPEG-TS/UDP or an MP4. You'd probably want to be using HTTP + a seekable media format like MP4 or MKV to do this kind of thing. There is no need to make every frame a key-frame if you do this.

[andrewlewis]

I'm not sure I have enough information about what sort of streams are being played to address this properly, but maybe this will be useful anyway:

Resetting ExoPlayer causes a delay and creates a freeze frame of course until it starts again.

I assume you're not trying to play audio. Here are some (untested) suggestions on what to try: you probably want to make sure the players buffer for as little time as possible, so you could try setting the player's buffer size to zero (minBufferMs/minRebufferMs). With this change, if a player buffers it will return to the READY state more quickly. To get perfect synchronization across devices you will also need to make the player media clocks have the same position. You can do that by synchronizing the devices' clocks accurately (e.g. using NTP) or equivalently periodically calculating the offset from the correct time in your own code. Then use this time to provide a MediaClock to ExoPlayer by making your video renderer implement MediaClock, providing a position that is consistent with the synchronized clock time. Then every device will have almost exactly the same player position so should show frames at the same time. You will also need some way to make sure the clock position and sample timestamps are consistent (e.g. loop the input stream and if the clock position modulo the input stream duration is 'too far' from the sample timestamp do a seek -- though note that this only works if the input needs to be looped and is seekable which may not apply for your use case).

Resetting a player (meaning releasing the old one) sometimes causes an ANR.

This sounds like an unrelated issue. If you want us to investigate this, please file a new issue including all the information in the issue template, especially a bug report.

Quality of the stream is lacking mostly because of the "every frame is a keyframe" restriction.

By making every frame a keyframe, you prevent the video compression algorithm from taking advantage of temporal redundancy, so the quality will be much lower at the same bitrate. You can increase the bitrate to deal with this, though I agree with Olly that 10 Mbit/s sounds quite high already for playback on a mobile device. Alternatives include increasing the keyframe interval slightly (so artifacts from dropped and out-of-order packets are visible for a short time), or using periodic intra refresh, which refreshes the whole picture gradually.

[boulaycote]

Precision: Stream is MPEG-TS over UDP.

I have achieved near perfect synchronization by lowering the buffer size to the bare minimum. Rebuffering, following your advice, happens very infrequently. I could not go around making every frame a keyframe as out of order packets is too frequent and blocking is too apparent.

Question: Devices are connected through ethernet to the video source, is 10 Mbit/s still high in that case? Or is it simply the processing power of a mobile device that cannot handle higher quality?

[andrewlewis]

Glad to hear you were able to get the devices synchronized.

Regarding the bitrate: 10 Mbit/s sounds like a relatively high bitrate for a mobile application, but if this is a kiosk-style use case where you can choose the device(s) and test them then it probably doesn't matter. If you increase the bitrate to get better quality, playback as a whole will gradually get more computationally expensive (especially with MPEG-TS) but you can just pick a value that works!