Support for seamless track selection between adaptive multi-angle videos (DASH/SS/HLS)

[WassimErriha]

I am working on a project where the user is able to switch between different camera angles during playback. At the moment, Exoplayer is capable of making a seamless switch between HLS variants. The solution for doing so is mentioned in #2250. Also, from the commit references in #2600 it looks like there is going to be support for HLS video renditions in the next release.
My question is;
Is it going to be possible to make a seamless switch between renditions the same way it is possible to switch between variants ?
Also, from your experience , will this be easier to implement with DASH or SS content as apposed to HLS ?
Any pointers to how I could achieve this will be much appreciated !
Thank you !

[marcbaechinger]

@AquilesCanta May I ask you to comment on the HLS variants/rendition question?

[WassimErriha]

Any comments about this ?

[WassimErriha]

@AquilesCanta I can provide more details if the question is vague.
Thank you!

[AquilesCanta]

Apologies for the delay.

Is it going to be possible to make a seamless switch between renditions the same way it is possible to switch between variants ?

Can you clarify what you mean by seamless?

There are two dimensions to the question:

• Doing seamless track selection. For in-adaptive-group track selection, you can track Support seamlessly changing tracks during adaptive playbacks (DASH/SS/HLS) #2250. However, I understand you want track selection between tracks in different TrackGroups. If that's the case, I think the best approach is implementing a custom media source that changes the source of the samples directly, so that the rendererer never perceives a SampleStream replacement. This would be equivalent to the media source adapting between variants, except in this case, the adaptation would happen between chunk sources.
• Support for multiple video track groups. Already supported in DASH. For HLS, multi-video-group support is tracked by Attempting to change tracks in multi-angle adaptive HLS playlist #2600.

Does this answer your question?

[WassimErriha]

Excuse me if I am using the wrong terms :)
What I meant by seamless is that when you force the player to switch between variants, re-buffering occurs. I used the solution you mentioned in #2250 to make the switch between variants without a re-buffer. My understanding was that if I can switch between variants in this way, I should be able to switch between renditions (or camera angles) without re-buffering as well.
Your second bullet point suggests that multiple video track groups is already supported in DASH. Does this mean that if I change my content from HLS to DASH, I can then use a custom AdaptiveTrackSelection to switch between camera angles (or adaptations) ?
If not, then your first suggestion should be what I'm after.

[AquilesCanta]

Does this mean that if I change my content from HLS to DASH, I can then use a custom AdaptiveTrackSelection to switch between camera angles (or adaptations) ?

No, unfortunately. The reason is DashMediaSource (or any MediaSource) will not put non-inter-adaptive tracks in the same TrackGroup, which means that the AdaptiveTrackSelection (which has information of a single group) will not be able to adapt between them.

That is why you will most likely need changes in the MediaSource.

[ojw28]

I'm curious how you would envisage "user is able to switch between different camera angles during playback" working in practice. To get it seamless you would either have to:

1. Not switch immediately. This can be confusing from a user perspective, because they've pressed a button and a couple of seconds later they can still be waiting for something to happen. I guess this can be mitigated to some extent with a UI component that shows the switch is "in progress", or something like that.
2. Be downloading all of the different camera angles all of the time, so that you have buffered media just in case the user switches.

[ojw28]

Got it, thanks. This request is effectively a combination of #2600 (specifically for HLS) and #2250 (which is what you're requesting here). Hence I think we can close this as a duplicate, primarily of the second of these.

[WassimErriha]

I was intacipating a work around that I can implement in the meantime if this is not too much of a hassle.
Is this on the roadmap yet ?
Thanks for your time !

[ojw28]

It's not prioritized as part of our current roadmap. It's also not possible for us to provide detailed guidance on this kind of workaround without it consuming an unreasonable amount of time, sorry.