-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Prefetch audio variants of a particular language for seamless audio switch. #6128
Comments
I agree to change the prefetch management. Please do a single PR with this change.
We currently use prefetch for Low Latency and will use it in Preload API. So there must be the following options at least: If disableAudioPrefetch is false, we ignore prefetchAudioLanguages. If disableAudioPrefetch is true, then we use prefetchAudioLanguages and only prefetch the specified audios. If you agree with this I will be happy to review the implementation of the two PRs |
Sounds good. I'll probably get started with what we have already and then update it with the new options. And I'll be happy to break up it up into multiple PRs if that's wanted. |
Implements the behavior outlines in shaka-project#6128. Adds `streaming.prefetchAudioLanguages` and `streaming.disableVideoPrefetch` options. Does not currently have `disableAudioPrefetch` and `disableTextPrefetch`.
Implements the behavior outlines in shaka-project#6128. Adds `streaming.prefetchAudioLanguages` and `streaming.disableVideoPrefetch` options. Does not currently have `disableAudioPrefetch` and `disableTextPrefetch`.
Implements the behavior outlines in shaka-project#6128. Adds `streaming.prefetchAudioLanguages` and `streaming.disableVideoPrefetch` options. Does not currently have `disableAudioPrefetch` and `disableTextPrefetch`.
this sounds like a really cool feature, I wanna make sure I understood it. So incentive is to have ABR like switch experience but for audio where user can switch languages, it will switch as quickly as possible but user will not experience a pause while audio changes? |
yup, that's exactly right. In reality, there's the slightest of gaps, but it's pretty unnoticeable. Most importantly is that the player doesn't need to wait to download more segments and can switch instantly. This is also why a low rebufferingGoal is needed. It represents the amount of buffer needed before playback starts, and given we've already prefetched at least one segment worth of content, we don't need shaka to wait before playback starts.
We're still doing more testing. However, we haven't seen many issues assuming rebufferingGoal isn't set to 0 or a very very low number. This is also why we thought that
The thought here is to minimize memory usage so that we don't keep the video segments downloaded if we're potentially keeping a bunch of audio segments downloaded. However, we're not sure if it's strictly necessary but can't imagine that it hurts to have this option. |
Have you read the FAQ and checked for duplicate open issues?
Yes, there isn't something specific, though it builds on top of #4784.
Is your feature request related to a problem? Please describe.
I want to switch audio streams with a small a gap in playback as possible.
Describe the solution you'd like
Extend segment prefetch to also prefetch alternate audio streams.
Describe alternatives you've considered
Additional context
We had a requirement for seamless audio switch, and to enable that we extended the segment prefetch feature to be able to prefetch the segments of the alternate audio streams. Then, when we want to switch streams, we use the segment prefetch that's mapped to the specific audio stream.
To do this, we added the following options, one of which can also help reduce prefetched segments:
streaming.prefetchAudioLanguages
: A list of languages that we match audio streams against for knowing whether we should prefetch that stream. i.e['en', 'es', 'jp']
to English, Spanish, and Japanese audio languages. Or['en']
if you had multiple different English language audio tracks.disableVideoPrefetch
: don't prefetch video segments.More specific technical details: modify segment prefetch to not delete segments on get. Instead, it has an evict method which is given a time, and it removes segments before this time. This allows keeping the current segment prefetched so that if we switch to another audio stream and then back, we already have the segment available. Then in the streaming engine, we create a map between all the audio streams we care about and their associated segment prefetches and when the variant switches we grab the relevant segment prefetch from the map. In conjunction with setting rebufferingGoal to 0 or a very small number, the switches between audio streams can happen quickly enough that they aren't noticeable.
This work is already complete, and we'd like to contribute it upstream. We tried to make the options be generic, but there's definitely room to refine these if necessary.
The text was updated successfully, but these errors were encountered: