Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CMAF Low Latency Chunks #1525

Closed
fadomire opened this issue Aug 6, 2018 · 49 comments
Closed

CMAF Low Latency Chunks #1525

fadomire opened this issue Aug 6, 2018 · 49 comments
Assignees
Labels
component: HLS The issue involves Apple's HLS manifest format status: archived Archived and locked; will not be updated type: enhancement New feature or request
Milestone

Comments

@fadomire
Copy link
Contributor

fadomire commented Aug 6, 2018

Hello,

i wanted to know if it is planned to support CMAF Low latency chunks ?

I saw reference to that in the following ticket comparing dash.js and shaka, but no status for shaka
#1351

thanks for the work by the way

@theodab theodab added type: question A question from the community type: enhancement New feature or request and removed needs triage type: question A question from the community labels Aug 6, 2018
@theodab
Copy link
Collaborator

theodab commented Aug 7, 2018

We will look into this feature. We will add it if we can, but we'll need to investigate how it would work with MediaSource.

@shaka-bot shaka-bot added this to the Backlog milestone Aug 7, 2018
@fadomire
Copy link
Contributor Author

fadomire commented Aug 7, 2018

thanks for the reply

it may be interesting to check how dashjs made their implementation when the subject come in the roadmap

one thread discussing low latency specifics :
Dash-Industry-Forum/dash.js#1474

@PatrikCarlander
Copy link

PatrikCarlander commented Mar 21, 2019

Is it possible to priorities CMAF support? CMAF is what the OTT industry asking for now. At NAB-Show (April 6 - 11, 2019 | Las Vegas) we have to show CMAF with Dashif instead of Shaka and it's sad.

@PatrikCarlander
Copy link

The live stream simulator has available test vectors which expose low latency chunking. This stream has 8s segments with 1s chunks:

https://vm2.dashif.org/livesim-chunked/chunkdur_1/ato_7/testpic4_8s/Manifest300.mpd

It is possible to set the ato and chunkdur to floating point numbers like chunkdur_0.5/ato_7.5/ in the URL. The original content is 30Hz, so setting ato to 7.9 and chunkdur to 0.1 is also possible.

@joeyparrish
Copy link
Member

Hi, everyone. Just to clarify: CMAF is already supported. Low-latency CMAF will be added later this year.

@joeyparrish joeyparrish added the component: HLS The issue involves Apple's HLS manifest format label Mar 21, 2019
@PatrikCarlander
Copy link

It was Low-latency CMAF we asked for.
thanks for the reply

@joeyparrish
Copy link
Member

@PatrikCarlander, yes, but I was clarifying based on your first comment "Is it possible to priorities CMAF support". But I think we are all on the same page now.

@Romain-Bernier
Copy link

Any update about Low Latency CMAF ?
Thanks

@joeyparrish
Copy link
Member

We haven't begun work on this yet. We anticipate working on this in Q3 2019.

@ramoncaldeira
Copy link

ramoncaldeira commented Jul 11, 2019

+1 for low-latency DASH

@PatrikCarlander
Copy link

Q3 2019 is ending soon, is this feature postponed? LL is a highly requested feature from our customers. We need this in Shaka to get LL in chromecast too.

@ismena
Copy link
Contributor

ismena commented Sep 16, 2019

Hi @PatrikCarlander
So, there's good news and there's bad news. The bad news is, we got entangled in pre-work for this feature (cleaning up and optimizing our hls code), so we weren't able to start actual work when we'd hoped to.
The good news is, that the pre-work is almost done (I think I saw the last optimization chunks go into into code review last week :)). We've already allocated time for LL and the developer has started research.
I'll sync with them next week (after the research) to see if we can give you a better estimate.
I will also do my best to cajole our TL @joeyparrish into making LL one of our biggest priorities for the rest of the year.

@PatrikCarlander
Copy link

PatrikCarlander commented Nov 20, 2019

This can be a good input to define the scope.
ExoPlayer Low Latency design doc

@michellezhuogg
Copy link
Contributor

Hello @PatrikCarlander ,

I saw that both Community Hls spec and Apple Hls spec are mentioned in the document. Which one are you looking forward to use?

@PatrikCarlander
Copy link

@michellezhuogg For a start ULL-DASH, after that we will see if Apple LLHLS will replace Community LHLS.

@joeyparrish joeyparrish modified the milestones: Backlog, Backlog 2 Jan 28, 2020
@quanghuy1288
Copy link

waiting for hls cmaf low-latency :)

@buhlaze
Copy link

buhlaze commented Mar 6, 2020

I am confused about cmaf. Is Cmaf just a mpeg-dash manifest with Availabilitytimeoffset added to it? I want to use it instead of mpeg-dash. Also, how do you make a CMAF manifest with ffmpeg with or without shaka player? I have looked everywhere and can't find an example of how to do so.

@leandromoreira
Copy link
Contributor

@buhlaze CMAF is a container (much like .TS or .MKV or yet .MP4)

@buhlaze
Copy link

buhlaze commented Mar 18, 2020

low latency CMAF, in order to take advantage of it, does it just involve adding Availabilitytimeoffset to your dash manifest in the BASEURL with segmented video X number of seconds and a player that supports low latency, for example LowLatencyEnabled? Could I please get an explanation?

@kanongil
Copy link

The preload hints don't support the BYTERANGE-START attribute. It incorrectly uses a non-specced BYTERANGE attribute.

@michellezhuogg michellezhuogg reopened this Oct 5, 2020
shaka-bot pushed a commit that referenced this issue Oct 5, 2020
BYTERANGE-START is the byte offset of the first byte of a preload
hinted resource. This attribute is optional, and defaults to 0.

Example:
'#EXT-X-PART:DURATION=1.02,URI="fs271.mp4",BYTERANGE=18000@43000'
'#EXT-X-PRELOAD-HINT:TYPE=PART,URI="fs271.mp4",BYTERANGE-START=61000'

Issue #1525

Change-Id: Ib3f7feb6bd5d3fd09b89962c067029a448d020e7
@joeyparrish
Copy link
Member

@kanongil, thanks for pointing that out! It has now been fixed in the master branch.

@vlee-harmonicinc
Copy link

It is great that shaka player support ll-hls!
Apart from delta update (_HLS_skip), is there any plan for Blocking Playlist Reload (_HLS_MSN, _HLS_part) as well?
https://tools.ietf.org/html/draft-pantos-hls-rfc8216bis-08#section-6.2.5.2

@joeyparrish
Copy link
Member

Our internal streaming model requires us to update all playlists, rather than just the media playlist for the currently-selected streams. So we decided that blocking playlist updates might not make sense for Shaka Player at this time.

@michellezhuogg, let's discuss this Tuesday and double-check our assumptions to make sure we made the right call on that.

@joeyparrish joeyparrish reopened this Nov 3, 2020
@joeyparrish
Copy link
Member

@michellezhuogg and I are not sure if we made the right decision about blocking playlist updates. We originally decided not to do them because we're updating all media playlists.

It might work to block on a media playlist update, even if we are going to update all playlists serially. If all are going to be blocked on the same segment number, for example, the first one would block, and subsequent playlist requests would not, since that segment would already be ready in all the others by the time the first was ready.

At least, that's what we're thinking now.

I'm also wondering if there are other configs that would need to be changed to accommodate this. Manifest request timeout comes to mind, but we could also override that per-request in the HLS parser without changing the configuration per se.

Any thoughts on this? Anyone interested to experiment with the HLS parser to see what breaks if we block?

vlee-harmonicinc pushed a commit to vlee-harmonicinc/shaka-player that referenced this issue Nov 5, 2020
@vlee-harmonicinc
Copy link

A client MUST NOT assume that segments with the same Media Sequence Number in different Variant Streams or Renditions have the same position in the presentation; Playlists MAY have independent Media Sequence Numbers.
https://tools.ietf.org/html/draft-pantos-hls-rfc8216bis-07#section-6.3.2

For example Apple's LL-HLS CMAF stream have different duration and MSN for audio and video. Blocking one media playlist still works if playback latency large enough to tolerate the difference, but probably not a prefect design.

I tested implement blocking playlist reload via saving next MSN and next part number in streamInfo and marking preload in segmentReference. It seems works, but there is some other issue related to lint/compile :P

By the way, how about #1936: Lazy-load HLS media playlists on adaptation ?

A client MUST load the Media Playlist file of every Rendition selected for playback in order to locate the media specific to that Rendition. But, to prevent unnecessary load on the server, it SHOULD NOT load the Playlist file of any other Rendition.

Lazy-load is not mandatory and not directly related to this issue, but LL-HLS better have lazy-load in practice. Since the partial segment of LL-HLS is short, update frequency of LL-HLS is much higher than standard HLS. For example, if playing a LL-HLS stream with 2 audio 5 video and 1s partial duration, updating all playlist requires downloading. >7 media playlist per second, while only 2 selected playlists are necessary.

@joeyparrish
Copy link
Member

The problem with lazy-loading is Shaka Player's current architecture. Manifest parsers don't know what the currently-selected stream is or when it changes, so there is no good way to know which playlist needs to be updated. We designed manifest parsing to be completely decoupled from streaming, but it seems that in this case, they are too decoupled.

michellezhuogg pushed a commit that referenced this issue Nov 12, 2020
With "lowLatencyMode" enabled, "rebufferingGoal" is set to 0.001, and inaccurateManifestTolerance is set to 0 by default. However, in some cases longer rebufferingGoal helps to avoid new rebuffering.
Besides  "lowLatencyMode", this code change adds the  "autoLowLatencyMode" config. When "lowLatencyMode" config is disabled and "autoLowLatencyMode" config is enabled, and the manifest provides low latency features, we automatically activate the lowLatencyMode.
If "lowLatencyMode" is enabled, "autoLowLatencyMode" has no effect.

Issue #1525
@avelad
Copy link
Collaborator

avelad commented Nov 18, 2020

I think it would be nice to add a table in the readme of the features needed for low latency to work. For example in DASH it is necessary the ReadableStream API

@maximk-1
Copy link

maximk-1 commented Dec 2, 2020

Hi shaka team!
Do you plan to release LL-DASH support this year?

@joeyparrish
Copy link
Member

@htleeab wrote:

For example Apple's LL-HLS CMAF stream have different duration and MSN for audio and video. Blocking one media playlist still works if playback latency large enough to tolerate the difference, but probably not a prefect design.

Based primarily on this, and on our current architecture (manifest parsers completely independent of streaming), @michellezhuogg and I have decided not to add blocking playlist updates for now. If a feature meant to reduce latency on updates ends up adding latency because of how it interacts with our existing architecture, it's best not to do it for now.

@Mkozlov1 wrote:

Do you plan to release LL-DASH support this year?

Well, we certainly planned it that way, but has anyone else noticed how 2020 has gone so far? We're doing what we can with very limited resources (team members temporarily allocated to other internal priorities). We intend to wrap up the features (basically done now) and close enough serious bugs to feel confident about the stability of the master branch before we release v3.1. We hope that can be accomplished before the end of the year, but if it takes until January 15th to make v3.1 usable, we'd rather have the release work as intended than have it technically come out before midnight on Dec 31.

With that, I think our initial LL feature work can be called complete. We're open to discussions of blocking playlist updates and other features with accompanying architecture changes at a later date.

Thanks!

@shaka-project shaka-project locked and limited conversation to collaborators Feb 6, 2021
@michellezhuogg
Copy link
Contributor

michellezhuogg commented Mar 25, 2021

Hello everyone,

The Low Latency HLS and Low Latency Dash features are ready for testing on our Nightly build demo:
https://nightly-dot-shaka-player-demo.appspot.com/demo/#audiolang=en-US;textlang=en-US;lowLatencyMode=true;uilang=en-US;panel=HOME;build=uncompiled
We would like to get a few sample manifests for testing, or any bug report before our official release of v3.1.
Thank you!

@shaka-project shaka-project unlocked this conversation Mar 25, 2021
@kanongil
Copy link

While it seems to be somewhat workable with LL-HLS on a good connection, the ABR experience quickly breaks down. There are several minor/major issues, but I won't list them, as it seems that this has hardly been tested. Note that using the Chrome network throttling is not representative of real-world issues, since it always triggers a navigator.connection 'change' event, which resets the estimator.

I will mention one non-ABR problem: Playback does not respect the INDEPENDENT=YES flag, and starts from whatever fragment, causing playback issues.

Anyway, I probably won't be using this as long as it doesn't support blocking playlist reloads.

@michellezhuogg
Copy link
Contributor

michellezhuogg commented Mar 29, 2021

@kanongil ,

Thank you for your feedback! Would you please share some sample low-latency manifests with us, so that we can test and triage the minor/major issues that you mentioned? You can send them to shaka-player-issues@google.com if they're private.
We'll also look into the problem with INDEPENDENT=YES flag. Thank you!

@michellezhuogg
Copy link
Contributor

Closing the issue due to inactivity. If anyone has further questions or comments, please open another new issue. Thank you!

shaka-bot pushed a commit that referenced this issue Apr 15, 2021
We should only request playlist delta updates when the lowLatencyMode is
enabled, and when the server supports the feature. We can check if the
playlist has a "CAN-SKIP-UNTIL" tag to know if the server supports that.

Issue #1525

Change-Id: I304a008aeb3a9e019f27304ba836d31538c94fbf
@shaka-bot shaka-bot added the status: archived Archived and locked; will not be updated label Jun 6, 2021
@shaka-project shaka-project locked and limited conversation to collaborators Jun 6, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
component: HLS The issue involves Apple's HLS manifest format status: archived Archived and locked; will not be updated type: enhancement New feature or request
Projects
None yet
Development

No branches or pull requests