Low latency streaming capabilities using <availabilityTimeOffset> and partial segments

[smedegaard]

My company's use case requires as low latency as possible. Real time would be ideal. Reading the white paper Overhead and Performance of Low Latency Live Streaming using MPEG-DASH made me want to test the capabilities of HTTP 1.1 to handle chunks of segments.

In this paper, we proposed to use the amendment 1 of DASH in combination with Gradual Decoding Refresh encoding and to deliver media frames up to the frame.
— From the conclusion of the white paper

We would love for something like this to be implemented in Dash.js

@bbcrddave mentioned on Slack that returning partial segments could be implemented with fetch API.

Any help and comments on getting this work started is greatly appreciated.

[davemevans]

I nentioned fetch because it allows partial arraybuffers to be returned so the response can be "streamed". This was not possible using XHR. Coupled with specialist server behaviour, something like described in the paper may be possible.

availabilityTimeOffset is not currently implemented at all.

[NicolasWeil]

Hey folks, we got 3 GPAC akamaized URLs for you to test with the fetch API - more to come soon with burnt-in timecode :
http://gpaclowlatency.akamaized.net/content/frag40/live-ast400.mpd
http://gpaclowlatency.akamaized.net/content/frag200/live-ast400.mpd
http://gpaclowlatency.akamaized.net/content/frag500/live-ast500.mpd
It's playing fine with super low latency in Osmo player, now we're eager to see playing also with the same kind of latency in dash.js !

[TobbeEdgeware]

@NicolasWeil Thanks for the URLs. How can the latency be measured?
If the video would be synchronized to wall clock as in the dash-live-source-simulator, it would be easy to do it, but that doesn't seem to be the case, or am I missing something?

[NicolasWeil]

@TobbeEdgeware You're right, there's no wall clock on these streams. They can only be used for testing the functional aspects of the fetch API integration, which is loading the chunks as they come and feed the buffer with it.
I'm working with Romain Bouqueau from GPAC Licensing in order to provide a wall-clocked reference stream at a later stage, I will update this thread when it's available. Thanks !

[istolga]

+1 for low latency DASH, we need it.

[dsparacio]

Just FYI - Here are some settings I have played with to get it down as far as possible without code changes. No claims here just some info. This is using 1 second segments.

player.enableManifestDateHeaderTimeSource(false);
player.setLiveDelayFragmentCount(0);
player.setFragmentLoaderRetryInterval(300);
player.setBandwidthSafetyFactor(0.5);
player.setStableBufferTime(2);
player.setBufferTimeAtTopQuality(2);

[istolga]

Thank you @AkamaiDASH. It is still the same 5 seconds delay

[wilaw]

Prioritizing this request from the DASH IOP group.

The live stream simulator now has available test vectors which expose low latency chunking. This stream has 8s segments with 1s chunks:

https://vm2.dashif.org/livesim-chunked/chunkdur_1/ato_7/testpic4_8s/Manifest300.mpd

It is possible to set the ato and chunkdur to floating point numbers like chunkdur_0.5/ato_7.5/ in the URL. The original content is 30Hz, so setting ato to 7.9 and chunkdur to 0.1 is also possible.

We'd like to see support for low latency implemented against these vectors. This has several sub-requirements

1. An implementation of FETCH in place of XHR, since XHR only allows the first byte to be read once the last byte has been received. Note that as of today, only Chrome and Safari support the ability to make multiple response reads. You will need to polyfill around this for other browsers and perhaps use the clone() method, while watching for the memory impact. If fetch is not supported in a browser, then the library should fall back to XHR and do the best it can with regards to latency.
2. Adjustment of the live stream edge finding algorithm, which can now be much more aggressive. We'd suggest either a new API which can place the library in an ultra-low latency mode, or else make robust setting the existing setLiveDelayFragmentCount(value) and setLiveDelay(value) to zero.
3. Review of the ABR switching logic, which may need some optimizations (or a new mode) when dealing with very small forward buffers. Additionally, the safety ratio of estimated throughput over bitrate-we-are-willing-to-play should be raised in this mode to protect against fluctuations in throughput draining the small buffer.

Edgeware already have a branch development with some of these changes. A player from this branch is available here: https://vm2.dashif.org/chunked-player/samples/dash-if-reference-player/ (Chrome only). It can play 8s segments with only 4s end-to-end latency. Since the buffer is ~3.5s, it should be possible to wring another 3s of latency out of that implementation (if the implementor wants to live with a 0.5s forward buffer.)

Contact @TobbeEdgeware for details about that branch and for cooperation in porting those enhancements to /development.

[epiclabsDASH]

@TobbeEdgeware, would it be possible adding a new test vector to the live stream simulator for low latency chunking that has more than one video rendition?

[yogevNisim]

What will be the expected latency of this new version?

[epiclabsDASH]

@yogevNisim, in our tests we got latencies around 2-4 seconds using 8 second segments and 1 second chunks. Our buffer is around 1.5-2 seconds so latency could be reduced a bit more although not too much because it could impact playback experience (rebuffering events).

We are going to send the PR we are working on, which is based on @TobbeEdgeware team one, shortly so anyone can start testing this feature.

[epiclabsDASH]

Guys, nightly version of dash.js already supports low latency and the plan is include it as an experimental feature in the next official release (to be released by April 30th). Please, feel free to test it . Feedback y more than welcomed.

Please, note low latency mode should be activated clicking "Low Latency Mode" checkbox in dash.js options panel.

[epiclabsDASH]

Already implemented and available since dash.js 2.6.8.