Meta issue for good DASH/HLS support in mpv

[ghost]

This is sort of a place holder issue, because it seems to come up all the time. May also serve for communication, random questions, or volunteers volunteering.

Existing support

FFmpeg has HLS support (aka m3u8). It also has DASH support (aka mpd), but this depends on an external library (XML parser) and thus is not enabled in FFmpeg by default.

ytdl_hook.lua and mpv's EDL support sometimes emulate DASH in special cases. It works by ytdl parsing the DASH manifest, returning it as JSON, and ytdl_hook.lua converting it to EDL. EDL has some mechanisms to make it efficient (for example not needing to open each EDL segment before playback start, which is normally done with EDL). This is here: https://github.com/mpv-player/mpv/blob/master/DOCS/edl-mpv.rst#mp4-dash

Problems

Most importantly, FFmpeg does not use HTTP2, which is apparently needed for getting pipelining at all. While FFmpeg's HLS demuxer tries to pipeline HTTP1.1 requests, this works badly and often not at all. It does not make any requests ahead of time. It does not open multiple HTTP connections at once (or use HTTP2 multiplexing). All this makes it slow due to added roundtrip latencies, or in other words, unnecessary waiting.

FFmpeg's DASH support has all the same problems as HLS, but in addition is often broken (judging by bug reports). It even seems to have some crashing bugs. The fact that it depends on an external library means we can't rely on its existence either.

Solution

I suspect for good support, we're going to need HTTP2. I have some libcurl support in a private branch, which may or may not help. But even then, FFmpeg would need to make requests ahead for better efficiency.

Someone could implement HLS/DASH support directly in mpv. Then we'd have control over fixing its issues. However, implementing these is probably not fun, given all the mess in the HLS "protocol", or DASH's fractal standard + XML nightmare.

Or someone needs to improve the existing support in FFmpeg. That means adding some sort of prefetch or parallel request support, and implementing either HTTP2 access, a libcurl wrapper, or something that would enable an API user (mpv) to receive prefetch requests. his someone would also need to be ready to fix all the issues in DASH whenever they happen, which should eventually result in solid DASH support.

[ghost]

Also I heard @tmm1 implemented some sort of custom HLS support on top of stream_cb. This is probably private code (that builds on top of libmpv), but I think it makes a nice anecdote about FFmpeg's HLS support, and how tmm1 specifically tried to improve it over the years. Correct me if I'm wrong.

[tmm1]

While FFmpeg's HLS demuxer tries to pipeline HTTP1.1 requests, this works badly and often not at all. It does not make any requests ahead of time. It does not open multiple HTTP connections at once (or use HTTP2 multiplexing).

FFmpeg currently uses two separate http connections, and will issue a http request for the next segment using the second connection while the first segment is being downloaded. Both connections also use http/1.1 keepalives so a new TCP+TLS negotiation is not required per segment.

While this improved throughput compared to the previous single-connection-per-segment, it's still pretty lacking and you're correct that further improvements are very hard to add given how hls.c works.

We ended up giving up on hls.c for these reasons, and now have our own hls client written in a higher level language which has a full http/2 client with pipelining and connection pooling. Our client includes bandwidth measurement and will open up as many parallel connections as possible to max out available bandwidth. To integrate with mpv, we expose an open/read/seek interface to C, and then use stream_cb to tie everything together. It's pretty hacky but it works well.

[hanguofu]

Thanks for creating this thread to brainstorm good ideas :) tmm1's comment gives us some light on segments congestion for HLS/DASH applications. My questions go a little deeper :

1. how to probe ( including hack ) the internal buffer of decoding pipeline so that we can pause the decoder at the last moment when we find out a bad internet connection for accessing segments ?
   Will the decoder resume to work ( including the display unit ) if we feed mpv with new segment during pause ? Is there any smarter solution for this manual intervention ?
2. What is the current solution to the above underflow case ?

[ghost]

In what context? Are you also using a stream_cb thing or so?

[hanguofu]

haven't decided yet . Can stream_cb be applied to DASH application ? If yes, we would like to have a trial provided that the decoder can be paused before underflow happened and it can resume to work after we feed mpv with new segments. Regards !

[ghost]

Yes, you could emulate DASH by using stream_cb and feeding it a concatenation of all segments (with the init segment as first thing). I think that's exactly what tmm1 does. You can block in the stream_cb read callback, then the player will pause and wait until there's more data available again as configured with https://mpv.io/manual/master/#options-cache-pause-wait . Though currently, it doesn't wait until the decoder/output has finished playing data buffered in these stages. Could be corrected I guess.

[tmm1]

One tricky thing that our solution is not good at handling: sometimes HLS and DASH will split out audio and video into separate segments. These would need to be merged back together before feeding to stream_cb/avformat-mpegts

[ghost]

You could use this: https://github.com/mpv-player/mpv/blob/master/DOCS/edl-mpv.rst#separate-files-for-tracks
Though it's a feature that could disappear any time.

[ghost]

(Turns out nobody uses HTTP2 for media delivery. They all just create multiple TCP connections go get around HTTP1.1 inability to multiplex/pipeline well.)

[AstralStorm]

Current hack using youtube-dl cannot work with DASH or HLS seeking for live streams.
HLS seeking is supported in Streamlink (hls-live-restream), but DASH is not due to internal structuring reasons.

mpv could be the first player that can properly seek those streams... A lot of work I bet. Perhaps libdash can be used for this.