Impossible to watch live HLS WebVTT subtitle streams #3087
Comments
|
Works just fine for me, except that it's so slow. |
|
What do you mean by working? I'm trying to say that mpv will wait until entire stream is downloaded and in case of live translations you can't wait, you need to display current video/audio/subtitle chunks right away. |
|
Your test stream has only subtitles, so that's not necessarily a good test case. |
|
Yes, sorry. There are lives at vlive.tv which illustatre my issue best. I'm spawning mpv like this |
|
Also, this is a bit arificial example, but works in the exact same way: Video will start only when entire sub stream is downloaded. |
|
Definitely seems to be a ffmpeg issue: |
ghost
added
the
|
Seems like this (and neighbour commits in |
|
It's unlikely that anything will be done after 2 years of silence... |
|
Is it possible to reread subtitle file somehow in mpv (maybe with Lua scripting)? It would be extra hacky but it's possible to fetch those subs in separate thread, parse and progressively add to the .vtt file. Any ideas except actually fix ffmpeg? |
|
You certainly could somehow load the file in Lua (for which you would need some sort of http lib for Lua) and add the subtitles via memory:// or so. But fixing it in ffmpeg is preferable. |
|
@anssih could you look at this please? I tried to apply your patch anssih/FFmpeg@7d7fc89 to the latest ffmpeg git, it aplied with one merge conflict, but seems like it doesn't produce any effect. Maybe I'm missing something and need to apply some more patches from your branch? It's just not clear after 3 years which ones are still releveant and which were merged. Any hints will be greatly appreciated. Thanks. |
|
@Kagami Yep, you are correct in that that commit (plus the two WebVTT MPEG-TS timestamp ones) is supposed to fix this - not sure why it isn't working for you, likely the code has changed too much since them. When I wrote that I was planning to rewrite it in a cleaner way before getting that merged in, unfortunately I ran out of time... I'm still hoping to get back to that, but can't make any promises on the timeframe. |
|
Let me try one more time, maybe I did something wrong. I need these 3 commits: |
|
@Kagami Yes. |
|
@anssih I built your hls branch and seems like it works in the same way as stock ffmpeg, it downloads the entire playlist first: The same for live stream: Maybe I don't understand something? |
|
That's because as I've earlier pointed out, |
|
Seems like it assumes that sub streams just don't exist because I get same behaviour with subs streamed over HTTP/Pipe. What if I will write subs to file in separate process (using |
|
Standalone subtitle streams (not wrapped in HLS, just text) are always fully downloaded by libavformat on opening.
Terrible, but should work. You'll have to mind the timestamp offsets of course. |
See mpv-player/mpv#3087 for details.
|
@wm4 note that there are plans to export HLS subtitles info in youtube-dl vlive extractor. It will freeze mpv on lives with subtitles... Probably some option needed to disable auto sub-add in ytdl_hook? |
|
Few patches that probably address this issue: https://ffmpeg.org/pipermail/ffmpeg-devel/2016-November/202451.html |
|
Didn't realise there was a thread here for this. @Kagami yep, I'm interested in this too, for the same reason actually, watching vlives. I've sent in those patches to the mailing list but they don't seem to be getting anyway so I've been using the squash_hls_refactor branch from my ffmpeg fork and it's working well enough, haven't noticed any issues. I wouldn't recommend anyone else use that branch though because I'm not merging updates from the official ffmpeg repo to it. |
|
There were some concerns about code duplication. Anyway, ping the patches if you think you've done everything you could to fix the things that were pointed out. |
|
I sent new patches with the refactoring work done here: https://patchwork.ffmpeg.org/patch/1995/ |
|
@coreynicholson, that's great! Speaking of watching vlives, there are also several issues which prevent watching streams properly: |
|
Ah, just noticed there was a reply to my patches, it seems it breaks some tests (http://ffmpeg.org/pipermail/ffmpeg-devel/2017-January/205150.html), so I need to look at that. |
Ah, missed that, sorry. I will build your branch and test then.
At least in gentoo that can be solved just by putting patches to |
Sure, I should mention however, that whilst I haven't any noticed any issues when downloading streams to mkv or mp4 using ffmpeg it doesn't seem to work that well with mpv. Some things I've noticed when building mpv with those patches:
I don't know what could be causing the first issue, but the delay in the second issue may be due to the cache and is probably unavoidable without downloading all the subtitle tracks preemptively. The third issue may be due to not discarding duplicate packets when seeking, not sure if this needs to be done on the mpv side or ffmpeg side. Having said all that, I used it to watch one of the year end award shows that was 3+ hours long and had subs without any problems. |
|
Normally enabling a subtitle should start where it does, no matter whether it's an external file or not. For internal subtitles (which applies to HLS), there's a heuristic to seek the entire demuxer back by 10 seconds, but it's active only if either packet DTS or packet position are strictly monotonically increasing. Overlapping subtitles could happen because if the packet position is >=0 (i.e. set), it uses that as unique ID to filter out duplicated subtitles that come from playing a range of the file again. |
|
Thank you @Kagami for this work. My question is : is there another mpv compilation script that uses your patch or your repo? |
|
I've tried to apply the changes on the current version of ffmpeg but failed at it. I got lost in the HTTP persistent code and probably failed to close or reuse the HTTP object at the right places, thus after a couple of segments being fetched it just hung. Using the current version of ffmpeg, you could replace completely /libavformat/hls.c with https://github.com/coreynicholson/FFmpeg/blob/56b5e362224a2b403c4a5bccbf01f0df61ba3760/libavformat/hls.c And also patch libavformat/webvttdec.c manually based on anssih/FFmpeg@65af023 MPV doesn't really require any changes. Ideally, the changes brought to hls.c should be upgraded to fit in the latest version of it which supports persistent connections. In any case, this falls into ffmpeg. Hope this helps. If someone has the courage to upgrade the hls.c file to the latest, that would be greatly appreciated. |
|
was already brought up upstream. |
|
Do you know if there is an open ticket for this in ffmpeg? |
|
i am not sure, i remember it being brought up on the mailing list i believe. just open a new one i would say. |
|
This problem has not been solved |
|
@Akemi Where we can see this was brought up upstream? I only find this old closed issue on ffmpeg bug tracker. Versions: Have this issue randomly. Using PipeWire (might be realated). Sometimes |
|
Fixed the above problem with |
It seems like libavformat doesn't return intermediate results of WebVTT HLS streams so mpv stucks until entire subtitle stream is downloaded which is inappropriate for live translations.
Consider this test stream. If you try to play it either directly or with
--sub-file, mpv hangs. Basically the real life use case is HLS video/audio + HLS subtitles, you can find such streams e.g. at vlive.tv.Test output:
I realize that this is probably ffmpeg's bug but creating issue anyway in case if you know some workaround. My knowledge of ffmpeg is poor so probably there is some option to fix this, etc.
mpv/ffmpeg are latest from git.
The text was updated successfully, but these errors were encountered: