Youtube stream only last 4 hours are downloaded.

[kwinz]

Checklist

• I'm reporting a broken site support
• I've verified that I'm running youtube-dl version 2020.03.24
• [X/~] I've checked that all provided URLs are alive and playable in a browser
• I've checked that all URLs and arguments with special characters are properly quoted or escaped
• I've searched the bugtracker for similar issues including closed ones

Description

For this video
https://www.youtube.com/watch?v=ntu4ik4otfo
on Desktop and youtube-dl only the last 4 hours are downloaded.
On the mobile app it is possible to view the full 12 hours of the live stream.

Verbose log

Only pasting the short log here, because the issue is easily reproducable. My log is not relevant:

kwinz@markus-home:/mnt/c/Users/Markus/Desktop$ youtube-dl -v -k https://www.youtube.com/watch?v=ntu4ik4otfo
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'-k', u'https://www.youtube.com/watch?v=ntu4ik4otfo']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2020.03.24
[debug] Python version 2.7.6 (CPython) - Linux-4.4.0-43-Microsoft-x86_64-with-Ubuntu-14.04-trusty
[debug] exe versions: ffmpeg 3.4, ffprobe 3.4
[debug] Proxy map: {}
[youtube] ntu4ik4otfo: Downloading webpage
[youtube] ntu4ik4otfo: Downloading m3u8 information
[youtube] ntu4ik4otfo: Downloading MPD manifest
[debug] Default format spec: bestvideo+bestaudio/best
[download] Hospitality House Party - Dock Special-ntu4ik4otfo.mp4 has already been downloaded and merged

[kwinz]

Something changed 1 hour ago and now the full 12 hours of the stream are visible on Desktop at the same URI. I don't know what was changed. https://www.youtube.com/watch?v=ntu4ik4otfo&lc=UgxY4u5nf5lzR2kn1c14AaABAg.97Hxh9vu8Sq97JreVRUSbw
I am very curious how youtube-dl handles this and what was changed, and why the mobile app has always been able to show the full 12 hours.

[dstftw]

Handles what? youtube-dl acts like browser no more no less.

[acroyear]

uh, i think this should have been given a little more consideration. I'm running into the same problem right now trying to grab a 7 hour live-stream and it is only returning the last 4 hours (to the second, exactly 4:00:00) even though the other 3+ are still available online. This happens with both merging separate streams as well as trying a lower-res version that is HLS and unified audio-video already. Still only getting the last 4 hours right now.

---

or maybe just document the 4 hour thing for live streams as a temporary thing.

I just went and tried it again today (about 12 hours later), just like the original issue reporter did, and now I'm getting all 7+ hours. It seems it is just a matter of live streams that it takes time before the video content is fully converted to a stable stream.

[Igetin]

Yes, YouTube takes some time to process longer streams after they have ended. Usually on mobile you can see the whole stream immediately, but for browsers it can take even over 24 hours to fully show up.

This is a bit problematic if you run automatic archivals periodically for certain channels, since you can’t really know if youtube-dl has actually fully downloaded livestreamed videos without going through all of them one by one. There won’t be any errors, because this is simply how YouTube serves the content to browsers. I don’t know if there’s any way to programmatically determine if a livestream has been processed after it has ended. So I’m not really sure if this can be handled in a good way from youtube-dl’s side.