Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Youtube live stream download ends after 6 hours #20516

Open
4ndriH opened this issue Mar 31, 2019 · 9 comments
Open

Youtube live stream download ends after 6 hours #20516

4ndriH opened this issue Mar 31, 2019 · 9 comments

Comments

@4ndriH
Copy link

@4ndriH 4ndriH commented Mar 31, 2019

I have been downloading live streams on YouTube and noticed that it always ends after 6 hours (+/- a few seconds). At first I thought its something with youtube-dl but after recording it with VLC I got the same result. I looked through the help article but could not find anything related to this. Is there anything you can do about it?
I am aware that this is not the most pressing issue there is because few people are downloading such long live streams (at least I think so) and it might prove difficult to find a solution for this because it seams to be something on YouTube's side.

Due to the immense code output I can not add much of value. I will try my best though.

C:\Users\XXX>youtube-dl -v https://www.youtube.com/watch?v=UVxU2HzPGug                                            
[debug] System config: []                                                                                               
[debug] User config: ['--no-mtime', '-o', 'C:/Users/XXX/Downloads/%(title)s.%(ext)s', '-f', 'bestvideo[ext=mp4]+bestaudio[ext=m4a]/mp4']                                                                                                  
[debug] Custom config: []                                                                                               
[debug] Command-line args: ['-v', 'https://www.youtube.com/watch?v=UVxU2HzPGug']                                        
[debug] Encodings: locale cp1252, fs mbcs, out cp850, pref cp1252                                                       
[debug] youtube-dl version 2019.03.18                                                                                   
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.17763                                                          
[debug] exe versions: ffmpeg N-92396-g55e021f39b, ffprobe N-92396-g55e021f39b                                           
[debug] Proxy map: {} 
[https @ 0000017b83195980] Opening 'https://r8---sn-nfpnnjvh-9ans.googlevideo.com/videoplayback/id/UVxU2HzPGug.1/itag/91/source/yt_live_broadcast/requiressl/yes/ratebypass/yes/live/1/cmbypass/yes/goi/160/sgoap/gir%3Dyes%3Bitag%3D139/sgovp/gir%3Dyes%3Bitag%3D160/hls_chunk_host/r8---sn-nfpnnjvh-9ans.googlevideo.com/ei/mJSgXN7hDdOT1gKwiLewBQ/gcr/ch/playlist_type/DVR/initcwndbps/16760/mm/32/mn/sn-nfpnnjvh-9ans/ms/lv/mv/m/pcm2cms/yes/pl/45/keepalive/yes/manifest_duration/30/mt/1554027565/disable_polymer/true/ip/2a02:1205:503c:3f00:ec95:e57f:1a4b:f6eb/ipbits/0/expire/1554049272/sparams/ip,ipbits,expire,id,itag,source,requiressl,ratebypass,live,cmbypass,goi,sgoap,sgovp,hls_chunk_host,ei,gcr,playlist_type,initcwndbps,mm,mn,ms,mv,pcm2cms,pl/signature/9C310477E509897A1866FF4CB32AF60A0689414C.01A6A51C9605FAD0CDFB9CFE4F854041E4729957/key/dg_yt0/playlist/index.m3u8/sq/2269225/goap/clen%3D18585%3Blmt%3D1553951362306609/govp/clen%3D22709%3Blmt%3D1553951362306606/dur/2.000/file/seg.ts' for reading
[https @ 0000017b831eac00] Opening 'https://r8---sn-nfpnnjvh-9ans.googlevideo.com/videoplayback/id/UVxU2HzPGug.1/itag/91/source/yt_live_broadcast/requiressl/yes/ratebypass/yes/live/1/cmbypass/yes/goi/160/sgoap/gir%3Dyes%3Bitag%3D139/sgovp/gir%3Dyes%3Bitag%3D160/hls_chunk_host/r8---sn-nfpnnjvh-9ans.googlevideo.com/ei/mJSgXN7hDdOT1gKwiLewBQ/gcr/ch/playlist_type/DVR/initcwndbps/16760/mm/32/mn/sn-nfpnnjvh-9ans/ms/lv/mv/m/pcm2cms/yes/pl/45/keepalive/yes/manifest_duration/30/mt/1554027565/disable_polymer/true/ip/2a02:1205:503c:3f00:ec95:e57f:1a4b:f6eb/ipbits/0/expire/1554049272/sparams/ip,ipbits,expire,id,itag,source,requiressl,ratebypass,live,cmbypass,goi,sgoap,sgovp,hls_chunk_host,ei,gcr,playlist_type,initcwndbps,mm,mn,ms,mv,pcm2cms,pl/signature/9C310477E509897A1866FF4CB32AF60A0689414C.01A6A51C9605FAD0CDFB9CFE4F854041E4729957/key/dg_yt0/playlist/index.m3u8/sq/2269226/goap/clen%3D19057%3Blmt%3D1553951362306617/govp/clen%3D20391%3Blmt%3D1553951362306613/dur/2.000/file/seg.ts' for reading
[https @ 0000017b83340e80] Opening 'https://manifest.googlevideo.com/api/manifest/hls_playlist/id/UVxU2HzPGug.1/itag/91/source/yt_live_broadcast/requiressl/yes/ratebypass/yes/live/1/cmbypass/yes/goi/160/sgoap/gir%3Dyes%3Bitag%3D139/sgovp/gir%3Dyes%3Bitag%3D160/hls_chunk_host/r8---sn-nfpnnjvh-9ans.googlevideo.com/ei/mJSgXN7hDdOT1gKwiLewBQ/gcr/ch/playlist_type/DVR/initcwndbps/16760/mm/32/mn/sn-nfpnnjvh-9ans/ms/lv/mv/m/pcm2cms/yes/pl/45/dover/11/keepalive/yes/playlist_duration/30/manifest_duration/30/mt/1554027565/disable_polymer/true/ip/2a02:1205:503c:3f00:ec95:e57f:1a4b:f6eb/ipbits/0/expire/1554049272/sparams/ip,ipbits,expire,id,itag,source,requiressl,ratebypass,live,cmbypass,goi,sgoap,sgovp,hls_chunk_host,ei,gcr,playlist_type,initcwndbps,mm,mn,ms,mv,pcm2cms,pl/signature/9C310477E509897A1866FF4CB32AF60A0689414C.01A6A51C9605FAD0CDFB9CFE4F854041E4729957/key/dg_yt0/playlist/index.m3u8' for reading
[https @ 0000017b831eac00] Opening 'https://r8---sn-nfpnnjvh-9ans.googlevideo.com/videoplayback/id/UVxU2HzPGug.1/itag/91/source/yt_live_broadcast/requiressl/yes/ratebypass/yes/live/1/cmbypass/yes/goi/160/sgoap/gir%3Dyes%3Bitag%3D139/sgovp/gir%3Dyes%3Bitag%3D160/hls_chunk_host/r8---sn-nfpnnjvh-9ans.googlevideo.com/ei/mJSgXN7hDdOT1gKwiLewBQ/gcr/ch/playlist_type/DVR/initcwndbps/16760/mm/32/mn/sn-nfpnnjvh-9ans/ms/lv/mv/m/pcm2cms/yes/pl/45/keepalive/yes/manifest_duration/30/mt/1554027565/disable_polymer/true/ip/2a02:1205:503c:3f00:ec95:e57f:1a4b:f6eb/ipbits/0/expire/1554049272/sparams/ip,ipbits,expire,id,itag,source,requiressl,ratebypass,live,cmbypass,goi,sgoap,sgovp,hls_chunk_host,ei,gcr,playlist_type,initcwndbps,mm,mn,ms,mv,pcm2cms,pl/signature/9C310477E509897A1866FF4CB32AF60A0689414C.01A6A51C9605FAD0CDFB9CFE4F854041E4729957/key/dg_yt0/playlist/index.m3u8/sq/2269227/goap/clen%3D18861%3Blmt%3D1553951362306624/govp/clen%3D24287%3Blmt%3D1553951362306623/dur/2.000/file/seg.ts' for reading
[https @ 0000017b83340e80] Opening 'https://manifest.googlevideo.com/api/manifest/hls_playlist/id/UVxU2HzPGug.1/itag/91/source/yt_live_broadcast/requiressl/yes/ratebypass/yes/live/1/cmbypass/yes/goi/160/sgoap/gir%3Dyes%3Bitag%3D139/sgovp/gir%3Dyes%3Bitag%3D160/hls_chunk_host/r8---sn-nfpnnjvh-9ans.googlevideo.com/ei/mJSgXN7hDdOT1gKwiLewBQ/gcr/ch/playlist_type/DVR/initcwndbps/16760/mm/32/mn/sn-nfpnnjvh-9ans/ms/lv/mv/m/pcm2cms/yes/pl/45/dover/11/keepalive/yes/playlist_duration/30/manifest_duration/30/mt/1554027565/disable_polymer/true/ip/2a02:1205:503c:3f00:ec95:e57f:1a4b:f6eb/ipbits/0/expire/1554049272/sparams/ip,ipbits,expire,id,itag,source,requiressl,ratebypass,live,cmbypass,goi,sgoap,sgovp,hls_chunk_host,ei,gcr,playlist_type,initcwndbps,mm,mn,ms,mv,pcm2cms,pl/signature/9C310477E509897A1866FF4CB32AF60A0689414C.01A6A51C9605FAD0CDFB9CFE4F854041E4729957/key/dg_yt0/playlist/index.m3u8' for reading
[https @ 0000017b831eac00] Opening 'https://r8---sn-nfpnnjvh-9ans.googlevideo.com/videoplayback/id/UVxU2HzPGug.1/itag/91/source/yt_live_broadcast/requiressl/yes/ratebypass/yes/live/1/cmbypass/yes/goi/160/sgoap/gir%3Dyes%3Bitag%3D139/sgovp/gir%3Dyes%3Bitag%3D160/hls_chunk_host/r8---sn-nfpnnjvh-9ans.googlevideo.com/ei/mJSgXN7hDdOT1gKwiLewBQ/gcr/ch/playlist_type/DVR/initcwndbps/16760/mm/32/mn/sn-nfpnnjvh-9ans/ms/lv/mv/m/pcm2cms/yes/pl/45/keepalive/yes/manifest_duration/30/mt/1554027565/disable_polymer/true/ip/2a02:1205:503c:3f00:ec95:e57f:1a4b:f6eb/ipbits/0/expire/1554049272/sparams/ip,ipbits,expire,id,itag,source,requiressl,ratebypass,live,cmbypass,goi,sgoap,sgovp,hls_chunk_host,ei,gcr,playlist_type,initcwndbps,mm,mn,ms,mv,pcm2cms,pl/signature/9C310477E509897A1866FF4CB32AF60A0689414C.01A6A51C9605FAD0CDFB9CFE4F854041E4729957/key/dg_yt0/playlist/index.m3u8/sq/2269228/goap/clen%3D19024%3Blmt%3D1553951362306632/govp/clen%3D23371%3Blmt%3D1553951362306633/dur/2.000/file/seg.ts' for reading
[https @ 0000017b83340e80] Opening 'https://manifest.googlevideo.com/api/manifest/hls_playlist/id/UVxU2HzPGug.1/itag/91/source/yt_live_broadcast/requiressl/yes/ratebypass/yes/live/1/cmbypass/yes/goi/160/sgoap/gir%3Dyes%3Bitag%3D139/sgovp/gir%3Dyes%3Bitag%3D160/hls_chunk_host/r8---sn-nfpnnjvh-9ans.googlevideo.com/ei/mJSgXN7hDdOT1gKwiLewBQ/gcr/ch/playlist_type/DVR/initcwndbps/16760/mm/32/mn/sn-nfpnnjvh-9ans/ms/lv/mv/m/pcm2cms/yes/pl/45/dover/11/keepalive/yes/playlist_duration/30/manifest_duration/30/mt/1554027565/disable_polymer/true/ip/2a02:1205:503c:3f00:ec95:e57f:1a4b:f6eb/ipbits/0/expire/1554049272/sparams/ip,ipbits,expire,id,itag,source,requiressl,ratebypass,live,cmbypass,goi,sgoap,sgovp,hls_chunk_host,ei,gcr,playlist_type,initcwndbps,mm,mn,ms,mv,pcm2cms,pl/signature/9C310477E509897A1866FF4CB32AF60A0689414C.01A6A51C9605FAD0CDFB9CFE4F854041E4729957/key/dg_yt0/playlist/index.m3u8' for reading
[https @ 0000017b829a69c0] HTTP error 403 Forbidden
[hls,applehttp @ 0000017b8222b240] keepalive request failed for 'https://manifest.googlevideo.com/api/manifest/hls_playlist/id/UVxU2HzPGug.1/itag/91/source/yt_live_broadcast/requiressl/yes/ratebypass/yes/live/1/cmbypass/yes/goi/160/sgoap/gir%3Dyes%3Bitag%3D139/sgovp/gir%3Dyes%3Bitag%3D160/hls_chunk_host/r8---sn-nfpnnjvh-9ans.googlevideo.com/ei/mJSgXN7hDdOT1gKwiLewBQ/gcr/ch/playlist_type/DVR/initcwndbps/16760/mm/32/mn/sn-nfpnnjvh-9ans/ms/lv/mv/m/pcm2cms/yes/pl/45/dover/11/keepalive/yes/playlist_duration/30/manifest_duration/30/mt/1554027565/disable_polymer/true/ip/2a02:1205:503c:3f00:ec95:e57f:1a4b:f6eb/ipbits/0/expire/1554049272/sparams/ip,ipbits,expire,id,itag,source,requiressl,ratebypass,live,cmbypass,goi,sgoap,sgovp,hls_chunk_host,ei,gcr,playlist_type,initcwndbps,mm,mn,ms,mv,pcm2cms,pl/signature/9C310477E509897A1866FF4CB32AF60A0689414C.01A6A51C9605FAD0CDFB9CFE4F854041E4729957/key/dg_yt0/playlist/index.m3u8', retrying with new connection: Server returned 403 Forbidden (access denied)
[https @ 0000017b832b69c0] HTTP error 403 Forbidden
[hls,applehttp @ 0000017b8222b240] Failed to reload playlist 0
[https @ 0000017b832b1880] HTTP error 403 Forbidden
[hls,applehttp @ 0000017b8222b240] Failed to reload playlist 0
frame=648060 fps= 30 q=-1.0 Lsize=  290429kB time=06:00:01.99 bitrate= 110.1kbits/s speed=   1x
video:151391kB audio:126574kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 4.483879%
[ffmpeg] Downloaded 297399166 bytes
[download] 100% of 283.62MiB in 05:59:59

That is the final output of the download. The part before "[https @ 0000017b829a69c0] HTTP error 403 Forbidden" fills the rest of my cmd window and is not of much use imo.

cheers

PS: In case the devs deem this unnecessary or unsolvable do not hesitate to close this Issue.

@github-userx
Copy link

@github-userx github-userx commented Apr 1, 2019

My guess would be that youtube/google itself resets your token/connection after this amount of time.

@4ndriH
Copy link
Author

@4ndriH 4ndriH commented Apr 1, 2019

My guess would be that youtube/google itself resets your token/connection after this amount of time.

Wouldn't this cause the stream you are watching on YT itself to stop as well? I had the stream open next to the CMD and it was still running there.
Thinking about it, it might still be a token/connection thing with YT just refreshing it periodically. In that case it should be possible add a similar function to youtube-dl (maybe only when a stream is detected).

@rautamiekka
Copy link

@rautamiekka rautamiekka commented Apr 1, 2019

My guess would be that youtube/google itself resets your token/connection after this amount of time.

Wouldn't this cause the stream you are watching on YT itself to stop as well? I had the stream open next to the CMD and it was still running there.

Nah. Like, even Google would think about that possibility, and stealthily reset.

@FarmaanElahi
Copy link

@FarmaanElahi FarmaanElahi commented Apr 2, 2019

The token reset seems like a possibility here. YouTube might be refreshing the token after every 6 hour

@DanGamingTV
Copy link

@DanGamingTV DanGamingTV commented Feb 14, 2020

I made a bash script to get a new m3u8 url when ffmpeg stops running.
Feel free to use and adapt this to whatever you need.

#Explanation:
#
#function get() will take 3 paramaters: param 1 is the youtube uri, param 2 is the directory to download to, param 3 is the filename format (includes extension, such as .ts)
#get() then passes these args to function newurl(), along with an extra param which is a starting number for multiple files in the format 1test.ts, 2test.ts and so on. a new video file is created when ffmpeg stops (due to the m3u8 url expiring after 6 hrs)
#
#newurl() has 4 input params, 1 is the url, 2 is the directory, 3 is the file name, 4 is the number (from either get() or again())
#newurl() sets its own local vars from the params it gets, and adds one to the temp file number when it gets called again from again()
#newurl() then passes those params over to again()
#
#again() takes 5 params, 1 is the file number, 2 is the m3u8 stream url, 3 is the directory, 4 is the file name, 5 is the youtube video url (to be returned to newurl() when the stream url expires)
#
#Example usage: get "https://www.youtube.com/watch?v=21X5lGlDOfg" "C:\tmp" "NASA_TV.ts"

again() {
	tempnum=$1
	ffmpeg -i $2 -c copy $3$1$4 && newurl $5 $3 $4 $1 #ffmpeg downloads youtube stream to directory, then calls newurl() when ffmpeg stops
}
newurl() {
	local m3url=$(youtube-dl -f best -g $1) #store m3u8 stream url from youtube-dl as string
	local tempnum=$4 #file number, this is sent to again() so again() can send it back
	local tempdir=$2 #also sent to again so it can be sent back
	local tempfilename=$3 #also sent to again so it can be sent back
	let tempnum+=1 #add 1 to file number recieved from arguments
	again ${tempnum} ${m3url} ${tempdir} ${tempfilename} $1 #send required args to again(), and also send url so it can be sent back to newurl()
}
get() {
	newurl $1 $2 $3 0 #send args
}
#get "url" "directory" "filename"
@4ndriH
Copy link
Author

@4ndriH 4ndriH commented Feb 16, 2020

As I have never used bash I do not understand how I get it to run.
I do get "hello world" scripts to run but I am lost with the get command you have.
Like do I just write: m3u8.sh get "URL" "dir" "file" or do I have to first enter the file and then the get command? Both do not seem to work for me.
I am on a windows 10 machine if this has an impact.

@DanGamingTV
Copy link

@DanGamingTV DanGamingTV commented Feb 21, 2020

Hi, you can run the script by installing git bash on windows.
I haven't coded it to work with command line parameters, so you'll have to edit the bash file and change the command to what you need.
When git bash is installed, .sh files will automatically be associated with it.
Make sure you have ffmpeg installed and in your PATH.

@4ndriH
Copy link
Author

@4ndriH 4ndriH commented Feb 21, 2020

Hi,
Got it working.
Thanks a lot for the script. It works like a charm!

@Vozf
Copy link

@Vozf Vozf commented Apr 6, 2020

Same issue. I dont think that it's a url problem as i can continue downloading just by entering the same command after crash. This is a major issue and it is a shame that for more than a year nobody has fixed it.

@ytdl-org ytdl-org locked and limited conversation to collaborators Apr 6, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants
You can’t perform that action at this time.