Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upGitHub is where the world builds software
Millions of developers and companies build, ship, and maintain their software on GitHub — the largest and most advanced development platform in the world.
Complete support for 1080p and 4k video on YouTube #3887
Comments
|
The higher quality video streams (1080p, 1440p, 2160p) are available as video-only format called DASH. You can run: youtube-dl --format 'bestvideo+bestvideo' 'https://www.youtube.com/watch?v=iNJdPyoqt8U'and youtube-dl will download the best audio and video streams and merge them together afterwards (you'll need to have ffmpeg or avconv installed for that). See also here: #2458. Also, the |
|
DASH is not a "video-only" format, its a streaming format that steams both audio and video simultaneously and allows for dynamic on the fly changes to the resolution as the client side detects changing line conditions. Here is the correct string if you wan to download both audio and video at the same time. You still have to mux them together. |
|
So this may not, strictly speaking, be a youtube-dl problem but it is related to this issue. It basically revolves around the new 'dash' split streams and the definition of "best". In their infinite wisdom google has posted 128k AAC as the best DASH stream for now when in many of the other premuxed streams it is better e.g. 192k AAC in many of the 720p muxed streams. Still worse is that the DASH video isn't even the same bit rate as the muxed streams at the same resolution. This is exacerbated by the fact that --format 'bestvideo+bestaudio' preferentially grabs the DASH streams so you won't actually get the "best" with 720p only titles (or if you limit). Example on this Kesha video (no comments, kind of a random choice) :-) https://www.youtube.com/watch?v=edP0L6LQzZE if you do a --format 'bestvideo+bestaudio' you get a 720p DASH post-muxed stream with video @2062 kb/s and audio @125 kb/s. However a direct download of the older pre-muxed 720p stream has video @2729 kb/s and audio @192 kb/s. Now the video might be almost imperceptibly different but the audio is definitely worse, over compressed, and hard filtered at 16kHz (I looked at the spectrum). I don't know if there is a good solution other than to prefer pre-muxed streams when they are both the same resolution or limited to the same resolution by --max-quality. The biggest problem is the optimal solution to 1080p and higher right now is to get the audio stream from the pre-muxed 720p which is a bandwidth and processing headache. If you don't you get crappy audio, at least on the couple I have checked. |
|
So just to document this someplace, and this is probably far from the best place, here is my ffmpeg command. It assumes a couple things: -You have downloaded the version with the best audio (might be 720p but I suppose it could be lower ones too with the same track and that would save bandwidth). -You have downloaded the video only "DASH" stream of your chosen 1080p+ resolution. -Both of these are hopefully the same length within a fraction of a second (for music videos with all the cuts, usual lack of actually seeing singing and mouth, and the fact that the audio was never recorded at the same time anyhow this tends not to be as critical as some sort of dialog). This might be an issue with longer videos. ffmpeg -i "Input Good Audio (720p?).mp4" -i "Input Good DASH Video Only (1080p+).mp4" -c copy -map 0 As mentioned and IMHO, at this point if you want 720p or that is the highest resolution available do not grab the "DASH" streams (which is what 'bestvideo+bestaudio' does), grab the muxed 720p with a "-f 22". |
|
Wow! A comment board for code development should not auto-parse ascii to emoticons! :-( |
|
OK last observation before I quit spamming this topic. I dug up that Kesha video from 2010 when I'd previously downloaded it (let's say for my teenage daughter, yeah that's it... OK so it wasn't that random...) and I'd grabbed the 720p stream. I presume this predates all the DASH stuff. The video in there was previously 2000 kb/s and the audio was a 127 kb/s band-limited, version, however not the same, nor quite as hard 16 kHz band-limited as the current 125 kb/s DASH version. It seem that some (many?) 720p muxed streams have better video and audio now but the equivalent DASH ones are worse! I can only hope that they are working from a lossless or very HQ master when doing all this transcoding! It would seem so audio wise at least looking at the new 720p spectrum. Because tables are good (V1, V2, etc. different, anonymous videos, no more baring my soul): Stream => 720p A/V (2010/11) - 720p A/V (2014) ------- DASH-A (2014) -------- DASH-V 720p (2014) *outlier video with crazy aspect ratio that was probably not encoded properly letterbox back in the day. The newer bit rates are fine given black bars. Just to mess with one's mind, on lower quality videos sometimes the DASH is the best because I think they have some arbitrary bandwidth limit for audio on the premuxed ones. E.G. I have one where the best premuxed is 360 with awful audio, the DASH is the familiar 125 kb/s which now matches pretty well with the best from 2010/11 I have. Sigh. Bottom line is that if you want the best audio and video be prepared to manually inspect each one without an HD stream and for those with HD the best audio (and 720p video!) is probably in the premuxed 720p stream. |
|
Thank you WCVanHorne, that makes sense and I really appreciate your excellent observations. Now that I have ffmpeg running on my machine (so I can much the DASH audio and video streams together) I can start experimenting. By the way, I already have the Kesha video so I can use that as my starting point, for research purposes, of course. |
|
Say related question: What exact ffmpeg command does youtube-dl use when it muxes the DASH m4a and mp4 streams together? When I do it manually with the command I posted (m4a as audio infile) I get a file seeming identical stream wise but about 1k smaller. It was mostly because I am using "-shortest" (related: why does youtube-dl not use shortest?) but when I tried it without "-shortest" I then get a file 8 bytes bigger. Now I'm probably being OCD but it should be repeatable. :-) PS. In all the fooling around I've been doing I've not noticed any streams, DASH or otherwise, different by more than 40-90 ms, basically a frame or two depending on frame rate. This is good news. |
|
Speaking of max quality, I noticed that the max audio option in youtube-dl is 128k, but in this script https://github.com/gantt/downloadyoutube the max detected is 256k. Is youtube-dl unable to detect the higher bitrate? |
|
The command I use is: |
|
Does anyone know the ffmpeg command to convert .mp4 to mpeg5 (h.265)? Id't like to take all my Kesha videos and compress them down as much as possible to save disk space. Thanks. |
|
OK so I might just be missing the explanation of the "+" ffmpeg merging syntax in the man page but nevertheless it works as I hoped: video+audio and only grabs the audio of the second stream. Thus most of my problems are solved in one step for now (providing it's a pretty standard HD release) by: youtube-dl -f 137+22 https://www.youtube.com/........ Substitute 137 with bestvideo or whatever and 22 is only good for now since that is the stream with the best audio by current empirical examination. |
|
ThomS8312: Have you actually downloaded one of those audio streams and confirmed it is 256k? |
|
Yeah, I use that script all the time. I just downloaded from that kesha video and get reports of approximately 256k. |
|
A couple things, I tried the script addon in Firefox and only get: But glanced at the code and saw: '141':'M4A 256kbps - audio' which is sort of what I guessed and tried already. However trying to download stream 141 off our poor Ke$ha example or a few others with youtube-dl just yields "ERROR: requested format not available". _However_ I sometimes get "Downloading js player vflgoB_xN" so there may be something there that youtube-dl needs to be tweaked to recognize. |
|
Ah right, forgot to mention you have to edit the script. Search for "SHOW_DASH_FORMATS=false" and change it to true. Dash isn't enabled by default. |
|
Actually found it on my own just about when you posted. ;-) However it still isn't showing 141 and it's only showing one DASH stream audio and video? I turned off flv and set all m4a's while I was in there. |
|
Ah! Found a nasty little hack in the code that only shows it if m4a is set to max. Bad hack. However for the record there is indeed a stream 141 with 256 kb/s (253 reported) audio! Sweet. Some serious brick wall filtering at 18 kHz though, even compared to 192 kb/s VBR mp3. |
|
Nice. If that stream can be detected by youtube-dl, it will make the script obsolete for me. Muxing the streams using ffmpeg manually is getting tiresome. It would make life easier if the process were automated. I prefer youtube-dl to take care of all my youtube needs nowadays. I'm trying to move away from scripts if I can. |
|
Ditto. I'm just refreshing and adding some videos to a collection and I'd like to get it down to a couple tools. Spent some quality time with rtmpdump the other day. Devs need to replicate that downloadyoutube routine that seems to grab an alternate, "new format" of the manifest. Curiously it seems to only be this 141 stream that's hidden there so far. I wonder if more are to come? Maybe the start of splitting off the audio for a parallel "youradio" thing? Given some of lame video filler, and waste that is for a lot of basically audio only youtube content, it might be a good idea. |
|
Oh also this issue should probably be renamed: "Complete support for DASH audio and video" or perhaps a new issue: "Support for 141 DASH audio streams" should be opened. I'm not sure enough of the etiquette on this project to know which is preferred. |
|
Please try to keep the discussion to one thing at a time. It is nigh impossible to follow so many issues at once and also debug all the misconceptions that occur. youtube-dl already supports downloading the DASH manifest; just pass in |
|
Well that's _very_ interesting. I would say also rather confusing too since by default many of the DASH streams are already shown and the ones displayed seem rather arbitrary (although I would guess they are the ones on the default manifest, thus calling it a DASH manifest is a little misleading, more like a hidden DASH manifest). The flag is also not under the 'Video Format Options' section where one might argue it belongs being stream rather than title oriented. No matter, but IMHO that flag should just be the default and it would have avoided most of this conversation. BTW in no way take this as a complaint. Great code! I know keeping up with Google machinations is a nightmare and n-dimensional, dynamic soduku for the stubborn and slightly mad. :-) |
|
So a summary from my perspective: -The best 720p video and ~196 kb/s (18 kHz cutoff) audio is currently in stream 22, the premuxed one. Probably often still the one to go to for with a simple "youtube-dl -f 22 ....." . -If you like the 720p video and want still better audio swapped in "youtube-dl -f 141+22 --youtube-include-dash-manifest ....." is an option if stream 141 exists. Note that this leaves a .m4a file that is really the .mp4 and I had to reverse the normal audio and video order to get it to work. This goes back to my question on the syntax of the ffmpeg command for these merging operations. I think it needs to have some of the a & v stream flags switched or explicitly called out so it is always video + audio. It seems to be making some assumption and getting a little confused when the first stream has both audio and video tracks. I think I should open this as a separate issue. -If you want a la carte have a look with "youtube-dl -F --youtube-include-dash-manifest ......". -If you consider 1080p & 256 kb/s audio archival you want "youtube-dl -f 137+141 --youtube-include-dash-manifest .....". Again if 141 doesn't exist "youtube-dl -f 137+22 ....." is your fallback. Of course you can sub in high resolution video streams for 137. -The 'bestvideo' and 'bestaudio' algorithms cannot always be trusted since usually the best 720p mp4 video stream and ~196 kb/s m4a audio stream are in 22, not available as DASH. In other words for our Ke$ha example the best is actually "bestaudio+22" (see previous note why reversed) not "bestvideo+bestaudio". Similarly if there is no stream 141 the best might actually be "bestvideo+22". -At lower maximum stream resolutions it's a bit of a crapshoot, poking around is probably required if you are OCD to make sure you get the best. PS. I've seen some interesting streams now such as 3D. PPS. The bitrates that Google reports for those webm streams seem to be bogus unless I have just caught some bad ones. PPPS. I wish they would also be honest with the bandwidth of the m4a streams. E.G. the ~48 kb/s 22050 Hz (presumably mono) one is really 44100 Hz stereo. On a related note even that 141 best audio stream @ ~256 kb/s is still wickedly band limited at 18kHz. Only younger ears and a good audio system would probably make this noticable but hard filtering like that usually means occasional artifacts (might be default aac thing IIRC). PPPPS. Let's not even get started on how there are some upsampled videos where the "720p" stream looks like junk compared to the clean 480p or whatever original. |
|
Thank you WCVanHorne, your information is exactly what I was looking for and is working perfectly for me, I appreciate your effort. I've worked around very loud power generation equipment (designing their control systems) that I've lost sensitivity to 18kHz and above. In fact I've tried several of the hearing test videos on youtube and have severe rolloff starting sharply at 12500 Hz. Have you tested these videos and do they actually produce any usable audio above 12 kHz? If so, do they deliver a flat response? |
|
@WCVanHorne The reason why If you can provide an example video URL - or better, command line - and describe how |
|
Well without it several streams are hidden, exactly of what a large part of this discussion was about. Just to take an example from a top current video: https://www.youtube.com/watch?v=XqLTe8h0-jo -F without --youtube-include-dash-manifest 171 webm audio only DASH audio , audio@128k (worst) -F with --youtube-include-dash-manifest 139 m4a audio only DASH audio 48k , audio@ 48k (22050Hz), 996.98KiB (worst) That's a total of 5 more streams, including the elusive, and otherwise inaccessible, 141 for ~256 kb/s audio. Without that stream visible the "bestaudio" shortcut was only returning 140 with ~128 kb/s audio (and the nasty 16 kHz cut-off). In other words: "youtube-dl -f bestvideo+bestaudio http:..............." gets you a 1080p video with ~128 kb/s mediocre audio. but "youtube-dl -f bestvideo+bestaudio --youtube-include-dash-manifest http:..............." gets you a 1080p video with ~256 kb/s high quality audio. This seems to fit the criteria of getting more information and improving download quality to T. PS. Oh and you get file sizes too. |
|
^404: Not found. |
and others
Support for best or default quality only downloads 720p resolution on YouTube when 1080p and 4k video is actual available. YouTube for some time has supported video in 1080p as well as 4k (1440p and 2160p) resolutions.
Since not everyone will have the need or desire to download 1080p, 1440, or 2160p resolutions, I suggest adding support and invoking it simply such as: -f 1080 or -f 1440 or -f 2160
Even attempting the download at mx quality results in a 720p video being delivered
Bobs-MacBook-Pro-7:Youtube boblopez$ sudo youtube-dl https://www.youtube.com/watch?v=iNJdPyoqt8U --max-quality FORMAT
[youtube] Setting language
[youtube] Confirming age
[youtube] iNJdPyoqt8U: Downloading webpage
[youtube] iNJdPyoqt8U: Downloading video info webpage
[youtube] iNJdPyoqt8U: Extracting video information
[download] Destination: COSTA RICA IN 4K 60fps (ULTRA HD) w_ Freefly Movi-iNJdPyoqt8U.mp4
[download] 100% of 93.40MiB in 00:35
Bobs-MacBook-Pro-7:Youtube boblopez$
Thank you.
Screenshot sample is from:
https://www.youtube.com/watch?v=iNJdPyoqt8U