Youtube serves different itags (was: youtube-dl 2013.10.07 does not always pick highest resolution)

[himadri0327]

Was downloading videos from youtube user DiapasonRecords and saw that it wasn't downloading the max resolution for few videos randomly.

[jaimeMF]

I think that the problem is that sometimes we don't receive all the available itags. For example in https://travis-ci.org/rg3/youtube-dl/jobs/12288711#L376 we don't get the best format (flv format and itag 34), and we have to fallback to the next format (mp4 format and itag 18). If this is a problem on Youtube's side, it will be probably impossible to solve.

[seojoohyun]

I've been having problems with it getting some of the tags too, I use a script that downloads tags 22 and 37, it does 22 okay and then won't get 37 at first, just have to keep retrying until it picks it up.

I've also noticed that on some videos Youtube player doesn't show all of the formats, when I click the quality selection on the Youtube player it might show only 240p, but then I can still download it at 360p with tags 34 or 43.

Quite possibly on youtube's side. Maybe an option to retry a specified number of times before failing?

[up4]

Just so you know, when I want highest quality, I download best DASH for both audio and video and recombine with FFMPEG. It's a cumbersome 3-step process, but it does work.

[himadri0327]

@up4 Thanks, but your comment is not related to this random issue. Also look at my request related to DASH #1643

[up4]

But does this random issue mean that the script mentioned on #1643 is not reliable?

[himadri0327]

@up4 Yes, looks like. The random problem on this thread was probably on youtube's end as per comments above but it's been a while that I encountered one. I know my showcase cmd-script in #1643 is far from reliable for other reasons too. It was only to showcase that request/suggestion.

Also these days, as you may also notice, most cases highest resolution is on dash format but time-to-time I do see videos that have highest resolution on a non-dash old school format. I myself evolved to a script that adds selecting highest-resolution (dash or non-dash, with a preference toward dash when equal) and error-handling loops to cover when inbuilt retry param somehow wasn't adequate in rare freak situation that I had encountered which is much needed for downloading a long list of videos more reliably.

[up4]

So, do you think it's a safe bet to assume that Youtube is going to gradually stabilize a DASH implementation for all higher-res? I have heard they are pushing very hard for 4K/H.265/DASH.

That would mean the FFMPEG-remuxing technique should get implemented from within youtube-dl so as to output a single file with both audio and video streams.

That is of course, if you see (as I do) the tendency towards the stabilization of the DASH delivery for Youtube & co.

What do you think?

[jaimeMF]

@up4 probably someday it will be automated in youtube-dl (see #1612), but someone has to do it (contributions are welcomed) ;)

[up4]

@jaimeMF I hereby pledge that if I am sufficiently annoyed by the extra work required for remuxing that I would write a patch, I will contribute it here. 👯

[jmcbee]

$ youtube-dl -F https://www.youtube.com/watch\?v\=iVAgTiBrrDA                                                                   

[youtube] iVAgTiBrrDA: Downloading webpage
[youtube] iVAgTiBrrDA: Extracting video information
[youtube] iVAgTiBrrDA: Downloading DASH manifest
[info] Available formats for iVAgTiBrrDA:
format code extension resolution  note 
249         webm      audio only  DASH audio , opus @ 50k (worst)
250         webm      audio only  DASH audio , opus @ 70k
251         webm      audio only  DASH audio , opus @160k
139         m4a       audio only  DASH audio   49k , audio@ 48k (22050Hz), 906.21KiB
171         webm      audio only  DASH audio  120k , audio@128k (44100Hz), 1.93MiB
140         m4a       audio only  DASH audio  129k , audio@128k (44100Hz), 2.35MiB
172         webm      audio only  DASH audio  167k , audio@256k (44100Hz), 2.76MiB
141         m4a       audio only  DASH audio  255k , audio@256k (44100Hz), 4.67MiB
278         webm      256x144     DASH video  118k , webm container, VP9, 12fps, video only, 1.53MiB
160         mp4       256x144     DASH video  109k , 12fps, video only, 1.95MiB
242         webm      426x240     DASH video  203k , 24fps, video only, 2.57MiB
133         mp4       426x240     DASH video  253k , 24fps, video only, 4.37MiB
243         webm      640x360     DASH video  354k , 24fps, video only, 4.72MiB
134         mp4       640x360     DASH video  638k , 24fps, video only, 6.13MiB
244         webm      854x480     DASH video  783k , 24fps, video only, 9.29MiB
135         mp4       854x480     DASH video 1117k , 24fps, video only, 12.36MiB
247         webm      1280x720    DASH video 1515k , 24fps, video only, 18.21MiB
136         mp4       1280x720    DASH video 2316k , 24fps, video only, 23.55MiB
248         webm      1920x1080   DASH video 2449k , 24fps, video only, 31.47MiB
137         mp4       1920x1080   DASH video 4126k , 24fps, video only, 45.42MiB
17          3gp       176x144     
36          3gp       320x240     
5           flv       400x240     
43          webm      640x360     
18          mp4       640x360     
22          mp4       1280x720    (best)

The best available is 1080p, but video only??

[jaimeMF]

@fbm-static Yes, for some time Youtube only provides the best quality formats splitted in two files. youtube-dl can download and merged them into a single file, you just need to run youtube-dl URL -f bestvideo+bestaudio (it requires ffmpeg or avconv).

[RickyRomero]

Interesting, I didn't know about the bestvideo/bestaudio format.

Here's a way you could resolve this bug, when no format is specified:

If ffmpeg is not installed
Automatically download the highest resolution multiplexed file, with a simple info message stating that ffmpeg is required for the highest quality if there's a higher resolution HTTP streaming version available.

If ffmpeg is installed
Automatically download the bestvideo+bestaudio files and mux them using ffmpeg.

[strelec]

I agree completely.

It took me tens of minutes to figure out why youtube-dl is not downloading the best quality.

Q: Is any quality lost with ffmpeg A+V merging? That shouldn't be the case, but asking nevertheless.

[RickyRomero]

@strelec: No re-encoding is performed. ffmpeg simply merges the raw audio and video streams into the final file.

[RickyRomero]

I think you guys can close this one now. youtube-dl is picking the best resolution by default and muxing to mkv.

Any objections?

[up4]

I'm fine with the infinitely long format string ;-)

Ok, seriously, you can close it. Thanks.