[Youtube] Confusing differences in bitrate

[cog-videogames]

• I've verified and I assure that I'm running youtube-dl 2017.12.31
• At least skimmed through the README, most notably the FAQ and BUGS sections
• Searched the bugtracker for similar issues including closed ones

Purpose

• Bug report (encountered problems with youtube-dl)

Verbose output:

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', '--ignore-config', '-o', 'EXAMPLE.%(ext)s', '--sleep-interval', '15', '--max-sleep-interval', '60', 'https://www.youtube.com/watch?v=H84po-1-GB0']
[debug] Encodings: locale cp1252, fs mbcs, out cp1252, pref cp1252
[debug] youtube-dl version 2017.12.31
[debug] Python version 3.4.4 - Windows-7-6.1.7601-SP1
[debug] exe versions: ffmpeg N-83280-gcba4f0e, ffprobe N-83280-gcba4f0e
[debug] Proxy map: {}

I've also put a full run in a gist.

Example URL:

• Single video: https://www.youtube.com/watch?v=H84po-1-GB0

Description:

TLDR: the newest version of youtube-dl seems to acquire a lower bitrate copies of some videos, and I cannot figure out why or how to resolve the issue. This report focuses on a single problem video.

---

Some time mid 2017, I downloaded the above video with an older version of youtube-dl. At that time, I was using -f 'bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]/best' --merge-output-format mp4. Since then I have since learned that the format string is not necessary.

The bitrate of this old download is good, 1876 kb/s:

$ ffprobe 'old.mp4'
...
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'old.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.65.100
  Duration: 04:30:37.72, start: 0.000000, bitrate: 1876 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 1742 kb/s, 30 fps, 30 tbr, 90k tbn, 60 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default)
    Metadata:
      handler_name    : SoundHandler

If I download the same video, with the latest youtube-dl, and the same long format string, the result has noticeably worse visual quality. I checked, and the bitrate has been reduced to less than half, 830 kb/s:

$ ffprobe 'new.mp4'
...
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'new.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.65.100
  Duration: 04:30:37.72, start: 0.000000, bitrate: 830 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 696 kb/s, 30 fps, 30 tbr, 90k tbn, 60 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default)
    Metadata:
      handler_name    : SoundHandler

Finally, if I omit the format flag, and let youtube-dl auto-detect as shown at the top of the ticket, it chooses an MKV container, and I notice that there is dash in the metadata, but the bitrate is still low, 803 kb/s:

$ ffprobe 'EXAMPLE.mkv'
...
Input #0, matroska,webm, from 'EXAMPLE.mkv':
  Metadata:
    COMPATIBLE_BRANDS: iso6avc1mp41
    MAJOR_BRAND     : dash
    MINOR_VERSION   : 0
    ENCODER         : Lavf57.65.100
  Duration: 04:30:37.68, start: -0.007000, bitrate: 803 kb/s
    Stream #0:0: Video: h264 (High), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 1k tbn, 60 tbc (default)
    Metadata:
      HANDLER_NAME    : VideoHandler
      DURATION        : 04:30:37.666000000
    Stream #0:1(eng): Audio: opus, 48000 Hz, stereo, fltp (default)
    Metadata:
      DURATION        : 04:30:37.681000000

As far as youtube-dl can tell, it is choosing good formats:

$ youtube-dl --list-formats 'https://www.youtube.com/watch?v=H84po-1-GB0'
[youtube] H84po-1-GB0: Downloading webpage
[youtube] H84po-1-GB0: Downloading video info webpage
[youtube] H84po-1-GB0: Extracting video information
[youtube] H84po-1-GB0: Downloading MPD manifest
[info] Available formats for H84po-1-GB0:
format code  extension  resolution note
139          m4a        audio only DASH audio   63k , m4a_dash container, mp4a.40.5@ 48k (22050Hz), 92.09MiB
249          webm       audio only DASH audio   74k , opus @ 50k, 91.56MiB
250          webm       audio only DASH audio   88k , opus @ 70k, 111.63MiB
140          m4a        audio only DASH audio  143k , m4a_dash container, mp4a.40.2@128k (44100Hz), 245.95MiB
171          webm       audio only DASH audio  151k , vorbis@128k, 184.54MiB
251          webm       audio only DASH audio  155k , opus @160k, 204.96MiB
160          mp4        256x144    DASH video  112k , mp4_dash container, avc1.4d400c, 30fps, video only, 65.76MiB
278          webm       256x144    144p  173k , webm container, vp9, 30fps, video only, 167.37MiB
133          mp4        426x240    DASH video  248k , mp4_dash container, avc1.4d4015, 30fps, video only, 110.22MiB
242          webm       426x240    240p  256k , vp9, 30fps, video only, 245.49MiB
243          webm       640x360    360p  484k , vp9, 30fps, video only, 449.27MiB
134          mp4        640x360    DASH video  643k , mp4_dash container, avc1.4d401e, 30fps, video only, 251.31MiB
244          webm       854x480    480p  887k , vp9, 30fps, video only, 693.99MiB
135          mp4        854x480    DASH video 1187k , mp4_dash container, avc1.4d401f, 30fps, video only, 466.99MiB
247          webm       1280x720   720p 1850k , vp9, 30fps, video only, 1.32GiB
136          mp4        1280x720   DASH video 2365k , mp4_dash container, avc1.4d401f, 30fps, video only, 815.38MiB
248          webm       1920x1080  1080p 3136k , vp9, 30fps, video only, 2.82GiB
137          mp4        1920x1080  DASH video 4360k , mp4_dash container, avc1.640028, 30fps, video only, 1.32GiB
17           3gp        176x144    small , mp4v.20.3, mp4a.40.2@ 24k
36           3gp        320x180    small , mp4v.20.3, mp4a.40.2
43           webm       640x360    medium , vp8.0, vorbis@128k
18           mp4        640x360    medium , avc1.42001E, mp4a.40.2@ 96k
22           mp4        1280x720   hd720 , avc1.64001F, mp4a.40.2@192k (best)

So, what's the deal? My initial ideas:

1. Is youtube silently serving me a worse version due to my location, ISP, or similar? I live in mainland US with a ~120 mb connection, but maybe the situation would be solved by turning on a VPN?
2. Has youtube silently degraded this video, and a high-quality version is no longer available?
3. Is the newer youtube-dl missing a format option, or not negotiating the best bitrate?

What are your thoughts?

[yan12125]

Format 137 for this video is broken at YouTube server side. You can download webm for better quality. (e.g. -f 'bestvideo[ext=webm]+bestaudio[ext=webm]')

[cog-videogames]

Thank you, that sounds like my number 2 scenario then. Two quick follow-ups:

1. What is a good way to tell when the "best" format is broken? I've observed this behavior in several files out of a playlist of ~150 videos. Can you suggest any automated way to detect which videos are busted?
2. In the non-broken case, generally on youtube, the best webm will be lower quality than the best mp4 quality, right? So I should probably only use your suggested format string on broken videos?

[Hrxn]

In the non-broken case, generally on youtube, the best webm will be lower quality than the best mp4 quality, right?

Eh.. no. Not as a general rule. Actually I think that webm will be better in many cases, definitely in the near future etc.