Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some videos play using a format on Youtube that youtube-dl -F does not detect as being available #16976

Closed
Tynach opened this issue Jul 15, 2018 · 3 comments

Comments

@Tynach
Copy link

@Tynach Tynach commented Jul 15, 2018

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2018.07.10. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

  • I've verified and I assure that I'm running youtube-dl 2018.07.10

Before submitting an issue make sure you have:

  • At least skimmed through the README, most notably the FAQ and BUGS sections
  • Searched the bugtracker for similar issues including closed ones
  • Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser

What is the purpose of your issue?

  • Bug report (encountered problems with youtube-dl)
  • Site support request (request for adding support for a new site)
  • Feature request (request for a new functionality)
  • Question
  • Other

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:

Add the -v flag to your command line you run youtube-dl with (youtube-dl -v <your command line>), copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'https://www.youtube.com/watch?v=0ViqVT9mBGU', u'-F', u'-v']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2018.07.10
[debug] Python version 2.7.12 (CPython) - Linux-4.15.0-24-generic-x86_64-with-neon-16.04-xenial
[debug] exe versions: ffmpeg N-91399-g9f0077c, ffprobe N-91399-g9f0077c, rtmpdump 2.4
[debug] Proxy map: {}
[youtube] 0ViqVT9mBGU: Downloading webpage
[youtube] 0ViqVT9mBGU: Downloading video info webpage
[youtube] 0ViqVT9mBGU: Downloading MPD manifest
[youtube] 0ViqVT9mBGU: Downloading MPD manifest
[info] Available formats for 0ViqVT9mBGU:
format code  extension  resolution note
139          m4a        audio only DASH audio   57k , m4a_dash container, mp4a.40.5@ 48k (22050Hz), 11.31MiB
140          m4a        audio only DASH audio  136k , m4a_dash container, mp4a.40.2@128k (44100Hz), 30.15MiB
160          mp4        256x144    DASH video  108k , mp4_dash container, avc1.4d400b, 30fps, video only
133          mp4        426x240    DASH video  242k , mp4_dash container, avc1.4d400c, 30fps, video only
134          mp4        640x360    DASH video  633k , mp4_dash container, avc1.4d401e, 30fps, video only, 25.61MiB
135          mp4        854x480    DASH video 1155k , mp4_dash container, avc1.4d4014, 30fps, video only
136          mp4        1280x720   DASH video 2310k , mp4_dash container, avc1.4d4016, 30fps, video only
298          mp4        1280x720   DASH video 3465k , mp4_dash container, avc1.4d4016, 60fps, video only
137          mp4        1920x1080  DASH video 4331k , mp4_dash container, avc1.64001e, 30fps, video only
299          mp4        1920x1080  DASH video 6794k , mp4_dash container, avc1.64002a, 60fps, video only, 346.63MiB
17           3gp        176x144    small , mp4v.20.3, mp4a.40.2@ 24k, 12.44MiB
36           3gp        320x180    small , mp4v.20.3, mp4a.40.2, 35.15MiB
18           mp4        640x360    medium , avc1.42001E, mp4a.40.2@ 96k, 75.06MiB
43           webm       640x360    medium , vp8.0, vorbis@128k, 79.26MiB
22           mp4        1280x720   hd720 , avc1.64001F, mp4a.40.2@192k (best)

Description of your issue, suggested solution and other information

I uploaded a 1080p, 60fps video to Youtube, but have noticed an odd colorspace issue with it. To investigate, I figured I'd use youtube-dl to download the files that Youtube encoded and see if some metadata was incorrectly set.

However, youtube-dl is unable to detect that the format that Youtube is using is even available. Youtube's website, when I right-click the video player and click 'Stats for nerds', lists these codecs as being used: vp9 (303) / mp4a.40.2 (140). However, the only 1080p, 60fps source available through youtube-dl is, as can be seen in the log above, AVC (H.264) with a numeric 'format code' of 299.

Curious if this was merely youtube-dl not being able to see that specific format code (303), I tried it with a different Youtube video that I knew had 1080p 60fps content. Turns out that youtube-dl detected it in VP9/webm format just fine:

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'https://www.youtube.com/watch?v=eR8HUy-oRk8', u'-F', u'-v']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2018.07.10
[debug] Python version 2.7.12 (CPython) - Linux-4.15.0-24-generic-x86_64-with-neon-16.04-xenial
[debug] exe versions: ffmpeg N-91399-g9f0077c, ffprobe N-91399-g9f0077c, rtmpdump 2.4
[debug] Proxy map: {}
[youtube] eR8HUy-oRk8: Downloading webpage
[youtube] eR8HUy-oRk8: Downloading video info webpage
[info] Available formats for eR8HUy-oRk8:
format code  extension  resolution note
249          webm       audio only DASH audio   65k , opus @ 50k, 2.25MiB
250          webm       audio only DASH audio   83k , opus @ 70k, 2.96MiB
140          m4a        audio only DASH audio  128k , m4a_dash container, mp4a.40.2@128k, 5.43MiB
171          webm       audio only DASH audio  129k , vorbis@128k, 5.02MiB
251          webm       audio only DASH audio  155k , opus @160k, 5.79MiB
278          webm       256x144    144p   98k , webm container, vp9, 30fps, video only, 3.75MiB
160          mp4        256x144    144p  115k , avc1.4d400c, 30fps, video only, 3.62MiB
242          webm       426x240    240p  230k , vp9, 30fps, video only, 8.02MiB
133          mp4        426x240    240p  255k , avc1.4d4015, 30fps, video only, 7.26MiB
243          webm       640x360    360p  432k , vp9, 30fps, video only, 14.68MiB
134          mp4        640x360    360p  645k , avc1.4d401e, 30fps, video only, 18.18MiB
244          webm       854x480    480p  779k , vp9, 30fps, video only, 26.01MiB
135          mp4        854x480    480p 1204k , avc1.4d401f, 30fps, video only, 34.49MiB
247          webm       1280x720   720p 1570k , vp9, 30fps, video only, 52.32MiB
136          mp4        1280x720   720p 2411k , avc1.4d401f, 30fps, video only, 64.83MiB
302          webm       1280x720   720p60 2683k , vp9, 60fps, video only, 90.04MiB
248          webm       1920x1080  1080p 2783k , vp9, 30fps, video only, 94.21MiB
298          mp4        1280x720   720p60 3496k , avc1.4d4020, 60fps, video only, 105.11MiB
303          webm       1920x1080  1080p60 4421k , vp9, 60fps, video only, 157.78MiB
137          mp4        1920x1080  1080p 4540k , avc1.640028, 30fps, video only, 120.48MiB
299          mp4        1920x1080  1080p60 5922k , avc1.64002a, 60fps, video only, 192.44MiB
17           3gp        176x144    small , mp4v.20.3, mp4a.40.2@ 24k, 3.53MiB
36           3gp        320x180    small , mp4v.20.3, mp4a.40.2, 9.81MiB
18           mp4        640x360    medium , avc1.42001E, mp4a.40.2@ 96k, 28.93MiB
43           webm       640x360    medium , vp8.0, vorbis@128k, 34.03MiB
22           mp4        1280x720   hd720 , avc1.64001F, mp4a.40.2@192k (best)

So I know that youtube-dl is at least sometimes detecting the presence of this format. However, I have also noticed that for this video where the detection succeeds, there is a webm audio format for DASH present as well. For the video I uploaded, only an mp4-based DASH audio source (format code 140) is present.


While typing all this up, I investigated another avenue, and fear it might render this bug report pointless (still going to submit it, in case there is something that youtube-dl should be doing but the bug wouldn't be easily detected normally).

When I use incognito mode, the Youtube website gives me only the mp4-based version of the video, which ironically does not have the colorspace metadata issue that the webm version appears to have. So it seems if I'm logged in, I see a VP9-encoded file with incorrect metadata, and when I'm logged out I see an AVC-encoded file with correct metadata.

And, it also seems I'm only given AVC/MP4 when I use Youtube from another account. Weird.

For anyone wondering about the colorspace issue, the video is being decoded using the BT. 709 YUV→RGB color matrix, when I explicitly put in the metadata of the uploaded video that it should be decoded with the BT. 470BG (Rec. 601) YUV→RGB color matrix.

The mp4 file that youtube-dl downloads includes the correct metadata, though it does erroneously say to use the 'tv' color range (when it's actually encoded with 'full' color range... And somehow web browsers detect it and ignore the metadata, playing the file correctly despite it).

@Hrxn
Copy link

@Hrxn Hrxn commented Jul 15, 2018

uploaded a 1080p, 60fps video to Youtube, but have noticed an odd colorspace issue with it. To investigate, I figured I'd use youtube-dl to download the files that Youtube encoded and see if some metadata was incorrectly set.

In effect, you noticed a missing format on a video you have uploaded yourself just recently?
Could be on YouTube's side, give it some time and then test the exact same video again, the missing format may appear now.

@dstftw
Copy link
Collaborator

@dstftw dstftw commented Jul 15, 2018

You've already answered your question: for some reason Youtube does not serve webm video only formats for this particular video (except for your account). Nothing to do with youtube-dl.

@dstftw dstftw closed this Jul 15, 2018
@Tynach
Copy link
Author

@Tynach Tynach commented Jul 16, 2018

Yeah, I did answer my own question.. Though to be fair I typed most of my report up first, then went digging deeper just to be sure, and figured hey - may as well still submit the report.

The time between me typing up the majority of it, and actually submitting it, was about 8 hours or so.

Would still be nice to have a way to download the vp9 version that's there when I'm logged in, but that's a completely different thing from this (definitely invalid) bug report.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.