Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add YouTube client for 1080p muxed videos #9546

Merged
merged 6 commits into from
May 12, 2024

Conversation

clienthax
Copy link
Contributor

@clienthax clienthax commented Mar 28, 2024

Description of your pull request and other information

Adds the ability for yt-dlp to return premuxed 1080p results for videos when available.

image
Template

Before submitting a pull request make sure you have:

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check all of the following options that apply:

  • I am the original author of this code and I am willing to release it under Unlicense
  • I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

@pukkandan pukkandan added the site-enhancement Feature request for some website label Mar 28, 2024
@pukkandan
Copy link
Member

pukkandan commented Mar 28, 2024

I am opposed to adding this to the default list. Opinions? @bashonly @coletdjnz

Also, make sure to document in readme

@bashonly

This comment was marked as outdated.

@bashonly
Copy link
Member

bashonly commented Mar 28, 2024

Before I got blocked by youtube (?) I was going to write that I think adding another default request to manifest.googlevideo.com is probably unwise and will get people rate-limited

@clienthax
Copy link
Contributor Author

clienthax commented Mar 28, 2024

I'm not too concerned about it being on by default or not, i just added it there while testing with vrchat to check the returned streams were compatible with avpro easily, as long as it can be enabled with a command line option I'm fine with it not being on by default.

I'm also not too sure what the difference between base_clients vs defaults is as I'm unfamiliar with the codebase.

@rredford6

This comment was marked as outdated.

@bashonly

This comment was marked as outdated.

@rredford6

This comment was marked as outdated.

@bashonly

This comment was marked as resolved.

@pukkandan
Copy link
Member

"Wrong player response" issue has been confirmed to not be due to this PR and should be tracked separately

@coletdjnz coletdjnz self-assigned this Mar 28, 2024
@coletdjnz
Copy link
Member

coletdjnz commented Mar 28, 2024

You can also get the 1080p muxed videos by using a safari user agent with the web client, so could also add a duplicate of web client with the safari user agent.

I'm guessing media_connect_frontend is android - the android clients are starting to require droidguard, making them unusable for yt-dlp. We will probably be removing them soon. nvm, it seems fine. Worth adding anyways as a backup.

yt_dlp/extractor/youtube.py Outdated Show resolved Hide resolved
clienthax and others added 2 commits March 28, 2024 23:09
Co-authored-by: coletdjnz <coletdjnz@protonmail.com>
Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
@clienthax
Copy link
Contributor Author

You can also get the 1080p muxed videos by using a safari user agent with the web client, so could also add a duplicate of web client with the safari user agent.

I'm guessing media_connect_frontend is android - the android clients are starting to require droidguard, making them unusable for yt-dlp. We will probably be removing them soon. nvm, it seems fine. Worth adding anyways as a backup.

Interesting, is it the same streams as this returns?

@coletdjnz
Copy link
Member

web client only, using safari UA:

[info] Available formats for i1csLh-0L9E:
ID  EXT   RESOLUTION FPS CH │   FILESIZE    TBR PROTO │ VCODEC           VBR ACODEC      ABR ASR MORE INFO
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
sb3 mhtml 48x27        1    │                   mhtml │ images                                   storyboard
sb2 mhtml 80x45        1    │                   mhtml │ images                                   storyboard
sb1 mhtml 160x90       1    │                   mhtml │ images                                   storyboard
sb0 mhtml 320x180      1    │                   mhtml │ images                                   storyboard
249 webm  audio only      2 │  965.01KiB    47k https │ audio only           opus        47k 48k [en] low, webm_dash
250 webm  audio only      2 │    1.25MiB    62k https │ audio only           opus        62k 48k [en] low, webm_dash
140 m4a   audio only      2 │    2.53MiB   126k https │ audio only           mp4a.40.2  126k 44k [en] medium, m4a_dash
251 webm  audio only      2 │    2.49MiB   125k https │ audio only           opus       125k 48k [en] medium, webm_dash
394 mp4   256x144     24    │    1.38MiB    69k https │ av01.0.00M.08    69k video only          144p, mp4_dash
160 mp4   256x144     24    │    1.96MiB    98k https │ avc1.4d400c      98k video only          144p, mp4_dash
91  mp4   256x144     24    │ ~  3.40MiB   170k m3u8  │ avc1.4D400C          mp4a.40.5           [en]
278 webm  256x144     24    │    1.62MiB    81k https │ vp9              81k video only          144p, webm_dash
395 mp4   426x240     24    │    3.18MiB   159k https │ av01.0.00M.08   159k video only          240p, mp4_dash
133 mp4   426x240     24    │    4.36MiB   219k https │ avc1.4d4015     219k video only          240p, mp4_dash
92  mp4   426x240     24    │ ~  6.22MiB   310k m3u8  │ avc1.4D4015          mp4a.40.5           [en]
242 webm  426x240     24    │    3.66MiB   183k https │ vp9             183k video only          240p, webm_dash
396 mp4   640x360     24    │    6.57MiB   329k https │ av01.0.01M.08   329k video only          360p, mp4_dash
134 mp4   640x360     24    │   10.65MiB   533k https │ avc1.4d401e     533k video only          360p, mp4_dash
93  mp4   640x360     24    │ ~ 16.06MiB   802k m3u8  │ avc1.4D401E          mp4a.40.2           [en]
18  mp4   640x360     24  2 │   14.64MiB   733k https │ avc1.42001E          mp4a.40.2       44k [en] 360p
243 webm  640x360     24    │    7.79MiB   390k https │ vp9             390k video only          360p, webm_dash
397 mp4   854x480     24    │   11.19MiB   561k https │ av01.0.04M.08   561k video only          480p, mp4_dash
135 mp4   854x480     24    │   18.54MiB   929k https │ avc1.4d401e     929k video only          480p, mp4_dash
94  mp4   854x480     24    │ ~ 27.09MiB  1353k m3u8  │ avc1.4D401E          mp4a.40.2           [en]
244 webm  854x480     24    │   13.83MiB   693k https │ vp9             693k video only          480p, webm_dash
398 mp4   1280x720    24    │   20.96MiB  1050k https │ av01.0.05M.08  1050k video only          720p, mp4_dash
136 mp4   1280x720    24    │   32.72MiB  1639k https │ avc1.4d401f    1639k video only          720p, mp4_dash
95  mp4   1280x720    24    │ ~ 51.47MiB  2571k m3u8  │ avc1.4D401F          mp4a.40.2           [en]
247 webm  1280x720    24    │   28.49MiB  1427k https │ vp9            1427k video only          720p, webm_dash
399 mp4   1920x1080   24    │   43.37MiB  2173k https │ av01.0.08M.08  2173k video only          1080p, mp4_dash
137 mp4   1920x1080   24    │   76.01MiB  3807k https │ avc1.640028    3807k video only          1080p, mp4_dash
96  mp4   1920x1080   24    │ ~ 94.57MiB  4724k m3u8  │ avc1.640028          mp4a.40.2           [en]
248 webm  1920x1080   24    │   49.01MiB  2455k https │ vp9            2455k video only          1080p, webm_dash
400 mp4   2560x1440   24    │  109.40MiB  5480k https │ av01.0.12M.08  5480k video only          1440p, mp4_dash
271 webm  2560x1440   24    │  149.59MiB  7493k https │ vp9            7493k video only          1440p, webm_dash
401 mp4   3840x2160   24    │  214.80MiB 10760k https │ av01.0.12M.08 10760k video only          2160p, mp4_dash
313 webm  3840x2160   24    │  304.37MiB 15246k https │ vp9           15246k video only          2160p, webm_dash

media connect frontend only:

[info] Available formats for i1csLh-0L9E:
ID  EXT   RESOLUTION FPS CH │  FILESIZE   TBR PROTO │ VCODEC        VBR ACODEC      ABR ASR MORE INFO
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────
sb3 mhtml 48x27        1    │                 mhtml │ images                                storyboard
sb2 mhtml 80x45        1    │                 mhtml │ images                                storyboard
sb1 mhtml 160x90       1    │                 mhtml │ images                                storyboard
sb0 mhtml 320x180      1    │                 mhtml │ images                                storyboard
140 m4a   audio only      2 │   2.53MiB  126k https │ audio only        mp4a.40.2  126k 44k [en] medium, m4a_dash
160 mp4   256x144     24    │   1.96MiB   98k https │ avc1.4d400c   98k video only          144p, mp4_dash
91  mp4   256x144     24    │ ~ 3.40MiB  170k m3u8  │ avc1.4d400c       mp4a.40.5           [en]
133 mp4   426x240     24    │   4.36MiB  219k https │ avc1.4d4015  219k video only          240p, mp4_dash
92  mp4   426x240     24    │ ~ 6.22MiB  310k m3u8  │ avc1.4d4015       mp4a.40.5           [en]
134 mp4   640x360     24    │  10.65MiB  533k https │ avc1.4d401e  533k video only          360p, mp4_dash
93  mp4   640x360     24    │ ~16.06MiB  802k m3u8  │ avc1.4d401e       mp4a.40.2           [en]
18  mp4   640x360     24  2 │  14.64MiB  733k https │ avc1.42001E       mp4a.40.2       44k [en] 360p
135 mp4   854x480     24    │  18.54MiB  929k https │ avc1.4d401e  929k video only          480p, mp4_dash
94  mp4   854x480     24    │ ~27.09MiB 1353k m3u8  │ avc1.4d401e       mp4a.40.2           [en]
22  mp4   1280x720    24  2 │ ≈35.33MiB 1765k https │ avc1.64001F       mp4a.40.2       44k [en] 720p
136 mp4   1280x720    24    │  32.72MiB 1639k https │ avc1.4d401f 1639k video only          720p, mp4_dash
95  mp4   1280x720    24    │ ~51.47MiB 2571k m3u8  │ avc1.4d401f       mp4a.40.2           [en]
137 mp4   1920x1080   24    │  76.01MiB 3807k https │ avc1.640028 3807k video only          1080p, mp4_dash
96  mp4   1920x1080   24    │ ~94.57MiB 4724k m3u8  │ avc1.640028       mp4a.40.2           [en]

(no need to add safari web client version atm though, can use --user-agent for that)

Copy link
Member

@coletdjnz coletdjnz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@clienthax
Copy link
Contributor Author

web client only, using safari UA:

[info] Available formats for i1csLh-0L9E:
ID  EXT   RESOLUTION FPS CH │   FILESIZE    TBR PROTO │ VCODEC           VBR ACODEC      ABR ASR MORE INFO
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
sb3 mhtml 48x27        1    │                   mhtml │ images                                   storyboard
sb2 mhtml 80x45        1    │                   mhtml │ images                                   storyboard
sb1 mhtml 160x90       1    │                   mhtml │ images                                   storyboard
sb0 mhtml 320x180      1    │                   mhtml │ images                                   storyboard
249 webm  audio only      2 │  965.01KiB    47k https │ audio only           opus        47k 48k [en] low, webm_dash
250 webm  audio only      2 │    1.25MiB    62k https │ audio only           opus        62k 48k [en] low, webm_dash
140 m4a   audio only      2 │    2.53MiB   126k https │ audio only           mp4a.40.2  126k 44k [en] medium, m4a_dash
251 webm  audio only      2 │    2.49MiB   125k https │ audio only           opus       125k 48k [en] medium, webm_dash
394 mp4   256x144     24    │    1.38MiB    69k https │ av01.0.00M.08    69k video only          144p, mp4_dash
160 mp4   256x144     24    │    1.96MiB    98k https │ avc1.4d400c      98k video only          144p, mp4_dash
91  mp4   256x144     24    │ ~  3.40MiB   170k m3u8  │ avc1.4D400C          mp4a.40.5           [en]
278 webm  256x144     24    │    1.62MiB    81k https │ vp9              81k video only          144p, webm_dash
395 mp4   426x240     24    │    3.18MiB   159k https │ av01.0.00M.08   159k video only          240p, mp4_dash
133 mp4   426x240     24    │    4.36MiB   219k https │ avc1.4d4015     219k video only          240p, mp4_dash
92  mp4   426x240     24    │ ~  6.22MiB   310k m3u8  │ avc1.4D4015          mp4a.40.5           [en]
242 webm  426x240     24    │    3.66MiB   183k https │ vp9             183k video only          240p, webm_dash
396 mp4   640x360     24    │    6.57MiB   329k https │ av01.0.01M.08   329k video only          360p, mp4_dash
134 mp4   640x360     24    │   10.65MiB   533k https │ avc1.4d401e     533k video only          360p, mp4_dash
93  mp4   640x360     24    │ ~ 16.06MiB   802k m3u8  │ avc1.4D401E          mp4a.40.2           [en]
18  mp4   640x360     24  2 │   14.64MiB   733k https │ avc1.42001E          mp4a.40.2       44k [en] 360p
243 webm  640x360     24    │    7.79MiB   390k https │ vp9             390k video only          360p, webm_dash
397 mp4   854x480     24    │   11.19MiB   561k https │ av01.0.04M.08   561k video only          480p, mp4_dash
135 mp4   854x480     24    │   18.54MiB   929k https │ avc1.4d401e     929k video only          480p, mp4_dash
94  mp4   854x480     24    │ ~ 27.09MiB  1353k m3u8  │ avc1.4D401E          mp4a.40.2           [en]
244 webm  854x480     24    │   13.83MiB   693k https │ vp9             693k video only          480p, webm_dash
398 mp4   1280x720    24    │   20.96MiB  1050k https │ av01.0.05M.08  1050k video only          720p, mp4_dash
136 mp4   1280x720    24    │   32.72MiB  1639k https │ avc1.4d401f    1639k video only          720p, mp4_dash
95  mp4   1280x720    24    │ ~ 51.47MiB  2571k m3u8  │ avc1.4D401F          mp4a.40.2           [en]
247 webm  1280x720    24    │   28.49MiB  1427k https │ vp9            1427k video only          720p, webm_dash
399 mp4   1920x1080   24    │   43.37MiB  2173k https │ av01.0.08M.08  2173k video only          1080p, mp4_dash
137 mp4   1920x1080   24    │   76.01MiB  3807k https │ avc1.640028    3807k video only          1080p, mp4_dash
96  mp4   1920x1080   24    │ ~ 94.57MiB  4724k m3u8  │ avc1.640028          mp4a.40.2           [en]
248 webm  1920x1080   24    │   49.01MiB  2455k https │ vp9            2455k video only          1080p, webm_dash
400 mp4   2560x1440   24    │  109.40MiB  5480k https │ av01.0.12M.08  5480k video only          1440p, mp4_dash
271 webm  2560x1440   24    │  149.59MiB  7493k https │ vp9            7493k video only          1440p, webm_dash
401 mp4   3840x2160   24    │  214.80MiB 10760k https │ av01.0.12M.08 10760k video only          2160p, mp4_dash
313 webm  3840x2160   24    │  304.37MiB 15246k https │ vp9           15246k video only          2160p, webm_dash

media connect frontend only:

[info] Available formats for i1csLh-0L9E:
ID  EXT   RESOLUTION FPS CH │  FILESIZE   TBR PROTO │ VCODEC        VBR ACODEC      ABR ASR MORE INFO
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────
sb3 mhtml 48x27        1    │                 mhtml │ images                                storyboard
sb2 mhtml 80x45        1    │                 mhtml │ images                                storyboard
sb1 mhtml 160x90       1    │                 mhtml │ images                                storyboard
sb0 mhtml 320x180      1    │                 mhtml │ images                                storyboard
140 m4a   audio only      2 │   2.53MiB  126k https │ audio only        mp4a.40.2  126k 44k [en] medium, m4a_dash
160 mp4   256x144     24    │   1.96MiB   98k https │ avc1.4d400c   98k video only          144p, mp4_dash
91  mp4   256x144     24    │ ~ 3.40MiB  170k m3u8  │ avc1.4d400c       mp4a.40.5           [en]
133 mp4   426x240     24    │   4.36MiB  219k https │ avc1.4d4015  219k video only          240p, mp4_dash
92  mp4   426x240     24    │ ~ 6.22MiB  310k m3u8  │ avc1.4d4015       mp4a.40.5           [en]
134 mp4   640x360     24    │  10.65MiB  533k https │ avc1.4d401e  533k video only          360p, mp4_dash
93  mp4   640x360     24    │ ~16.06MiB  802k m3u8  │ avc1.4d401e       mp4a.40.2           [en]
18  mp4   640x360     24  2 │  14.64MiB  733k https │ avc1.42001E       mp4a.40.2       44k [en] 360p
135 mp4   854x480     24    │  18.54MiB  929k https │ avc1.4d401e  929k video only          480p, mp4_dash
94  mp4   854x480     24    │ ~27.09MiB 1353k m3u8  │ avc1.4d401e       mp4a.40.2           [en]
22  mp4   1280x720    24  2 │ ≈35.33MiB 1765k https │ avc1.64001F       mp4a.40.2       44k [en] 720p
136 mp4   1280x720    24    │  32.72MiB 1639k https │ avc1.4d401f 1639k video only          720p, mp4_dash
95  mp4   1280x720    24    │ ~51.47MiB 2571k m3u8  │ avc1.4d401f       mp4a.40.2           [en]
137 mp4   1920x1080   24    │  76.01MiB 3807k https │ avc1.640028 3807k video only          1080p, mp4_dash
96  mp4   1920x1080   24    │ ~94.57MiB 4724k m3u8  │ avc1.640028       mp4a.40.2           [en]

(no need to add safari web client version atm though, can use --user-agent for that)

Hm.. I can't seem to reproduce this, Am I doing something wrong here?
yt-dlp https://www.youtube.com/watch?v=i1csLh-0L9E --list-formats --user-agent "Mozilla/5.0 (Macintosh; Intel Mac OS X 14_4_1) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.3.1 Safari/605.1.15" --extractor-args "youtube:player_client=web"
image

@bashonly
Copy link
Member

bashonly commented Mar 29, 2024

try --impersonate safari instead of passing UA

@clienthax
Copy link
Contributor Author

try --impersonate safari instead of passing UA

Yep, works with that, thanks

@clienthax
Copy link
Contributor Author

Is there a way to only impersonate safari for a specific extractor?

@bashonly
Copy link
Member

there will be when #9474 is merged

MoojMidge added a commit to MoojMidge/plugin.video.youtube that referenced this pull request Mar 30, 2024
- Possibly allows 1080p non-adaptive formats to be used
- yt-dlp/yt-dlp#9546
@@ -238,6 +238,16 @@
},
'INNERTUBE_CONTEXT_CLIENT_NAME': 85
},
# This client has 1080p non split stream videos
'media_connect_frontend': {
Copy link
Member

@bashonly bashonly Apr 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as pukkandan mentioned on discord, _ has special meaning in client names, so we need to change this to something like mediaconnect

@bashonly bashonly added the pending-fixes PR has had changes requested label Apr 8, 2024
@bashonly bashonly removed the pending-fixes PR has had changes requested label May 11, 2024
@bashonly bashonly merged commit cf212d0 into yt-dlp:master May 12, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
site-enhancement Feature request for some website
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

None yet

5 participants