Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Login fails/age-restriction #31034

Closed
infinitejesp opened this issue Jun 18, 2022 · 16 comments · Fixed by #31043
Closed

Login fails/age-restriction #31034

infinitejesp opened this issue Jun 18, 2022 · 16 comments · Fixed by #31043

Comments

@infinitejesp
Copy link

I have followed the checklist, and really need some help, I can't figure this out myself. :((((

Hello, I have looked and tried all apparent workarounds to age-restricted videos, and haven't gotten any to work. I'm using Win10 command prompt. I'd like to download a playlist of my own age-restricted favorites that I've pulled out as time has gone by, but even singe videos, it doesn't work. I just can't log in, getting a 400 error no matter what. I've also tried to just enter the username, and then password when prompted, but that method, for some reason I can't type anything at all and have to close CMD altogether.

@rautamiekka
Copy link
Contributor

The verbose log would still be useful.

@dirkf
Copy link
Contributor

dirkf commented Jun 19, 2022

Duplicate of #29811.

As a rule, the login function of any extractor is liable to break because sites change the procedure, by adding captchas or whatever. If the login function doesn't succeed, or even always, omit -u .../--username ... and use --cookies ....

I have an open dev branch for this issue but the situation was too fluid at the time to progress it. Maybe now.

... using the TVHTML5_SIMPLY_EMBEDDED_PLAYER client (YouTube-DL hasn't implemented this), but the response is throttled. ...

This is the sort of thing that dev branch does. Is this the normal n-parameter throttling?

The side-effect of using different player data is that different sets of format sets are extracted; however, that's probably better than nothing at all.

Otherwise you can make a request using OAuth header:

Is this bearer token something we have to guess, or can it be extracted from the player (presumably), or is it fixed for as long as no-one at YT changes it?

... or similar request using Cookie header.

As proposed above? The cookies have to be associated with a login that passes whatever restriction YT wishes to impose.

@dirkf
Copy link
Contributor

dirkf commented Jun 19, 2022

The pytube implementation could be used but both it and exchange.go require someone to operate a browser alongside the downloader which is not a big improvement on the cookie procedure, even if it doesn't actually need G login.

Surely there must be a way of extracting and posting the so-called user code from the so-called verification url? Or is this actually an unreadable picture of a user code?

@dirkf
Copy link
Contributor

dirkf commented Jun 19, 2022

Yes, thanks.

The problem we're trying to solve is to be able to finesse the YT age restriction without providing a YT login by which the user can be tracked. This should be a solution that doesn't require personally identifiable information to be provided and can be implemented entirely within yt-dl. If the user has to open a browser to log in to G and then respond to some challenge, we've lost, even if the OAuth method might mean that the video access transaction can't directly be linked to the user.

Formerly at least some age restricted videos could be accessed by specifying certain player details and/or using the embed version of the video URL. The inability to reproduce that now is why my dev branch is still only that and not a commit or a PR.

Looking at yt-dlp, the age-gate bypass now depends on the user being recognised as logged-in. If yt-dlp hasn't cracked it, I'm unlikely to spend more time on it. But brainwaves are always welcome.

@dirkf
Copy link
Contributor

dirkf commented Jun 19, 2022

Surely everyone who wants Cr381pDsSsA already has it?

The benefit of the cookie solution is that is already implemented and understood, even if yt-dlp's --cookies-from-browser ... option is simpler.

If there is a subset of age-restricted videos that can be accessed using some player details based on TVHTML5_SIMPLY_EMBEDDED_PLAYER with no user interaction, that mechanism should be included, like this, in fact (bingo):

--- old/youtube_dl/extractor/youtube.py
+++ new/youtube_dl/extractor/youtube.py
@@ -1631,31 +1631,47 @@ class YoutubeIE(YoutubeBaseInfoExtractor):
             player_response = self._call_api(
                 'player', {'videoId': video_id}, video_id)
 
+        def is_agegated(playability):
+            if not isinstance(playability, dict):
+                return
+
+            if playability.get('desktopLegacyAgeGateReason'):
+                return True
+
+            reasons = filter(None, (playability.get(r) for r in ('status', 'reason')))
+            AGE_GATE_REASONS = (
+                'confirm your age', 'age-restricted', 'inappropriate',  # reason
+                'age_verification_required', 'age_check_required',  # status
+            )
+            return any(expected in reason for expected in AGE_GATE_REASONS for reason in reasons)
+
         def get_playability_status(response):
             return try_get(response, lambda x: x['playabilityStatus'], dict) or {}
 
         playability_status = get_playability_status(player_response)
-        if (playability_status.get('reason') == 'Sign in to confirm your age'
+        if (is_agegated(playability_status)
             and int_or_none(self._downloader.params.get('age_limit'), default=18) >= 18):
+
             self.report_age_confirmation()
             query = {
                 'playbackContext': {'contentPlaybackContext': {'html5Preference': 'HTML5_PREF_WANTS'}},
                 'contentCheckOk': True,
                 'racyCheckOk': True,
                 'context': {
-                    'client': {'clientName': 'ANDROID', 'clientVersion': '16.20', 'hl': 'en', 'clientScreen': 'EMBED'},
+                    'client': {'clientName': 'TVHTML5_SIMPLY_EMBEDDED_PLAYER', 'clientVersion': '2.0', 'hl': 'en', 'clientScreen': 'EMBED'},
                     'thirdParty': {'embedUrl': 'https://google.com'},
                 },
                 'videoId': video_id,
             }
             headers = {
-                'X-YouTube-Client-Name': '3',
-                'X-YouTube-Client-Version': '16.20',
+                'X-YouTube-Client-Name': '85',
+                'X-YouTube-Client-Version': '2.0',
                 'Origin': 'https://www.youtube.com'
             }
+
             video_info = self._call_api('player', query, video_id, fatal=False, headers=headers)
             age_gate_status = get_playability_status(video_info)
-            if age_gate_status.get('status') == 'OK':
+            if not is_agegated(age_gate_status):
                 player_response = video_info
                 playability_status = age_gate_status

@dirkf
Copy link
Contributor

dirkf commented Jun 20, 2022

Yes, it didn't work with the right Android version number either: my diff is against last December's attempt.

The TVHTML5_SIMPLY_EMBEDDED_PLAYER hack above passes some of the tests but the media links repeatably get 403 for test video HsUATh_Nc2U (age-gated video with embed allowed in public site). Invidious appears to have the same problem.

Example links, first from yt-dlp -v -f 248 --test HsUATh_Nc2U, which is OK, then from yt-dl -v -f 248 --test HsUATh_Nc2U, which gives 403 (apparently even after adding Sec-Fetch-Mode:navigate):

https://rr3---sn-cu-aigsl.googlevideo.com/videoplayback?expire=1655746652&ei=_FuwYoyRFp6_mLAP9q6fqAI&ip=51.6.99.99&id=o-AJOaqwXPO4AeZ4Zx8DAxF7xrGkMixDDkuyPdPCutHHqk&itag=248&aitags=133%2C134%2C135%2C136%2C137%2C160%2C242%2C243%2C244%2C247%2C248%2C278%2C394%2C395%2C396%2C397%2C398%2C399&source=youtube&requiressl=yes&mh=Vq&mm=31%2C29&mn=sn-cu-aigsl%2Csn-cu-c9is&ms=au%2Crdu&mv=m&mvi=3&pl=25&initcwndbps=1677500&vprv=1&mime=video%2Fwebm&ns=trnT8HugVOs-uLMfb7erWmoG&gir=yes&clen=32667395&dur=176.999&lmt=1589605644861388&mt=1655724768&fvip=7&keepalive=yes&fexp=24001373%2C24007246&c=TVHTML5_SIMPLY_EMBEDDED_PLAYER&txp=5535432&n=n3DrqcB9OjEIJA&sparams=expire%2Cei%2Cip%2Cid%2Caitags%2Csource%2Crequiressl%2Cvprv%2Cmime%2Cns%2Cgir%2Cclen%2Cdur%2Clmt&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRQIhAL243cUSh81nPuPnbZ562gTirzpSuTlPhs79aBebNnoxAiAzBPm7759SnFFRNo8Trx0d799gLnu80Ht0_Z24haqnzA%3D%3D&sig=AOq0QJ8wRAIgDaowXK79w49tTNorAUfFfAYsgCM358db38vwILjt77kCIAbZaO5sNeA2JIRrHUUZJ3TBNxJBoSEIfp-BuZrmQr1D
https://rr3---sn-cu-aigsl.googlevideo.com/videoplayback?expire=1655746758&ei=ZlywYvy4KZGJxgL9vL6ABw&ip=51.6.99.99&id=o-AB5U1B7w_wNKO3Lhpo6t4KdIzdYGNCAN9Gg56qH6cZoK&itag=248&aitags=133%2C134%2C135%2C136%2C137%2C160%2C242%2C243%2C244%2C247%2C248%2C278%2C394%2C395%2C396%2C397%2C398%2C399&source=youtube&requiressl=yes&mh=Vq&mm=31%2C29&mn=sn-cu-aigsl%2Csn-cu-c9is&ms=au%2Crdu&mv=m&mvi=3&pcm2cms=yes&pl=25&initcwndbps=1588750&vprv=1&mime=video%2Fwebm&ns=yenuJgkWTYpAovBOKb6PukoG&gir=yes&clen=32667395&dur=176.999&lmt=1589605644861388&mt=1655725009&fvip=7&keepalive=yes&fexp=24001373%2C24007246&c=TVHTML5_SIMPLY_EMBEDDED_PLAYER&txp=5535432&n=SuJPYjwyVfVb_bW&sparams=expire%2Cei%2Cip%2Cid%2Caitags%2Csource%2Crequiressl%2Cvprv%2Cmime%2Cns%2Cgir%2Cclen%2Cdur%2Clmt&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpcm2cms%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRQIgP8PKBGyAwgV9nTvryLf7wF56nufquCxDAkied78Y-hsCIQC_FmETngNekT1s8XmAr_eRiwrMJEHVfdSVJYjM3vfOwQ%3D%3D&sig=OsyhH-U35jDKUOJ8i_%3DRcQpWDV-CLHIYY1NQ0tiOVvAEiAR-6kZ2PlJ7v0ZsfPAiX2fbBq9pgH3Bgnd5jDShzJ-yAAhIgRw8JQ0qOAO

These parameters differ (1 vs 2):

  • ei, 1 character shorter in 2
  • id
  • pcm2cms=yes in 2 only
  • initcwndbps 1677500 vs 1588750
  • ns
  • mt 1655724768 vs 1655725009 (presumably when the link was generated)
  • n (after descrambling) 1 character longer in 2 (??)
  • lsparams includes %2Cpcm2cms matching the query param above
  • lsig
  • sig, 5 characters longer in 2 (??)

@pukkandan
Copy link
Contributor

pukkandan commented Jun 20, 2022

@dirkf Your API call with TVHTML5_SIMPLY_EMBEDDED_PLAYER is missing signatureTimestamp. See https://github.com/yt-dlp/yt-dlp/blob/a70635b8a1bcf42bf587fe3cd7503f1d092009ce/yt_dlp/extractor/youtube.py#L2621-L2643

I regret suggesting this option to them

@89z You never suggested this or had any input to its implementation afair!! In fact, you only ever opposed it (#29201 (comment)) claiming "oauth is better" as always

@dirkf
Copy link
Contributor

dirkf commented Jun 20, 2022

Your API call with TVHTML5_SIMPLY_EMBEDDED_PLAYER is missing signatureTimestamp.

So it is. I wonder why the defective API data is OK with HtVdAasjOgU but 403 for HsUATh_Nc2U.

@garoto
Copy link

garoto commented Jun 21, 2022

With @dirkf not being able to make a new release since he started maintaining this repo, together with this grifter who deletes all his posts for god knows whatever reason, one starts to question what is even the point of this repo being resuscitated to begin with. Perhaps @rg3 might wanna consider shutting it all down for good, because holy shit, very little is happening.

@garoto
Copy link

garoto commented Jun 21, 2022

Damn, my long going github ruse was exposed.

@pukkandan
Copy link
Contributor

pukkandan commented Jun 21, 2022

Your API call with TVHTML5_SIMPLY_EMBEDDED_PLAYER is missing signatureTimestamp.

So it is. I wonder why the defective API data is OK with HtVdAasjOgU but 403 for HsUATh_Nc2U.

Probably because only the latter uses a rolling cipher (sp)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants
@rautamiekka @dirkf @pukkandan @garoto @infinitejesp and others