Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Youtube cookies expire within hours #8227

Open
8 of 9 tasks
m1bcodes opened this issue Sep 28, 2023 · 24 comments
Open
8 of 9 tasks

Youtube cookies expire within hours #8227

m1bcodes opened this issue Sep 28, 2023 · 24 comments
Labels
account-needed Account details are needed to test/fix this external issue Issue with an external tool site-bug Issue with a specific website

Comments

@m1bcodes
Copy link

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

  • I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

Please make sure the question is worded well enough to be understood

I have been used to youtube coookies lasting a couple of weeks. But since a few weeks I have to update the cookies for yt-dlp daily.
Does anyone know what is happening here?
My guess is that youtube change its policies. If that is the case, is there a way to automate the process?
My current "workflow":

  1. Opening youtube.com in firefox,
  2. log in to my channel
  3. export cookies (all domains) using "Export cookies"
  4. Use the cookies file with yt-dlp

Provide verbose output that clearly demonstrates the problem

  • Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
  • If using API, add 'verbose': True to YoutubeDL params instead
  • Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

[debug] Command-line config: ['-vU']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.09.24 [088add956] (pip)
[debug] Python 3.11.5 (CPython armv7l 32bit) - Linux-5.10.63-v7+-armv7l-with-glibc2.28 (OpenSSL 1.1.1n  15 Mar 2022, glibc 2.28)
[debug] exe versions: ffmpeg 4.1.11-0, ffprobe 4.1.11-0
[debug] Optional libraries: Cryptodome-3.19.0, brotli-1.1.0, certifi-2023.07.22, mutagen-1.47.0, sqlite3-3.27.2, websockets-11.0.3
[debug] Proxy map: {}
[debug] Loaded 1886 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version: stable@2023.09.24, Current version: stable@2023.09.24
yt-dlp is up to date (stable@2023.09.24)
@m1bcodes m1bcodes added the question Question label Sep 28, 2023
@bashonly

This comment was marked as outdated.

@wutongskype
Copy link

This has also happened to me. It usually happens when I download a member-only video of a certain channel, and the cookies of yt-dlp often expire, and it will prompt that member permissions are required.

Like this
ERROR: [youtube] IIqoefSX61U: This video is available to this channel's members on level: Lifeano Premium (or any higher level). Join this channel to get access to members-only content and other exclusive perks.

@coletdjnz coletdjnz added site-bug Issue with a specific website needs-investigating Needs research, information is welcome and removed question Question labels Nov 3, 2023
@coletdjnz
Copy link
Member

I'm noticing this too. Will need some investigation.

I suspect there might have been a change where active requests in the browser are causing cookies to be replemised.

@wutongskype
Copy link

I'm noticing this too. Will need some investigation.

I suspect there might have been a change where active requests in the browser are causing cookies to be replemised.

I first tried to export cookies and use --cookies "%~dp0\cookies.txt" to specify cookies, but after a few minutes, it became invalid. When I downloaded member-only videos, it prompted me to need a higher level. permission.

Later, I used the --cookies-from-browser firefox parameter and logged in to YouTube in the browser. Everything worked fine at first, but after a few minutes to dozens of minutes, the same cookie failure occurred.

@wutongskype
Copy link

I'm noticing this too. Will need some investigation.

I suspect there might have been a change where active requests in the browser are causing cookies to be replemised.

I also tried adding a plug-in to firefox to automatically refresh the page, but it also didn't work.

If this is a new mechanism added by YouTube, then maybe a function needs to be added,
For example, re-obtain cookies from the browser when downloading each playlist and video.

@coletdjnz coletdjnz reopened this Nov 3, 2023
@coletdjnz
Copy link
Member

coletdjnz commented Nov 4, 2023

From a quick look: the browser appears to be callinghttps://accounts.youtube.com/RotateCookiesPage?origin=https://www.youtube.com&yt_pid=1. This runs another javascript rube goldberg machine to transform a value akin to nsig. It then calls https://accounts.youtube.com/RotateCookies with that transformed value which refreshes the cookies.

I'm guessing this is required or else the cookies are invalidated by YouTube if they are not refreshed.

We could attempt to implement this, but, I don't think its really feasible or worthwhile to maintain.

Probably better off spending that time implementing oauth support for YouTube.

@coletdjnz coletdjnz added help-wanted Extra attention is needed account-needed Account details are needed to test/fix this labels Nov 4, 2023
@jim60105
Copy link

jim60105 commented Nov 4, 2023

From a quick look: the browser appears to be callinghttps://accounts.youtube.com/RotateCookiesPage?origin=https://www.youtube.com&yt_pid=1. This runs another javascript rube goldberg machine to transform a value akin to nsig. It then calls https://accounts.youtube.com/RotateCookies with that transformed value which refreshes the cookies.

I'm guessing this is required or else the cookies are invalidated by YouTube if they are not refreshed.

We could attempt to implement this, but, I don't think its really feasible or worthwhile to maintain.

Probably better off spending that time implementing oauth support for YouTube.

A simple workaround would be to generate the cookies file in an incognito window, where the cookies are never used again in browser and therefore will never be rotated.

A few weeks ago, I saw this workaround somewhere (without knowing why) and tried it out. And my cookies file is still valid until now.

@coletdjnz
Copy link
Member

From a quick look: the browser appears to be callinghttps://accounts.youtube.com/RotateCookiesPage?origin=https://www.youtube.com&yt_pid=1. This runs another javascript rube goldberg machine to transform a value akin to nsig. It then calls https://accounts.youtube.com/RotateCookies with that transformed value which refreshes the cookies.
I'm guessing this is required or else the cookies are invalidated by YouTube if they are not refreshed.
We could attempt to implement this, but, I don't think its really feasible or worthwhile to maintain.
Probably better off spending that time implementing oauth support for YouTube.

A simple workaround would be to generate the cookies file in an incognito window, where the cookies are never used again in browser and therefore will never be rotated.

A few weeks ago, I saw this workaround somewhere (without knowing why) and tried it out. And my cookies file is still valid until now.

Good catch. You would just have to make sure the cookies are not rotated in the process. But, I suppose you can just open another tab and close the YouTube tab, then export cookies.

@wutongskype
Copy link

From a quick look: the browser appears to be callinghttps://accounts.youtube.com/RotateCookiesPage?origin=https://www.youtube.com&yt_pid=1. This runs another javascript rube goldberg machine to transform a value akin to nsig. It then calls https://accounts.youtube.com/RotateCookies with that transformed value which refreshes the cookies.
I'm guessing this is required or else the cookies are invalidated by YouTube if they are not refreshed.
We could attempt to implement this, but, I don't think its really feasible or worthwhile to maintain.
Probably better off spending that time implementing oauth support for YouTube.

A simple workaround would be to generate the cookies file in an incognito window, where the cookies are never used again in browser and therefore will never be rotated.
A few weeks ago, I saw this workaround somewhere (without knowing why) and tried it out. And my cookies file is still valid until now.

Good catch. You would just have to make sure the cookies are not rotated in the process. But, I suppose you can just open another tab and close the YouTube tab, then export cookies.

I found that a similar cookie invalidation problem also occurs on bilibili. I’m not sure whether incognito mode can solve this problem because I haven’t found a way to export cookies in incognito mode.

@naodesu
Copy link

naodesu commented Nov 4, 2023

workaround would be to generate the cookies file in an incognito window, where the cookies are never used again in browser and therefore will never be rotated.

Works for me too. But I used a separate Firefox profile, exported cookies and never opened that profile again until I need to re-export cookies again.

@bashonly
Copy link
Member

bashonly commented Nov 4, 2023

Firefox containers may be useful for this

@jim60105
Copy link

jim60105 commented Nov 5, 2023

I’m not sure whether incognito mode can solve this problem because I haven’t found a way to export cookies in incognito mode.

Toggle the "Allow in Incognito" option on the Chrome Extension setting page of your cookie export extension.

@jim60105
Copy link

jim60105 commented Nov 5, 2023

A simple workaround would be to generate the cookies file in an incognito window, where the cookies are never used again in browser and therefore will never be rotated.
A few weeks ago, I saw this workaround somewhere (without knowing why) and tried it out. And my cookies file is still valid until now.

Good catch. You would just have to make sure the cookies are not rotated in the process. But, I suppose you can just open another tab and close the YouTube tab, then export cookies.

I'm not sure if the last saved cookies will be rotated the next time I open Youtube in browser, but incognito mode apparently can avoid this possibility.

@lbmaian
Copy link

lbmaian commented Nov 13, 2023

FWIW, for a separate project that requires YouTube cookies, I ended up using yt-dlp as a library for its yt_dlp.cookies.extract_cookies_from_browser such that whenever I get playabilityStatus == UNPLAYABLE (or just inexplicably empty data), I try refetching cookies and trying again once in case it was due to video being members-only and cookies expiring.

It seems to be working out. The same solution could be applied here.

@szv99
Copy link

szv99 commented Nov 27, 2023

I've uploaded a cookie regenerator on my gist. It can be easily ported to youtube just by changing the urls, I guess.

https://gist.github.com/szv99/f78c032736443fab51075bc45f9faf09

@m1bcodes
Copy link
Author

In the meantime I figured out what was wrong on my side:
I was using firefox all the time with the same account logged in as I am using for yt-dlp. So for some reason this caused the cookies to expire.

So I deleted the youtube and google related firefox cache and since then the issue did not occur again.

-- sometimes the bug sits in front of the screen...

@bashonly
Copy link
Member

I was using [my browser] all the time with the same account logged in as I am using for yt-dlp.

I'd imagine this is common practice for many/most users of yt-dlp, and as such we should keep this issue open until there is some kind of solution applied to yt-dlp

@bashonly bashonly reopened this Nov 28, 2023
@pukkandan
Copy link
Member

#8227 (comment) looks very promising

@coletdjnz
Copy link
Member

I've uploaded a cookie regenerator on my gist. It can be easily ported to youtube just by changing the urls, I guess.

https://gist.github.com/szv99/f78c032736443fab51075bc45f9faf09

Wouldn't this invalidate the cookies in the browser if they don't rotate before this script is run?

@szv99
Copy link

szv99 commented Nov 28, 2023

I've uploaded a cookie regenerator on my gist. It can be easily ported to youtube just by changing the urls, I guess.
https://gist.github.com/szv99/f78c032736443fab51075bc45f9faf09

Wouldn't this invalidate the cookies in the browser if they don't rotate before this script is run?

I hadn't thought about that but also I haven't experienced any problems.
I got logged off from my google account when I kinda spammed the rotate cookie request and got ratelimited.

@theo77186
Copy link

From a quick look: the browser appears to be callinghttps://accounts.youtube.com/RotateCookiesPage?origin=https://www.youtube.com&yt_pid=1. This runs another javascript rube goldberg machine to transform a value akin to nsig. It then calls https://accounts.youtube.com/RotateCookies with that transformed value which refreshes the cookies.

I'm guessing this is required or else the cookies are invalidated by YouTube if they are not refreshed.

We could attempt to implement this, but, I don't think its really feasible or worthwhile to maintain.

Probably better off spending that time implementing oauth support for YouTube.

Blocking these 2 URLs (using an ad-blocker) can be a workaround for this, in addition to generate the cookies from a private window/a browser you never use.

@coletdjnz
Copy link
Member

coletdjnz commented Dec 22, 2023

TL;DR of current known workarounds:

  1. Export cookies in a private/incognito window
    • Open new private/incognito tab, log into youtube, open a new tab, close youtube tab then export cookies. Be sure to close the window after to ensure the cookies are never used in the browser.
  2. Block https://accounts.youtube.com/RotateCookiesPage and https://accounts.youtube.com/RotateCookies in your browser you are using cookies from (not recommended)
  3. Use --cookies-from-browser if your action is quick
  4. Log in using oauth2 with this experimental plugin: https://github.com/coletdjnz/yt-dlp-youtube-oauth2

@gamer191
Copy link
Collaborator

gamer191 commented Dec 22, 2023

I'm guessing this is required or else the cookies are invalidated by YouTube if they are not refreshed.

Rotating cookies is a genuine security feature. I doubt this is YouTube cracking down on us, therefore I doubt they’ll start blocking us because we don’t rotate cookies
EDIT: I’m assuming, of course, that the output of the cookie rotation isn’t predictable. If it is then I have no idea what the point is

@codenamedpkt
Copy link

https://github.com/coletdjnz/yt-dlp-youtube-oauth2

Yes! Time to grab this link above and integrate to official builds right now. Seems Google started deprecate third party cookies, first party cookies deprecates soon in short to middle. They were annouced and deploying new mechanism on Chromium based browser. YES! TIME TO DEPLOY OAUTH LOGIN SUPPORT RIGHT NOW.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
account-needed Account details are needed to test/fix this external issue Issue with an external tool site-bug Issue with a specific website
Projects
Status: done
Development

No branches or pull requests