New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

--cookies: be more flexible in the file format #4539

Open
ghost opened this Issue Dec 22, 2014 · 7 comments

Comments

Projects
None yet
6 participants
@ghost

ghost commented Dec 22, 2014

youtube-dl -e "https://www.youtube.com/watch?v=5GTXaObrXr0" --print-traffic --cookie my_cookies.txt
Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/usr/local/bin/youtube-dl/main.py", line 19, in
File "/usr/local/bin/youtube-dl/youtube_dl/init.py", line 361, in main
File "/usr/local/bin/youtube-dl/youtube_dl/init.py", line 330, in _real_main
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 280, in init
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1439, in _setup_opener
File "/usr/lib/python2.7/cookielib.py", line 1763, in load
self._really_load(f, filename, ignore_discard, ignore_expires)
File "/usr/lib/python2.7/_MozillaCookieJar.py", line 55, in _really_load
filename)
cookielib.LoadError: 'my_cookies.txt' does not look like a Netscape format cookies file

my_cookies.txt source:

.youtube.com TRUE / FALSE 1440244957 PREF f1=50000000
.youtube.com TRUE / FALSE 1482278977 gojf ddd4fbef2abea5cf5ded8493c2ed5f07cwsAAABFUGRhSDcyaEw0RQ==
.youtube.com TRUE / FALSE 1440244956 VISITOR_INFO1_LIVE euPxAjtmFn0

@jaimeMF

This comment has been minimized.

Collaborator

jaimeMF commented Dec 23, 2014

You have to use tabs instead of spaces in the cookie file.
For example curl won't raise an error but it won't add any cookies to the request unless you use tabs.

@jaimeMF jaimeMF closed this Dec 23, 2014

@phihag

This comment has been minimized.

Collaborator

phihag commented Dec 23, 2014

Reopening - we should support tabs as well.

@phihag phihag reopened this Dec 23, 2014

@jaimeMF jaimeMF changed the title from --cookie to --cookies: be more flexible in the file format Mar 13, 2015

@jaimeMF

This comment has been minimized.

Collaborator

jaimeMF commented Mar 13, 2015

Apparently you also need to add # Netscape HTTP Cookie File (with spaces not tabs) at the start of the file.

@lakhman

This comment has been minimized.

lakhman commented Jul 2, 2015

How to use youtube-dl with cookies to download videos from a session

For other's who end up here looking for some docs on how to use youtube-dl and examples of a cookies.txt file.


Valid Cookie File

Your cookies.txt file should look like this:

Cookies.txt

# Netscape HTTP Cookie File
# http://curl.haxx.se/rfc/cookie_spec.html
# This is a generated file!  Do not edit.
# ALL SPACES MUST BE TABS! - IT WILL THROW AN ERROR! 

.domain.com TRUE    /   FALSE   1438536522  PHPSESSID   sf0fknhqbs7r4cj6ts4soknnk1

Remember TABS ONLY (Some editors may convert tabs to spaces on save so make sure to check this if you're still getting this error!)

Usage

You can use it like so (Jazzed up in a bash script for multiple items):

#!/usr/bin/env bash

# Set our sources to loop through * download dir
source=("http://domain.com/video1" "http://domain.com/video2") # space-delimited, no comma's!
download_dir="/Users/your.username/Desktop/youtube-dl"

# Traverse to the directory we want to download into
mkdir -p $download_dir && cd $download_dir

# Loop each url and youtube-dl it
for i in "${source[@]}"
do
trap "echo Exited!; exit;" SIGINT SIGTERM # gracefully exit with a ctrl+c if called
echo $i
youtube-dl --cookies cookies.txt $i
done

Further Details for each cookie value

domain - The domain that created AND that can read the variable. 
flag - A TRUE/FALSE value indicating if all machines within a given domain can access the variable.
path - The path within the domain that the variable is valid for. 
secure - A TRUE/FALSE value indicating if a secure connection with the domain is needed. 
expiration - The UNIX time that the variable will expire on 
name - The name of the variable.
value - The value of the variable.

Invalid Cookie file error

Example of an error with invalid cookie.txt content (spaces instead of tabs or invalid content).

/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/_MozillaCookieJar.py:109: UserWarning: cookielib bug!
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/_MozillaCookieJar.py", line 71, in _really_load
    line.split("\t")
ValueError: need more than 1 value to unpack

  _warn_unhandled_exception()
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/bin/youtube-dl/__main__.py", line 19, in <module>
  File "/usr/local/bin/youtube-dl/youtube_dl/__init__.py", line 405, in main
  File "/usr/local/bin/youtube-dl/youtube_dl/__init__.py", line 372, in _real_main
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 330, in __init__
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1810, in _setup_opener
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/cookielib.py", line 1763, in load
    self._really_load(f, filename, ignore_discard, ignore_expires)
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/_MozillaCookieJar.py", line 111, in _really_load
    (filename, line))
cookielib.LoadError: invalid Netscape format cookies file u'cookies.txt': '.domain.com  TRUE    /   FALSE   1438536522  PHPSESSID   sf0fknhqbs7r4cj6ts4soknnk1'

youtube-dl cookies example, youtube-dl auth, youtube-dl session, youtube-dl PHPSESSID, invalid netscape cookie file

@krokofant

This comment has been minimized.

krokofant commented May 15, 2017

I keep getting an assertion error when a cookie has a . prefix.

    assert domain_specified == initial_dot
AssertionError

like

.example.com	FALSE	/	FALSE	152872222	__qca	P0-122312-332131
@mcchots

This comment has been minimized.

mcchots commented Jun 24, 2017

I've worked around the assert error using the solution found here to regenerate the cookie file with curl
curl -b oldcookiefile.txt --cookie-jar newcookiefile.txt 'http://url'

@fasiha

This comment has been minimized.

fasiha commented Nov 9, 2017

I had to remove decimal portions of timestamps from the expiration column, converting them to integers. Then python2’s cookielib would import it and so youtube-dl works.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment