Discovery subscriber required pages not working

[derekcentrico]

Checklist

• [X ] I'm reporting a broken site support
• [X ] I've verified that I'm running youtube-dl version 2019.07.02
• I've checked that all provided URLs are alive and playable in a browser
• I've checked that all URLs and arguments with special characters are properly quoted or escaped
• I've searched the bugtracker for similar issues including closed ones

Verbose log

youtube-dl -v --cookies=/home/files/cookiediscovery.txt -f bestvideo[ext!=webm]+bestaudio[ext!=webm]/best[ext!=webm] https://www.discovery.com/tv-shows/naked-and-afraid/full-episodes/stalked-on-the-savannah? 
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'--cookies=/home/files/cookiediscovery.txt', u'-f', u'bestvideo[ext!=webm]+bestaudio[ext!=webm]/best[ext!=webm]', u'https://www.discovery.com/tv-shows/naked-and-afraid/full-episodes/stalked-on-the-savannah?']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.07.02
[debug] Python version 2.7.15+ (CPython) - Linux-4.15.0-52-generic-x86_64-with-Ubuntu-18.04-bionic
[debug] exe versions: ffmpeg 3.4.6, ffprobe 3.4.6
[debug] Proxy map: {}
[Discovery] stalked-on-the-savannah: Downloading webpage
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/bin/youtube-dl/__main__.py", line 19, in <module>
  File "/usr/local/bin/youtube-dl/youtube_dl/__init__.py", line 474, in main
  File "/usr/local/bin/youtube-dl/youtube_dl/__init__.py", line 464, in _real_main
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2008, in download
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 796, in extract_info
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 530, in extract
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/discovery.py", line 75, in _real_extract
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 2819, in _get_cookies
  File "/usr/lib/python2.7/Cookie.py", line 583, in __init__
    if input: self.load(input)
  File "/usr/lib/python2.7/Cookie.py", line 643, in load
    for k, v in rawdata.items():
AttributeError: 'unicode' object has no attribute 'items'

Description

I am attempting to download videos that require a subscription. They play fine in the browser. I created the cookies and confirmed they are Netscape compatible. The error above occurs solely with subscriptions, but does not occur when downloading free content from Discovery. I am a Comcast Xfinity user. I've searched all over Google and read several bug tracker notes that came up for Discovery over the years, but none solve my issue.

cat cookiediscovery.txt

Netscape HTTP Cookie File

http://curl.haxx.se/rfc/cookie_spec.html

This is a generated file! Do not edit.

[remitamine]

https://github.com/ytdl-org/youtube-dl#how-do-i-pass-cookies-to-youtube-dl

[derekcentrico]

You closed and put this as invalid. However my cookie file and usage exactly matches your citation which I read well before posting. What are you asserting I'm failing at here?

[derekcentrico]

The only thing I can think of your assertion is to the use of = in the flag. I tried both with and without based on other examples I've seen in my searching. Here it is without... same error.

youtube-dl -v --cookies /home/files/cookiediscovery.txt -f bestvideo[ext!=webm]+bestaudio[ext!=webm]/best[ext!=webm] https://www.discovery.com/tv-shows/naked-and-afraid/full-episodes/stalked-on-the-savannah?

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'--cookies', u'/home/files/cookiediscovery.txt', u'-f', u'bestvideo[ext!=webm]+bestaudio[ext!=webm]/best[ext!=webm]', u'https://www.discovery.com/tv-shows/naked-and-afraid/full-episodes/stalked-on-the-savannah?']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.07.02
[debug] Python version 2.7.15+ (CPython) - Linux-4.15.0-52-generic-x86_64-with-Ubuntu-18.04-bionic
[debug] exe versions: ffmpeg 3.4.6, ffprobe 3.4.6
[debug] Proxy map: {}
[Discovery] stalked-on-the-savannah: Downloading webpage
Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/usr/local/bin/youtube-dl/main.py", line 19, in
File "/usr/local/bin/youtube-dl/youtube_dl/init.py", line 474, in main
File "/usr/local/bin/youtube-dl/youtube_dl/init.py", line 464, in _real_main
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2008, in download
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 796, in extract_info
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 530, in extract
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/discovery.py", line 75, in _real_extract
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 2819, in _get_cookies
File "/usr/lib/python2.7/Cookie.py", line 583, in init
if input: self.load(input)
File "/usr/lib/python2.7/Cookie.py", line 643, in load
for k, v in rawdata.items():
AttributeError: 'unicode' object has no attribute 'items'

[remitamine]

obviously, the cookie file you have posted is empty, when you export the cookie file from the browser when you're logged, the file should contain cookies for the domain you're trying to download from(eosAf cookie for discovery domains).

[remitamine]

and for error you have posted, it's a python 2 specific bug, there is already an issue opened for it(#20673).

[derekcentrico]

Apologies. I pasted the top portion of said cookie file to show Netscape format and not Chrome variety. I didn't realize I should post the whole thing. Additionally, thank you for letting me know it is a known python error. I will try changing my shell to python3 and hope for a patch so my automation stuff will function using cookies.

[derekcentrico]

For what it's worth I built from latest source code and it won't work. Additionally, I made a well-discussed file change to adobepass.py for Comcast OAUTH which did not work either. (see https://github.com/bhamboss/youtube-dl/blob/patch-1/youtube_dl/extractor/adobepass.py) I passed just --cookie and --cookie with --ap-mso related flags as well.

403 forbidden.

I don't necessarily want to be a pain in the rear by posting a new bug report to have an open status for assistance so I'm posting here for now.

With --cookies and --ap-mso:

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', '--ap-mso', 'Comcast_SSO', '--ap-username', 'PRIVATE', '--ap-password', 'PRIVATE', '--cookies', '/home/derek/Desktop/cookies.txt', '-f', 'bestvideo[ext!=webm]+bestaudio[ext!=webm]/best[ext!=webm]', 'https://www.discovery.com/tv-shows/naked-and-afraid/full-episodes/stalked-on-the-savannah?']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.07.14
[debug] Python version 3.6.8 (CPython) - Linux-4.18.0-15-generic-x86_64-with-Ubuntu-18.04-bionic
[debug] exe versions: none
[debug] Proxy map: {}
[Discovery] stalked-on-the-savannah: Downloading webpage
[Discovery] stalked-on-the-savannah: Downloading JSON metadata
ERROR: This video is only available via cable service provider subscription that is not currently supported. You may want to use --cookies.
Traceback (most recent call last):
File "./youtube-dl/youtube_dl/extractor/common.py", line 627, in _request_webpage
return self._downloader.urlopen(url_or_request)
File "./youtube-dl/youtube_dl/YoutubeDL.py", line 2227, in urlopen
return self._opener.open(req, timeout=self._socket_timeout)
File "/usr/lib/python3.6/urllib/request.py", line 532, in open
response = meth(req, response)
File "/usr/lib/python3.6/urllib/request.py", line 642, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python3.6/urllib/request.py", line 570, in error
return self._call_chain(*args)
File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
result = func(*args)
File "/usr/lib/python3.6/urllib/request.py", line 650, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "./youtube-dl/youtube_dl/extractor/discovery.py", line 102, in _real_extract
display_id, headers=headers)
File "./youtube-dl/youtube_dl/extractor/common.py", line 892, in _download_json
expected_status=expected_status)
File "./youtube-dl/youtube_dl/extractor/common.py", line 870, in _download_json_handle
expected_status=expected_status)
File "./youtube-dl/youtube_dl/extractor/common.py", line 660, in _download_webpage_handle
urlh = self._request_webpage(url_or_request, video_id, note, errnote, fatal, data=data, headers=headers, query=query, expected_status=expected_status)
File "./youtube-dl/youtube_dl/extractor/common.py", line 645, in _request_webpage
raise ExtractorError(errmsg, sys.exc_info()[2], cause=err)
youtube_dl.utils.ExtractorError: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
File "./youtube-dl/youtube_dl/extractor/common.py", line 627, in _request_webpage
return self._downloader.urlopen(url_or_request)
File "./youtube-dl/youtube_dl/YoutubeDL.py", line 2227, in urlopen
return self._opener.open(req, timeout=self._socket_timeout)
File "/usr/lib/python3.6/urllib/request.py", line 532, in open
response = meth(req, response)
File "/usr/lib/python3.6/urllib/request.py", line 642, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python3.6/urllib/request.py", line 570, in error
return self._call_chain(*args)
File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
result = func(*args)
File "/usr/lib/python3.6/urllib/request.py", line 650, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "./youtube-dl/youtube_dl/extractor/discovery.py", line 102, in _real_extract
display_id, headers=headers)
File "./youtube-dl/youtube_dl/extractor/common.py", line 892, in _download_json
expected_status=expected_status)
File "./youtube-dl/youtube_dl/extractor/common.py", line 870, in _download_json_handle
expected_status=expected_status)
File "./youtube-dl/youtube_dl/extractor/common.py", line 660, in _download_webpage_handle
urlh = self._request_webpage(url_or_request, video_id, note, errnote, fatal, data=data, headers=headers, query=query, expected_status=expected_status)
File "./youtube-dl/youtube_dl/extractor/common.py", line 645, in _request_webpage
raise ExtractorError(errmsg, sys.exc_info()[2], cause=err)
youtube_dl.utils.ExtractorError: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "./youtube-dl/youtube_dl/YoutubeDL.py", line 796, in extract_info
ie_result = ie.extract(url)
File "./youtube-dl/youtube_dl/extractor/common.py", line 530, in extract
ie_result = self._real_extract(url)
File "./youtube-dl/youtube_dl/extractor/discovery.py", line 112, in _real_extract
' is not currently supported. You may want to use --cookies.', expected=True)
youtube_dl.utils.ExtractorError: This video is only available via cable service provider subscription that is not currently supported. You may want to use --cookies.

With just --cookies:

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', '--cookies', '/home/derek/Desktop/cookies.txt', '-f', 'bestvideo[ext!=webm]+bestaudio[ext!=webm]/best[ext!=webm]', 'https://www.discovery.com/tv-shows/naked-and-afraid/full-episodes/stalked-on-the-savannah?']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.07.14
[debug] Python version 3.6.8 (CPython) - Linux-4.18.0-15-generic-x86_64-with-Ubuntu-18.04-bionic
[debug] exe versions: none
[debug] Proxy map: {}
[Discovery] stalked-on-the-savannah: Downloading webpage
[Discovery] stalked-on-the-savannah: Downloading JSON metadata
ERROR: This video is only available via cable service provider subscription that is not currently supported. You may want to use --cookies.
Traceback (most recent call last):
File "./youtube-dl/youtube_dl/extractor/common.py", line 627, in _request_webpage
return self._downloader.urlopen(url_or_request)
File "./youtube-dl/youtube_dl/YoutubeDL.py", line 2227, in urlopen
return self._opener.open(req, timeout=self._socket_timeout)
File "/usr/lib/python3.6/urllib/request.py", line 532, in open
response = meth(req, response)
File "/usr/lib/python3.6/urllib/request.py", line 642, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python3.6/urllib/request.py", line 570, in error
return self._call_chain(*args)
File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
result = func(*args)
File "/usr/lib/python3.6/urllib/request.py", line 650, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "./youtube-dl/youtube_dl/extractor/discovery.py", line 102, in _real_extract
display_id, headers=headers)
File "./youtube-dl/youtube_dl/extractor/common.py", line 892, in _download_json
expected_status=expected_status)
File "./youtube-dl/youtube_dl/extractor/common.py", line 870, in _download_json_handle
expected_status=expected_status)
File "./youtube-dl/youtube_dl/extractor/common.py", line 660, in _download_webpage_handle
urlh = self._request_webpage(url_or_request, video_id, note, errnote, fatal, data=data, headers=headers, query=query, expected_status=expected_status)
File "./youtube-dl/youtube_dl/extractor/common.py", line 645, in _request_webpage
raise ExtractorError(errmsg, sys.exc_info()[2], cause=err)
youtube_dl.utils.ExtractorError: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
File "./youtube-dl/youtube_dl/extractor/common.py", line 627, in _request_webpage
return self._downloader.urlopen(url_or_request)
File "./youtube-dl/youtube_dl/YoutubeDL.py", line 2227, in urlopen
return self._opener.open(req, timeout=self._socket_timeout)
File "/usr/lib/python3.6/urllib/request.py", line 532, in open
response = meth(req, response)
File "/usr/lib/python3.6/urllib/request.py", line 642, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python3.6/urllib/request.py", line 570, in error
return self._call_chain(*args)
File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
result = func(*args)
File "/usr/lib/python3.6/urllib/request.py", line 650, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "./youtube-dl/youtube_dl/extractor/discovery.py", line 102, in _real_extract
display_id, headers=headers)
File "./youtube-dl/youtube_dl/extractor/common.py", line 892, in _download_json
expected_status=expected_status)
File "./youtube-dl/youtube_dl/extractor/common.py", line 870, in _download_json_handle
expected_status=expected_status)
File "./youtube-dl/youtube_dl/extractor/common.py", line 660, in _download_webpage_handle
urlh = self._request_webpage(url_or_request, video_id, note, errnote, fatal, data=data, headers=headers, query=query, expected_status=expected_status)
File "./youtube-dl/youtube_dl/extractor/common.py", line 645, in _request_webpage
raise ExtractorError(errmsg, sys.exc_info()[2], cause=err)
youtube_dl.utils.ExtractorError: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "./youtube-dl/youtube_dl/YoutubeDL.py", line 796, in extract_info
ie_result = ie.extract(url)
File "./youtube-dl/youtube_dl/extractor/common.py", line 530, in extract
ie_result = self._real_extract(url)
File "./youtube-dl/youtube_dl/extractor/discovery.py", line 112, in _real_extract
' is not currently supported. You may want to use --cookies.', expected=True)
youtube_dl.utils.ExtractorError: This video is only available via cable service provider subscription that is not currently supported. You may want to use --cookies.

[Diegus83]

There has been a commit a couple days ago that added the new "go.discovery.com" URLs and it is working with the proper cookies extracted from the browser.

You can run the latest source without compiling by doing
python -m youtube_dl

If you have multiple versions of python like it is the case of Mac users with Brew installed, I would type instead
python3 -m youtube_dl

Since otherwise you may end up running version 2.7 of Python which won't work.