Forbidden (403) Message From Crunchyroll

[jhowellburke]

Unable to download videos from Crunchyroll I keep getting a Forbidden error message:

[wanderer@hive ~]$ youtube-dl --sub-lang enUS --write-sub https://www.crunchyroll.com/the-promised-neverland --merge-output-format mp4 --verbose
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--sub-lang', 'enUS', '--write-sub', 'https://www.crunchyroll.com/the-promised-neverland', '--merge-output-format', 'mp4', '--verbose']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2020.05.29
[debug] Python version 3.7.7 (CPython) - Linux-5.6.13-200.fc31.x86_64-x86_64-with-fedora-31-Thirty_One
[debug] exe versions: ffmpeg 4.2.3, ffprobe 4.2.3
[debug] Proxy map: {}
[crunchyroll:playlist] the-promised-neverland: Downloading webpage
ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 627, in _request_webpage
return self._downloader.urlopen(url_or_request)
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2238, in urlopen
return self._opener.open(req, timeout=self._socket_timeout)
File "/usr/lib64/python3.7/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib64/python3.7/urllib/request.py", line 641, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib64/python3.7/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib64/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "/usr/lib64/python3.7/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)

[twm]

I've been seeing the same since May 28th.

Upgrade from youtube-dl 2020.5.8 to youtube-dl-2020.5.29 didn't help.

I have tried updating my user agent and cookies but I still get the same result:

$ /home/htpc/.local/bin/youtube-dl --no-call-home --no-progress --cookie /home/htpc/cookies.txt --user-agent 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:76.0) Gecko/20100101 Firefox/76.0' https://www.crunchyroll.com/my-next-life-as-a-villainess-all-routes-lead-to-doom --match-title 'Episode \d+' --flat-playlist --dump-json --verbose
[debug] System config: []
[debug] User config: ['--write-sub', '--sub-format=ass/srt/best', '--sub-lang=enUS']
[debug] Custom config: []
[debug] Command-line args: ['--no-call-home', '--no-progress', '--cookie', '/home/htpc/cookies.txt', '--user-agent', 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:76.0) Gecko/20100101 Firefox/76.0', 'https://www.crunchyroll.com/my-next-life-as-a-villainess-all-routes-lead-to-doom', '--match-title', 'Episode \\d+', '--flat-playlist', '--dump-json', '--verbose']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2020.05.29
[debug] Python version 3.5.2 (CPython) - Linux-4.15.0-47-generic-x86_64-with-Ubuntu-16.04-xenial
[debug] exe versions: ffmpeg 2.8.15, ffprobe 2.8.15, rtmpdump 2.4
[debug] Proxy map: {}
ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "/home/htpc/.local/venvs/youtube-dl/lib/python3.5/site-packages/youtube_dl/extractor/common.py", line 627, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/home/htpc/.local/venvs/youtube-dl/lib/python3.5/site-packages/youtube_dl/YoutubeDL.py", line 2238, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python3.5/urllib/request.py", line 472, in open
    response = meth(req, response)
  File "/usr/lib/python3.5/urllib/request.py", line 582, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.5/urllib/request.py", line 510, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 590, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)

[narutojawad]

#25437

[qwertz19281]

Works for me using python 3.7+, since TLS 1.3 was implemented for python ssl

[skid9000]

Python 3.8 on Arch...

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-vvv', '--username', 'PRIVATE', '--password', 'PRIVATE', '--write-sub', '--sub-lang', 'frFR', '--sub-format', 'ass', '-f', 'best[height=1080]', '--user-agent', 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:65.0) Gecko/20100101 Firefox/76.0', 'https://www.crunchyroll.com/fr/nyanko-days/episode-12-with-my-cats-728677']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2020.05.29
[debug] Python version 3.8.3 (CPython) - Linux-5.6.15-arch1-1-x86_64-with-glibc2.2.5
[debug] exe versions: ffmpeg 4.2.3, ffprobe 4.2.3, rtmpdump 2.4
[debug] Proxy map: {}
[crunchyroll] Downloading login page
ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "/usr/lib/python3.8/site-packages/youtube_dl/extractor/common.py", line 627, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/usr/lib/python3.8/site-packages/youtube_dl/YoutubeDL.py", line 2238, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python3.8/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
    response = self.parent.error(
  File "/usr/lib/python3.8/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)

[pccanales]

Same here:

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['https://www.crunchyroll.com/es/my-next-life-as-a-villainess-all-routes-lead-to-doom/episode-10-the-moment-of-my-doom-arrived-part-1-794628', '--username', 'PRIVATE', '--password', 'PRIVATE', '-v']
[debug] Encodings: locale cp1252, fs mbcs, out cp850, pref cp1252
[debug] youtube-dl version 2020.06.06
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.19041
[debug] exe versions: ffmpeg N-91441-g3a2d21bc5f, ffprobe N-91441-g3a2d21bc5f
[debug] Proxy map: {}
[crunchyroll] Downloading login page
ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpd0boo4lk\build\youtube_dl\extractor\common.py", line 627, in _request_webpage
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpd0boo4lk\build\youtube_dl\YoutubeDL.py", line 2238, in urlopen
File "C:\Python\Python34\lib\urllib\request.py", line 470, in open
File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response
File "C:\Python\Python34\lib\urllib\request.py", line 508, in error
File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default

[krackers]

While enabling tlsv1.3 might be one solution, connections from tlsv1.2 and older clients are still supported, so there should be some way to get past the 403 to at least the 503 captcha page. Merely changing the user-agent to something older isn't sufficient though, so perhaps it's also looking at the list of cipher suites as an indication of how recent a client is.

Interestingly, curl using a user-agent of "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36" built against openssl 1.0.2 (which will default to tls 1.2) results in 403, but forcing tlsv1.1 works fine.

[twm]

Confirmed: it works for me with Python 3.8.2 (as in Ubuntu 20.04) and cookies captured with Firefox.

[skid9000]

And with login/password ?

[dirkf]

Continued in #28398.