[anime-on-demand.de] download broken

[tadev]

Checklist

• I'm reporting a broken site support
• I've verified that I'm running youtube-dl version 2020.03.24
• I've checked that all provided URLs are alive and playable in a browser
• I've checked that all URLs and arguments with special characters are properly quoted or escaped
• I've searched the bugtracker for similar issues including closed ones

Verbose log

youtube-dl --add-metadata --no-geo-bypass --all-subs --write-sub -o "%%(playlist)s/%%(playlist_index)s - %%(title)s.%%(ext)s" --username "PRIVATE" --password "PRIVATE" -f "best[format_id*=ger-Dub][height=720]/best[format_id*=ger-Dub]" -v https://www.anime-on-demand.de/anime/179

[debug] System config: []
[debug] User config: ['--no-call-home']
[debug] Custom config: []
[debug] Command-line args: ['--add-metadata', '--no-geo-bypass', '--all-subs', '--write-sub', '-o', '%(playlist)s/%(playlist_index)s - %(title)s.%(ext)s', '--username', 'PRIVATE', '--password', 'PRIVATE', '-f', 'best[format_id*=ger-Dub][height=720]/best[format_id*=ger-Dub]', '-v', 'https://www.anime-on-demand.de/anime/179']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252
[debug] youtube-dl version 2020.03.24
[debug] Python version 3.8.2 (CPython) - Windows-10-10.0.14393-SP0
[debug] exe versions: ffmpeg git-2020-03-01-1281399, ffprobe git-2020-03-01-1281399
[debug] Proxy map: {}
[AnimeOnDemand] Downloading login page
[AnimeOnDemand] Logging in
[AnimeOnDemand] 179: Downloading webpage
[AnimeOnDemand] episode-1: Downloading ger-Dub videomaterial JSON
[AnimeOnDemand] episode-1: Downloading m3u8 information
[AnimeOnDemand] episode-1: Downloading ger-OmU videomaterial JSON
[AnimeOnDemand] episode-1: Downloading m3u8 information
[AnimeOnDemand] episode-1: Downloading jap-OmU videomaterial JSON
[AnimeOnDemand] episode-1: Downloading m3u8 information
[AnimeOnDemand] episode-1: Downloading jap-Dub videomaterial JSON
[AnimeOnDemand] episode-1: Downloading m3u8 information
[AnimeOnDemand] episode-2: Downloading ger-Dub videomaterial JSON
[AnimeOnDemand] episode-2: Downloading m3u8 information
[AnimeOnDemand] episode-2: Downloading ger-OmU videomaterial JSON
[AnimeOnDemand] episode-2: Downloading m3u8 information
[AnimeOnDemand] episode-2: Downloading jap-OmU videomaterial JSON
[AnimeOnDemand] episode-2: Downloading m3u8 information
[AnimeOnDemand] episode-2: Downloading jap-Dub videomaterial JSON
[AnimeOnDemand] episode-2: Downloading m3u8 information
[download] Downloading playlist: Black Butler: Book of Murder
[AnimeOnDemand] playlist Black Butler: Book of Murder: Collected 2 video ids (downloading 2 of them)
[download] Downloading video 1 of 2
[debug] Invoking downloader on 'https://cloudfront.anime-on-demand.de/2116_0864fff2924f3f669b2b9737c9d86360.smil/chunklist_b4352000.m3u8'
[hlsnative] Downloading m3u8 manifest
ERROR: unable to download video data: HTTP Error 403: Forbidden
Traceback (most recent call last):
  File "c:\python3\lib\site-packages\youtube_dl\YoutubeDL.py", line 1926, in process_info
    success = dl(filename, info_dict)
  File "c:\python3\lib\site-packages\youtube_dl\YoutubeDL.py", line 1865, in dl
    return fd.download(name, info)
  File "c:\python3\lib\site-packages\youtube_dl\downloader\common.py", line 366, in download
    return self.real_download(filename, info_dict)
  File "c:\python3\lib\site-packages\youtube_dl\downloader\hls.py", line 62, in real_download
    urlh = self.ydl.urlopen(self._prepare_url(info_dict, man_url))
  File "c:\python3\lib\site-packages\youtube_dl\YoutubeDL.py", line 2238, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "c:\python3\lib\urllib\request.py", line 531, in open
    response = meth(req, response)
  File "c:\python3\lib\urllib\request.py", line 640, in http_response
    response = self.parent.error(
  File "c:\python3\lib\urllib\request.py", line 569, in error
    return self._call_chain(*args)
  File "c:\python3\lib\urllib\request.py", line 502, in _call_chain
    result = func(*args)
  File "c:\python3\lib\urllib\request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

Description

Error 403 when trying to download the chunklist.m3u8
It appears that youtube-dl isn't passsing the appropiate headers, referer and GET parameters when trying to download the chunk list. Tested with both cookie-jar and username/password login.

Suggestion: add the GET parameters of the playlist URL to the chunklist URL & add the appropiate headers

At the time of opening this issue the site associated with this extractor has a free weekend.

Here is a request comparison between youtube-dl and chrome, copied from devtools:
Broken youtube-dl request:

[debug] Invoking downloader on 'https://cloudfront.anime-on-demand.de/2116_0864fff2924f3f669b2b9737c9d86360.smil/chunklist_b4352000.m3u8'
[hlsnative] Downloading m3u8 manifest

GET /2116_0864fff2924f3f669b2b9737c9d86360.smil/chunklist_b4352000.m3u8 HTTP/1.1
Host: cloudfront.anime-on-demand.de
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3617.0 Safari/537.36
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip, deflate
Accept-Language: en-us,en;q=0.5
Cookie: <cut out>
Connection: close
'

ERROR: unable to download video data: HTTP Error 403: Forbidden

Working Browser request:

curl 'https://cloudfront.anime-on-demand.de/2116_0864fff2924f3f669b2b9737c9d86360.smil/chunklist_b4352000.m3u8?Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly9jbG91ZGZyb250LmFuaW1lLW9uLWRlbWFuZC5kZS8yMTE2XzA4NjRmZmYyOTI0ZjNmNjY5YjJiOTczN2M5ZDg2MzYwLnNtaWwvKiIsIkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTU4NjU4MDE4NX19fV19&Signature=Hv3CEnxjkGcMh0u~grhI5wu5smjz9zjAzHSJsth4DIfVq~RxCe-sUhFIJCcVQX1nfz9WNX1ZlalhA0WwnTn8wuHHptkVRRfsADS0nmFiu~oQ2QqieaO95v-C5MuI82q4EH-3M9w2G0GX9LRcC7Mso4nGtugNp~LBepwueuGTOJzIU7ESxCtXF0NHQyZwW9tpQFxkiZR~MZWdSRUBUDrSW7K4oD5OvDpLrh3WY~KtzEKOot1Negasj6NrRRobjNFv~uJcCPpR~s-DqGO1JvWF7T6EZ-WxTRl9pibMP3hFSZDk5ixmpit1AvuPNdwBKaLwvsFTAIIf2vtXU8ixpFPhyg__&Key-Pair-Id=APKAJE3RLFJLGFELB7WA' \
  -H 'authority: cloudfront.anime-on-demand.de' \
  -H 'user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.92 Safari/537.36' \
  -H 'accept: */*' \
  -H 'origin: https://www.anime-on-demand.de' \
  -H 'sec-fetch-site: same-site' \
  -H 'sec-fetch-mode: cors' \
  -H 'sec-fetch-dest: empty' \
  -H 'referer: https://www.anime-on-demand.de/anime/179' \
  -H 'accept-language: en-US,en;q=0.9,de;q=0.8' \
  --compressed

[Oneboy1979]

Hi! I testet the downloader with the following Syntax:
youtube-dl https://www.anime-on-demand.de/anime/198 --username "PRIVATE" --password "PRIVATE" -f "best[format_id*=ger-Dub][height=720]/best[format_id*=ger-Dub]"

[AnimeOnDemand] Downloading login page
[AnimeOnDemand] Logging in
[AnimeOnDemand] 198: Downloading webpage
[AnimeOnDemand] episode-1: Downloading ger-Dub videomaterial JSON
[AnimeOnDemand] episode-1: Downloading m3u8 information
[AnimeOnDemand] episode-1: Downloading ger-OmU videomaterial JSON
[AnimeOnDemand] episode-1: Downloading m3u8 information
[AnimeOnDemand] episode-1: Downloading jap-OmU videomaterial JSON
[AnimeOnDemand] episode-1: Downloading m3u8 information
[AnimeOnDemand] episode-1: Downloading jap-Dub videomaterial JSON
[AnimeOnDemand] episode-1: Downloading m3u8 information
[download] Downloading playlist: Another
[AnimeOnDemand] playlist Another: Collected 13 video ids (downloading 13 of them)
[download] Downloading video 1 of 13
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 185
[download] Destination: Another, Ep. 1-episode-1.mp4
[download] 100% of 348.39MiB in 02:20
[ffmpeg] Fixing malformed AAC bitstream in "Another, Ep. 1-episode-1.mp4"

And it works as expected! I only used the first file, because i only have a none payed Account.

regards Oneboy1979

[tadev]

Tried It again just now, same params as in my first post. Looks like it was a cloudfront related issue, download now works after the provider allowed users to select a different server location.