akamaized.net URLs result in 403, but they work in Postman

[raterus]

Checklist

• I'm reporting a broken site support
• I've verified that I'm running youtube-dl version 2019.08.02
• I've checked that all provided URLs are alive and playable in a browser
• I've checked that all URLs and arguments with special characters are properly quoted or escaped
• I've searched the bugtracker for similar issues including closed ones

Verbose log

C:\Video>youtube-dl -v --user-agent "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36" "https://video-rightnow.akamaized.net/33/351333/prod-09/hls/351333.m3u8?hdnts=st=1565029438~exp=1565029798~acl=/*~id=2baf06bd-599b-4b06-9dda-f1bd69decbf6~hmac=81482c114aa1968a6b401cae502083a0585ba98bee4bdc6fc5a08e94d470a365"
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', '--user-agent', 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36', 'https://video-rightnow.akamaized.net/33/351333/prod-09/hls/351333.m3u8?hdnts=st=1565029438~exp=1565029798~acl=/*~id=2baf06bd-599b-4b06-9dda-f1bd69decbf6~hmac=81482c114aa1968a6b401cae502083a0585ba98bee4bdc6fc5a08e94d470a365']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2019.08.02
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.15063
[debug] exe versions: none
[debug] Proxy map: {}
[generic] *~id=2baf06bd-599b-4b06-9dda-f1bd69decbf6~hmac=81482c114aa1968a6b401cae502083a0585ba98bee4bdc6fc5a08e94d470a365: Requesting header
WARNING: Could not send HEAD request to https://video-rightnow.akamaized.net/33/351333/prod-09/hls/351333.m3u8?hdnts=st=1565029438~exp=1565029798~acl=/*~id=2baf06bd-599b-4b06-9dda-f1bd69decbf6~hmac=81482c114aa1968a6b401cae502083a0585ba98bee4bdc6fc5a08e94d470a365: HTTP Error 403: Forbidden
[generic] *~id=2baf06bd-599b-4b06-9dda-f1bd69decbf6~hmac=81482c114aa1968a6b401cae502083a0585ba98bee4bdc6fc5a08e94d470a365: Downloading webpage
ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpjwvl_v2x\build\youtube_dl\extractor\common.py", line 627, in _request_webpage
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpjwvl_v2x\build\youtube_dl\YoutubeDL.py", line 2227, in urlopen
  File "C:\Python\Python34\lib\urllib\request.py", line 470, in open
  File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response
  File "C:\Python\Python34\lib\urllib\request.py", line 508, in error
  File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
  File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default

Description

My Command:

youtube-dl -v --user-agent "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36" "https://video-rightnow.akamaized.net/33/351333/prod-09/hls/351333.m3u8?hdnts=st=1565029438~exp=1565029798~acl=/*~id=2baf06bd-599b-4b06-9dda-f1bd69decbf6~hmac=81482c114aa1968a6b401cae502083a0585ba98bee4bdc6fc5a08e94d470a365"

I'm having a 403 issue downloading from akamaized.net. I pulled this request into postman and determined that the User-Agent header is required, and have provided a chrome user agent. I receive 200 responses in Postman, however receive 403 responses from youtube-dl.

Here is the code snippet from Postman showing the working HTTP request. This is the complete command I'm using:

I've tried adding these other headers Postman automatically addes to the request in youtube-dl, but these don't work either:

GET /33/351333/prod-09/hls/351333_360p_800kbs.m3u8?hdntl=exp=1565115903~acl=/*~id=2baf06bd-599b-4b06-9dda-f1bd69decbf6~data=hdntl~hmac=01722be7ff07c473c729f40a741601dd3a11eb15f116ab8b49d6681ec1b0b475& aka_me_session_id=AAAAAAAAAAD%2fxUldAAAAALTQN1JSv0BABnYGpaWuAUsXpDsg1+lbSIKu99K6f0ha2kSMOSZx9P8%2f7jqelxm%2f575MJpnQCCHR HTTP/1.1
Host: video-rightnow.akamaized.net
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36
Accept: */*
Cache-Control: no-cache
Postman-Token: b8fcf5b2-159f-4867-bd39-dfe097038278,49c34f13-9f80-4b48-a42b-7cd42456c41b
Host: video-rightnow.akamaized.net
Accept-Encoding: gzip, deflate
cache-control: no-cache

Any help troubleshooting this issue would be most appreciated. I also tried to download using FFMPEG, and received the same 403 errors despite adding the User-Agent header.

Thanks!

[dstftw]

You must provide exactly the same environment as browser does including headers, cookies and so on and mimic all other related behavior.

[raterus]

I don't see how this is user error, since I can get this to work fine in Postman. It appears to me that youtube-dl is mangling the url I pass it from this log entry and not sending to the webserver what I've specified.

[generic] *~id=2baf06bd-599b-4b06-9dda-f1bd69decbf6~hmac=81482c114aa1968a6b401cae502083a0585ba98bee4bdc6fc5a08e94d470a365: Requesting header

[raterus]

Here is the command that finally ended up working for me

.\youtube-dl -v --ffmpeg-location c:\video\ffmpeg\bin
--prefer-ffmpeg --external-downloader ffmpeg
-f 'bestvideo[height<=480]+bestaudio/best[height<=480]' -o "%(title)s.%(ext)s"
--restrict-filenames --user-agent "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36"
--add-header "User-Agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36" --add-header Referer:https://www.rightnowmedia.org/Content/KidsSeries/351326?episode=1
--add-header Origin:https://www.rightnowmedia.org "https://video-rightnow.akamaized.net/46/352046/prod-09/hls/352046.m3u8?hdnts=st=1565097632~exp=1565097992~acl=/*~id=0a295a69-60df-429d-afc6-ee70a2e1e44d~hmac=0defb05db33e637203f2b6ab7a6b2b865ad7c38e9e28554c58b456917369e5d4"