[FoxNews] HTTP Error 403: Forbidden

[john-x]

Please follow the guide below

• You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
• Put an x into all the boxes [ ] relevant to your issue (like this: [x])
• Use the Preview tab to see what your issue will actually look like

---

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2018.10.05. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

• I've verified and I assure that I'm running youtube-dl 2018.10.05

Before submitting an issue make sure you have:

• At least skimmed through the README, most notably the FAQ and BUGS sections
• Searched the bugtracker for similar issues including closed ones
• Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser

What is the purpose of your issue?

• Bug report (encountered problems with youtube-dl)
• Site support request (request for adding support for a new site)
• Feature request (request for a new functionality)
• Question
• Other

---

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

---

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:

Add the -v flag to your command line you run youtube-dl with (youtube-dl -v <your command line>), copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):

[debug] System config: []
[debug] User config: ['-f', '(bestvideo+bestaudio)[ext=webm][height <=? 720]/best[height=720]/(bestvideo+bestaudio)[height <=? 720]/best', '--prefer-free-formats', '--add-metadata', '-c', '--sub-lang', 'enUS', '--prefer-ffmpeg', '--cookies=/home/username/.config/cookies.txt', '--user-agent', 'Mozilla/5.0 (X11; Linux x86_64; rv:52.9) Gecko/20100101 Goanna/4.1 Firefox/52.9 PaleMoon/28.1.0']
[debug] Custom config: []
[debug] Command-line args: ['-s', 'https://video.foxnews.com/v/5614615980001/?#sp=watch-live', '-v']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2018.10.05
[debug] Python version 3.6.6 (CPython) - Linux-4.9.0-8-amd64-x86_64-with-debian-9.5
[debug] exe versions: ffmpeg 3.2.12-1, ffprobe 3.2.12-1, phantomjs 2.1.1, rtmpdump 2.4
[debug] Proxy map: {}
[foxnews] Downloading Akamai AMP feed
[foxnews] 5614615980001: Downloading f4m manifest
WARNING: Unable to download f4m manifest: HTTP Error 403: Forbidden
[foxnews] 5614615980001: Downloading m3u8 information
WARNING: Failed to download m3u8 information: HTTP Error 403: Forbidden
ERROR: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 792, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 507, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/foxnews.py", line 73, in _real_extract
    'http://%s/v/feed/video/%s.js?template=fox' % (host, video_id))
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/amp.py", line 89, in _extract_feed_info
    self._sort_formats(formats)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 1286, in _sort_formats
    raise ExtractorError('No video formats found')
youtube_dl.utils.ExtractorError: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
...
<end of log>

---

If the purpose of this issue is a site support request please provide all kinds of example URLs support for which should be included (replace following example URLs by yours):

• Single video: https://video.foxnews.com/v/5614615980001/?#sp=watch-live

Note that youtube-dl does not support sites dedicated to copyright infringement. In order for site support request to be accepted all provided example URLs should not violate any copyrights.

---

Description of your issue, suggested solution and other information

Streams under this site that require authentication return a HTTP Error 403: Forbidden (even when a Netscape cookie file is used after logging in through the browser).

[john-x]

That seems odd. The plugin is there and I'm able to open other streams of theirs that don't require authentication (like their live radio stream: https://video.foxnews.com/v/5599024308001). As long as I have the right cookies it should work, right?.
Is there another bug report that references this issue? I searched before posting but didn't seem to find one.

If it's any help here's the print traffic from my previous attempt as well:

[foxnews] Downloading Akamai AMP feed
[foxnews] 5614615980001: Downloading f4m manifest
send: b'GET /v/feed/video/5614615980001.js?template=fox HTTP/1.1\r\nHost: video.foxnews.com\r\nCookie: AMCV_17FC406C5357BA6E0A490D4D%40AdobeOrg=2121618341%7CMCIDTS%7C17812%7CMCMID%7C17246021787248651375926418534040882674%7CMCAAMLH-1539092008%7C7%7CMCAAMB-1539495248%7CRKhpRz8krg2tLO6pguXWp5olkAcUniQYPHaMWWgdJ3xzPWQmdj0y%7CMCOPTOUT-1538897648s%7CNONE%7CMCAID%7CNONE; FXN_flk=1; FXN_v_idpp=https%3A%2F%2Fvideo.foxnews.com%2Fv%2F5614615980001; ajs_anonymous_id=%22ee3dfdba-07b0-482b-8b97-2d26c6cd3c2b%22; ajs_group_id=null; ajs_user_id=null; optimizelyEndUserId=oeu1538878511172r0.4282492071524938; s_pers=%20omtr_lv%3D1538890966636%7C1633498966636%3B%20omtr_lv_s%3DLess%2520than%25201%2520day%7C1538892766636%3B%20s_nr%3D1538890966650-Repeat%7C1541482966650%3B; seg_xid=b4cb5020-c818-4271-9126-da04521dc172; seg_xid_fd=sync.fxnetworks.com; seg_xid_ts=1537424468728; _cb=BNTofwDlbdYGC7b74U; _cb_ls=1; _chartbeat2=.1538878544616.1538890454248.1.DxxLA0R_LFXZFElxZVnE8DIqbgF.2; _v__chartbeat3=B-YGO3PAOzqDv3SHD; live_fn=https%3A//video.foxnews.com/v/5614615980001/%3F%23sp%3Dwatch-live\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.9) Gecko/20100101 Goanna/4.1 Firefox/52.9 PaleMoon/28.1.0\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Encoding: gzip, deflate\r\nAccept-Language: en-us,en;q=0.5\r\nConnection: close\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Server header: X-Powered-By header: Access-Control-Allow-Origin header: Content-Encoding header: Content-Length header: Content-Type header: Cache-Control header: Date header: Connection header: Vary WARNING: Unable to download f4m manifest: HTTP Error 403: Forbidden
[foxnews] 5614615980001: Downloading m3u8 information
send: b'GET /z/FNCGO_436151@436151/manifest.f4m?hdcore=3.4.0&plugin=aasp-3.4.0.132.124 HTTP/1.1\r\nHost: fncgohds-lh.akamaihd-staging.net\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.9) Gecko/20100101 Goanna/4.1 Firefox/52.9 PaleMoon/28.1.0\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Encoding: gzip, deflate\r\nAccept-Language: en-us,en;q=0.5\r\nConnection: close\r\n\r\n'
reply: 'HTTP/1.1 403 Forbidden\r\n'
header: Server header: Mime-Version header: Content-Type header: Content-Length header: Expires header: Cache-Control header: Pragma header: Date header: Connection header: Set-Cookie header: X-Akamai-Staging WARNING: Failed to download m3u8 information: HTTP Error 403: Forbidden
ERROR: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
send: b'GET /hls/live/263399/FNCGOHLSv2/master.m3u8 HTTP/1.1\r\nHost: fncgohls-i.akamaihd.net\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.9) Gecko/20100101 Goanna/4.1 Firefox/52.9 PaleMoon/28.1.0\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Encoding: gzip, deflate\r\nAccept-Language: en-us,en;q=0.5\r\nConnection: close\r\n\r\n'
reply: 'HTTP/1.1 403 Forbidden\r\n'
header: Server header: Mime-Version header: Content-Length header: Expires header: Cache-Control header: Pragma header: Date header: Connection header: Access-Control-Allow-Headers header: Access-Control-Expose-Headers header: Access-Control-Allow-Methods header: Access-Control-Allow-Origin header: Content-Type 

[harlanb]

I started seeing this last year some time. Still happening:
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'--prefer-ffmpeg', u'-o', u'%(series)s - s%(season_number)02de%(episode_number)02d - %(title)s.%(ext)s', u'https://www.fox.com/watch/2e9b46c3bec1d71febd4ab82a929a37e/']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2018.10.05
[debug] Python version 2.7.15rc1 (CPython) - Linux-4.15.0-32-generic-x86_64-with-Ubuntu-18.04-bionic
[debug] exe versions: ffmpeg 4.0.2, ffprobe 4.0.2
[debug] Proxy map: {}
[FOX] 2e9b46c3bec1d71febd4ab82a929a37e: Downloading JSON metadata
[FOX] 2e9b46c3bec1d71febd4ab82a929a37e: Downloading webpage
[uplynk:preplay] 751cbef9e7a74795864f510a1459fb6a: Downloading JSON metadata
[uplynk:preplay] 751cbef9e7a74795864f510a1459fb6a: Downloading m3u8 information
[uplynk:preplay] 751cbef9e7a74795864f510a1459fb6a: Downloading JSON metadata
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on u'http://content-auso2.uplynk.com/751cbef9e7a74795864f510a1459fb6a/k.m3u8'
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 636
[download] Destination: The Gifted - s02e04 - outMatched.mp4
[download] 0.2% of ~1.65GiB at Unknown speed ETA Unknown ETAERROR: unable to download video data: HTTP Error 403: Forbidden
Traceback (most recent call last):
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1908, in process_info
success = dl(filename, info_dict)
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1847, in dl
return fd.download(name, info)
File "/usr/local/bin/youtube-dl/youtube_dl/downloader/common.py", line 364, in download
return self.real_download(filename, info_dict)
File "/usr/local/bin/youtube-dl/youtube_dl/downloader/hls.py", line 168, in real_download
self._prepare_url(info_dict, decrypt_info['URI'])).read()
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2211, in urlopen
return self._opener.open(req, timeout=self._socket_timeout)
File "/usr/lib/python2.7/urllib2.py", line 435, in open
response = meth(req, response)
File "/usr/lib/python2.7/urllib2.py", line 548, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python2.7/urllib2.py", line 473, in error
return self._call_chain(*args)
File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
result = func(*args)
File "/usr/lib/python2.7/urllib2.py", line 556, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 403: Forbidden

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'--prefer-ffmpeg', u'-o', u'%(series)s - s%(season_number)02de%(episode_number)02d - %(title)s.%(ext)s', u'https://www.fox.com/watch/a7980139ac6b1bed706dfa6abf8a2d0b/']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2018.10.05
[debug] Python version 2.7.15rc1 (CPython) - Linux-4.15.0-32-generic-x86_64-with-Ubuntu-18.04-bionic
[debug] exe versions: ffmpeg 4.0.2, ffprobe 4.0.2
[debug] Proxy map: {}
[FOX] a7980139ac6b1bed706dfa6abf8a2d0b: Downloading JSON metadata
[FOX] a7980139ac6b1bed706dfa6abf8a2d0b: Downloading webpage
[uplynk:preplay] cc4ee5577e85427bb8696d0e5e83b2fe: Downloading JSON metadata
[uplynk:preplay] cc4ee5577e85427bb8696d0e5e83b2fe: Downloading m3u8 information
[uplynk:preplay] cc4ee5577e85427bb8696d0e5e83b2fe: Downloading JSON metadata
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on u'http://content-ause5.uplynk.com/cc4ee5577e85427bb8696d0e5e83b2fe/k.m3u8'
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 650
[download] Destination: Lethal Weapon - s03e04 - Leo Getz Justice.mp4
[download] 0.2% of ~1.63GiB at Unknown speed ETA Unknown ETAERROR: unable to download video data: HTTP Error 403: Forbidden
Traceback (most recent call last):
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1908, in process_info
success = dl(filename, info_dict)
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1847, in dl
return fd.download(name, info)
File "/usr/local/bin/youtube-dl/youtube_dl/downloader/common.py", line 364, in download
return self.real_download(filename, info_dict)
File "/usr/local/bin/youtube-dl/youtube_dl/downloader/hls.py", line 168, in real_download
self._prepare_url(info_dict, decrypt_info['URI'])).read()
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2211, in urlopen
return self._opener.open(req, timeout=self._socket_timeout)
File "/usr/lib/python2.7/urllib2.py", line 435, in open
response = meth(req, response)
File "/usr/lib/python2.7/urllib2.py", line 548, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python2.7/urllib2.py", line 473, in error
return self._call_chain(*args)
File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
result = func(*args)
File "/usr/lib/python2.7/urllib2.py", line 556, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 403: Forbidden