[Fifa] 403: Forbidden

[Munto]

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

• I understand that I will be blocked if I remove or skip any mandatory* field

Checklist

• I'm reporting a broken site
• I've verified that I'm running yt-dlp version 2022.11.11 (update instructions) or later (specify commit)
• I've checked that all provided URLs are playable in a browser with the same IP and same login details
• I've checked that all URLs and arguments with special characters are properly quoted or escaped
• I've searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
• I've read the guidelines for opening an issue
• I've read about sharing account credentials and I'm willing to share it if required

Region

No response

Provide a description that is worded well enough to be understood

I wanted to watch some videos from fifa's website, but I have been rejected!
After some investigations, it seems they added few parameters in the queryString to retrieve the playURL (ad.sid & ad.ppid)
The expected value is available in the session cookie ffsid. I don't know how to access a cookie from an extractor, so I wasn't able to test it.
I am curious to see how you will fix it =)

Provide verbose output that clearly demonstrates the problem

• Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
• Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

[debug] Command-line config: ['-vU', 'https://www.fifa.com/fifaplus/en/watch/6yY38lFnzDNgPLLOfh4V1r']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.11.11 [8b64402]
[debug] Python 3.10.8 (CPython x86_64 64bit) - Linux-5.15.83-1-lts-x86_64-with-glibc2.36 (OpenSSL 3.0.7 1 Nov 2022, glibc 2.36)
[debug] exe versions: ffmpeg 5.1.2 (setts), ffprobe 5.1.2, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.12.0, certifi-2022.09.24, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1723 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.11.11, Current version: 2022.11.11
yt-dlp is up to date (2022.11.11)
[debug] [Fifa] Extracting URL: https://www.fifa.com/fifaplus/en/watch/6yY38lFnzDNgPLLOfh4V1r
[Fifa] 6yY38lFnzDNgPLLOfh4V1r: Downloading webpage
[Fifa] 6yY38lFnzDNgPLLOfh4V1r: Downloading Video Details
[Fifa] 6yY38lFnzDNgPLLOfh4V1r: Downloading Preplay Parameters
[Fifa] 6yY38lFnzDNgPLLOfh4V1r: Downloading Content Data
ERROR: [Fifa] 6yY38lFnzDNgPLLOfh4V1r: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "/usr/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 674, in extract
    ie_result = self._real_extract(url)
  File "/usr/lib/python3.10/site-packages/yt_dlp/extractor/fifa.py", line 67, in _real_extract
    content_data = self._download_json(
  File "/usr/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 1034, in download_content
    res = getattr(self, download_handle.__name__)(url_or_request, video_id, **kwargs)
  File "/usr/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 998, in download_handle
    res = self._download_webpage_handle(
  File "/usr/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 868, in _download_webpage_handle
    urlh = self._request_webpage(url_or_request, video_id, note, errnote, fatal, data=data, headers=headers, query=query, expected_status=expected_status)
  File "/usr/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 825, in _request_webpage
    raise ExtractorError(errmsg, cause=err)

  File "/usr/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 807, in _request_webpage
    return self._downloader.urlopen(self._create_request(url_or_request, data, headers, query))
  File "/usr/lib/python3.10/site-packages/yt_dlp/YoutubeDL.py", line 3692, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python3.10/urllib/request.py", line 525, in open
    response = meth(req, response)
  File "/usr/lib/python3.10/urllib/request.py", line 634, in http_response
    response = self.parent.error(
  File "/usr/lib/python3.10/urllib/request.py", line 563, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.10/urllib/request.py", line 496, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.10/urllib/request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

[dirkf]

Re-porting from the WIP yt-dl back-port (patch against yt-dlp 2022.11.22):

--- old/yt_dlp/extractor/fifa.py
+++ new/yt_dlp/extractor/fifa.py
@@ -17,7 +17,7 @@
             'description': 'md5:f4520d0ee80529c8ba4134a7d692ff8b',
             'ext': 'mp4',
             'categories': ['FIFA Tournaments'],
-            'thumbnail': 'https://digitalhub.fifa.com/transform/fa6f0b3e-a2e9-4cf7-9f32-53c57bcb7360/2006_Final_ITA_FRA',
+            'thumbnail': 'https://digitalhub.fifa.com/transform/135e2656-3a51-407b-8810-6c34bec5b59b/FMR_2006_Italy_France_Final_Hero',
             'duration': 8165,
         },
         'params': {'skip_download': 'm3u8'},
@@ -54,7 +54,7 @@
         webpage = self._download_webpage(url, video_id)
 
         preconnect_link = self._search_regex(
-            r'<link[^>]+rel\s*=\s*"preconnect"[^>]+href\s*=\s*"([^"]+)"', webpage, 'Preconnect Link')
+            r'<link\b[^>]+\brel\s*=\s*"preconnect"[^>]+href\s*=\s*"([^"]+)"', webpage, 'Preconnect Link')
 
         video_details = self._download_json(
             f'{preconnect_link}/sections/videoDetails/{video_id}', video_id, 'Downloading Video Details', fatal=False)
@@ -62,22 +62,12 @@
         preplay_parameters = self._download_json(
             f'{preconnect_link}/videoPlayerData/{video_id}', video_id, 'Downloading Preplay Parameters')['preplayParameters']
 
-        cid = preplay_parameters['contentId']
         content_data = self._download_json(
-            f'https://content.uplynk.com/preplay/{cid}/multiple.json', video_id, 'Downloading Content Data', query={
-                'v': preplay_parameters['preplayAPIVersion'],
-                'tc': preplay_parameters['tokenCheckAlgorithmVersion'],
-                'rn': preplay_parameters['randomNumber'],
-                'exp': preplay_parameters['tokenExpirationDate'],
-                'ct': preplay_parameters['contentType'],
-                'cid': cid,
-                'mbtracks': preplay_parameters['tracksAssetNumber'],
-                'ad': preplay_parameters['adConfiguration'],
-                'ad.preroll': int(preplay_parameters['adPreroll']),
-                'ad.cmsid': preplay_parameters['adCMSSourceId'],
-                'ad.vid': preplay_parameters['adSourceVideoID'],
-                'sig': preplay_parameters['signature'],
-            })
+            # 1. query string is expected to be sent as-is
+            # 2. `sig` must be appended
+            # 3. if absent, the call appears to work but the manifest is bad (404)
+            'https://content.uplynk.com/preplay/{contentId}/multiple.json?{queryStr}&sig={signature}'.format(**preplay_parameters),
+            video_id, 'Downloading Content Data')
 
         formats, subtitles = self._extract_m3u8_formats_and_subtitles(content_data['playURL'], video_id)
 

[Munto]

Tested and approved! It's clearly simpler, nice one.
Should I close the issue?