MediaWiki HTML5 generic video support

[NanaMizukiAnother7]

Checklist

• I'm requesting a site-specific feature
• I've verified that I'm running yt-dlp version 2022.06.29 (update instructions) or later (specify commit)
• I've checked that all provided URLs are playable in a browser with the same IP and same login details
• I've searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
• I've read the guidelines for opening an issue
• I've read about sharing account credentials and I'm willing to share it if required

Region

USA

Example URLs

https://closinglogosgroup.miraheze.org/wiki/Supriya_International_(India)

Provide a description that is worded well enough to be understood

Add a generic support for MediaWiki HTML5 videos.

Provide verbose output that clearly demonstrates the problem

• Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
• Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

[debug] Command-line config: ['-s', 'https://closinglogosgroup.miraheze.org/wiki/Supriya_International_(India)', '-v']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8 (No VT), error utf-8 (No VT), screen utf-8 (No VT)
[debug] yt-dlp version 2022.06.29 [9d339c4] (win32_exe)
[debug] Python 3.8.10 (CPython 64bit) - Windows-7-6.1.7601-SP1
[debug] Checking exe version: ffmpeg -bsfs
[debug] Checking exe version: ffprobe -bsfs
[debug] exe versions: ffmpeg 2022-06-22-git-fed07efcde-full_build-www.gyan.dev (setts), ffprobe 2022-06-22-git-fed07efcde-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {}
[debug] [generic] Extracting URL: https://closinglogosgroup.miraheze.org/wiki/Supriya_International_(India)
[generic] Supriya_International_(India): Requesting header
WARNING: [generic] Falling back on generic information extractor.
[generic] Supriya_International_(India): Downloading webpage
[generic] Supriya_International_(India): Extracting information
[debug] Looking for video embeds
[download] Downloading playlist: Supriya International (India)
[generic] Playlist Supriya International (India): Downloading 2 videos of 2
[download] Downloading video 1 of 2
[debug] [youtube] Extracting URL: https://www.youtube-nocookie.com/embed/bWn8KrO2hsw
[youtube] bWn8KrO2hsw: Downloading webpage
[youtube] bWn8KrO2hsw: Downloading android player API JSON
[youtube] bWn8KrO2hsw: Downloading MPD manifest
[youtube] bWn8KrO2hsw: Downloading MPD manifest
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, codec:vp9.2, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), acodec, lang, proto, filesize, fs_approx, tbr, vbr, abr, asr, vext, aext, hasaud, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] bWn8KrO2hsw: Downloading 1 format(s): 248+251
[download] Downloading video 2 of 2
[debug] [youtube] Extracting URL: https://www.youtube-nocookie.com/embed/1NA2mWV8fTk
[youtube] 1NA2mWV8fTk: Downloading webpage
[youtube] 1NA2mWV8fTk: Downloading android player API JSON
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, codec:vp9.2, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), acodec, lang, proto, filesize, fs_approx, tbr, vbr, abr, asr, vext, aext, hasaud, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] 1NA2mWV8fTk: Downloading 1 format(s): 271+251
[download] Finished downloading playlist: Supriya International (India)

[HobbyistDev]

From your log, it's already supported, it just didn't list HTML5 video. It seems the GenericIE didn't extract these tag

<source src="//static.miraheze.org/closinglogosgroupwiki/transcoded/3/33/Supriya_International_logo_%281980%29_%28From_-_Lorry%29.mp4/Supriya_International_logo_%281980%29_%28From_-_Lorry%29.mp4.180p.vp9.webm](view-source:https://static.miraheze.org/closinglogosgroupwiki/transcoded/3/33/Supriya_International_logo_%281980%29_%28From_-_Lorry%29.mp4/Supriya_International_logo_%281980%29_%28From_-_Lorry%29.mp4.180p.vp9.webm)" type="video/webm; codecs=&quot;vp9, opus&quot;" data-title="Low bandwidth VP9 (180P)" data-shorttitle="VP9 180P" data-transcodekey="180p.vp9.webm" data-width="320" data-height="162" data-bandwidth="194136" data-framerate="60" />

<source src="//static.miraheze.org/closinglogosgroupwiki/transcoded/3/33/Supriya_International_logo_%281980%29_%28From_-_Lorry%29.mp4/Supriya_International_logo_%281980%29_%28From_-_Lorry%29.mp4.240p.vp9.webm](view-source:https://static.miraheze.org/closinglogosgroupwiki/transcoded/3/33/Supriya_International_logo_%281980%29_%28From_-_Lorry%29.mp4/Supriya_International_logo_%281980%29_%28From_-_Lorry%29.mp4.240p.vp9.webm)" type="video/webm; codecs=&quot;vp9, opus&quot;" data-title="Small VP9 (240P)" data-shorttitle="VP9 240P" data-transcodekey="240p.vp9.webm" data-width="426" data-height="216" data-bandwidth="212928" data-framerate="60" />

and many more in source src=<video_link>

[Lesmiscore]

blob: is a dummy url that cannot be used for extraction

[dirkf]

In yt-dl, once the YT videos are found, the HTML5 extraction is skipped. If the YT extraction is disabled, the 3 HTML5 videos are found.

The yt-dlp generic code is similar.

The question is whether the Generic extractor should exhaustively test the page until all supported media is found, or just the first type as now, or something between the two. Maybe just --generic-extractor-extract-all, or a similar yt-dlp extractor option?

[pukkandan]

@dirkf See #4307

For the time being I went with "something between the two"