[VK Video] unknown_video extension while downloading some VK video playlists #5675

iG8R · 2022-11-29T23:45:49Z

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

I understand that I will be blocked if I remove or skip any mandatory* field

Checklist

I'm reporting a broken site
I've verified that I'm running yt-dlp version 2022.11.11 (update instructions) or later (specify commit)
I've checked that all provided URLs are playable in a browser with the same IP and same login details
I've checked that all URLs and arguments with special characters are properly quoted or escaped
I've searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
I've read the guidelines for opening an issue
I've read about sharing account credentials and I'm willing to share it if required

Region

Ukraine, but not limited to.

Provide a description that is worded well enough to be understood

Reason to uncheck sharing account credentials: No account needed. Reproduces without cookies.

URL https://vk.com/video/playlist/-24440848_54694294
yt-dlp creates a file called VIDEONAME.unknown_video, instead of a file called VIDEONAME.mp4

Provide verbose output that clearly demonstrates the problem

Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

[debug] Command-line config: ['-vU', 'https://vk.com/video/playlist/-24440848_54694294']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.11.11 [8b64402] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19043-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 4.1.3, ffprobe 4.1.3
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.09.24, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1723 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.11.11, Current version: 2022.11.11
yt-dlp is up to date (2022.11.11)
[debug] [vk:uservideos] Extracting URL: https://vk.com/video/playlist/-24440848_54694294
[vk:uservideos] -24440848_54694294: Downloading webpage
[download] Downloading playlist: Расколотая битвой синева небес: Целый год
[vk:uservideos] -24440848: Downloading JSON metadata
[vk:uservideos] Playlist Расколотая битвой синева небес: Целый год: Downloading 20 videos of 20
[download] Downloading video 1 of 20
[debug] [vk] Extracting URL: http://vk.com/video-24440848_456240567
[vk] -24440848_456240567: Downloading JSON metadata
[vk] -24440848_456240567: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] -24440848_456240567: Downloading 1 format(s): url1080
[debug] Invoking http downloader on "https://vkvd17.mycdn.me/?expires=1670023931822&srcIp=104.28.192.97&pr=40&srcAg=CHROME&ms=185.226.52.148&type=5&sig=MUJTO_j6BWs&ct=0&urls=45.136.22.143&clientType=13&appId=512000384397&id=3432280689393"
[debug] File locking is not supported. Proceeding without locking
[download] Destination: Расколотая битвой синева небес： Целый год (серия 20) [-24440848_456240567].unknown_video
[download]  42.0% of  683.68MiB at    1.19MiB/s ETA 05:32


[debug] Command-line config: ['-UvF', 'https://vk.com/video/playlist/-24440848_54694294']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.11.11 [8b64402] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19043-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 4.1.3, ffprobe 4.1.3
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.09.24, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1723 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.11.11, Current version: 2022.11.11
yt-dlp is up to date (2022.11.11)
[debug] [vk:uservideos] Extracting URL: https://vk.com/video/playlist/-24440848_54694294
[vk:uservideos] -24440848_54694294: Downloading webpage
[download] Downloading playlist: Расколотая битвой синева небес: Целый год
[vk:uservideos] -24440848: Downloading JSON metadata
[vk:uservideos] Playlist Расколотая битвой синева небес: Целый год: Downloading 20 videos of 20
[download] Downloading video 1 of 20
[debug] [vk] Extracting URL: http://vk.com/video-24440848_456240567
[vk] -24440848_456240567: Downloading JSON metadata
[vk] -24440848_456240567: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[info] Available formats for -24440848_456240567:
ID       EXT           RESOLUTION FPS │   FILESIZE   TBR PROTO │ VCODEC    VBR ACODEC  ABR
──────────────────────────────────────────────────────────────────────────────────────────
hls-200  mp4           256x106     25 │ ~ 31.01MiB  201k m3u8  │ unknown  201k unknown  0k
hls-390  mp4           426x178     25 │ ~ 60.38MiB  390k m3u8  │ unknown  390k unknown  0k
url240   unknown_video 240p           │                  https │ unknown       unknown
hls-790  mp4           640x268     25 │ ~122.32MiB  791k m3u8  │ unknown  791k unknown  0k
hls-1313 mp4           852x356     25 │ ~203.18MiB 1314k m3u8  │ unknown 1314k unknown  0k
url360   unknown_video 360p           │                  https │ unknown       unknown
url480   unknown_video 480p           │                  https │ unknown       unknown
hls-2690 mp4           1280x536    25 │ ~416.19MiB 2691k m3u8  │ unknown 2691k unknown  0k
url720   unknown_video 720p           │                  https │ unknown       unknown
hls-4684 mp4           1920x804    25 │ ~724.46MiB 4684k m3u8  │ unknown 4684k unknown  0k
url1080  unknown_video 1080p          │                  https │ unknown       unknown

The text was updated successfully, but these errors were encountered:

bashonly · 2022-12-01T04:51:15Z

In the extractor, ext is not explicity set for the direct http formats, and the video URLs look like this:

https://vkvd17.mycdn.me/?srcIp=1.2.3.4&pr=40&expires=1670327657930&srcAg=CHROME&fromCache=1&ms=1.2.3.4&type=5&sig=ctT8xnSEW5U&ct=0&urls=1.2.3.4&clientType=13&appId=512000384397&zs=43&id=3432280689393

so yt-dlp doesn't have any idea which file extension to use.

Does anyone know if VK hosts any direct http videos that are not mp4?
If they're all mp4, we could add 'ext': 'mp4' in here:

yt-dlp/yt_dlp/extractor/vk.py

Lines 491 to 499 in 0e96b40

    
           if (format_id.startswith(('url', 'cache')) 
        
                   or format_id in ('extra_data', 'live_mp4', 'postlive_mp4')): 
        
               height = int_or_none(self._search_regex( 
        
                   r'^(?:url|cache)(\d+)', format_id, 'height', default=None)) 
        
               formats.append({ 
        
                   'format_id': format_id, 
        
                   'url': format_url, 
        
                   'height': height, 
        
               })

iG8R · 2022-12-01T11:30:11Z

@bashonly

Does anyone know if VK hosts any direct http videos that are not mp4?

All videos I came across in VK were mp4 but I can't say that VK has only mp4.

iG8R · 2022-12-02T14:57:59Z

IMHO, this issue appears due to the fact yt-dlp selects the last format in the format-list and this in turn occurs since it tries to follow bestvideo*+bestaudio/best.
For instance:
1. video with resulted unknow_video extension.

[debug] Command-line config: ['-vU', 'http://vk.com/video-24440848_456240567']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.11.11 [8b64402] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19043-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 4.1.3, ffprobe 4.1.3
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.09.24, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1723 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.11.11, Current version: 2022.11.11
yt-dlp is up to date (2022.11.11)
[debug] [vk] Extracting URL: http://vk.com/video-24440848_456240567
[vk] -24440848_456240567: Downloading JSON metadata
[vk] -24440848_456240567: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] -24440848_456240567: Downloading 1 format(s): url1080
[debug] Invoking http downloader on "https://vkvd17.mycdn.me/?expires=1670251065151&srcIp=104.28.224.99&pr=40&srcAg=CHROME&ms=185.226.52.148&type=5&sig=RK7RNlZNwyc&ct=0&urls=45.136.22.143&clientType=13&appId=512000384397&id=3432280689393"
[download] Resuming download at byte 29359104
[debug] File locking is not supported. Proceeding without locking
[download] Destination: Расколотая битвой синева небес： Целый год (серия 20) [-24440848_456240567].unknown_video
[download]   5.9% of  683.68MiB at    6.74MiB/s ETA 01:35
ERROR: Interrupted by user

[debug] Command-line config: ['-vUF', 'http://vk.com/video-24440848_456240567']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.11.11 [8b64402] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19043-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 4.1.3, ffprobe 4.1.3
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.09.24, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1723 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.11.11, Current version: 2022.11.11
yt-dlp is up to date (2022.11.11)
[debug] [vk] Extracting URL: http://vk.com/video-24440848_456240567
[vk] -24440848_456240567: Downloading JSON metadata
[vk] -24440848_456240567: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[info] Available formats for -24440848_456240567:
ID       EXT           RESOLUTION FPS │   FILESIZE   TBR PROTO │ VCODEC    VBR ACODEC  ABR
──────────────────────────────────────────────────────────────────────────────────────────
hls-200  mp4           256x106     25 │ ~ 31.01MiB  201k m3u8  │ unknown  201k unknown  0k
hls-390  mp4           426x178     25 │ ~ 60.38MiB  390k m3u8  │ unknown  390k unknown  0k
url240   unknown_video 240p           │                  https │ unknown       unknown
hls-790  mp4           640x268     25 │ ~122.32MiB  791k m3u8  │ unknown  791k unknown  0k
hls-1313 mp4           852x356     25 │ ~203.18MiB 1314k m3u8  │ unknown 1314k unknown  0k
url360   unknown_video 360p           │                  https │ unknown       unknown
url480   unknown_video 480p           │                  https │ unknown       unknown
hls-2690 mp4           1280x536    25 │ ~416.19MiB 2691k m3u8  │ unknown 2691k unknown  0k
url720   unknown_video 720p           │                  https │ unknown       unknown
hls-4684 mp4           1920x804    25 │ ~724.46MiB 4684k m3u8  │ unknown 4684k unknown  0k
url1080  unknown_video 1080p          │                  https │ unknown       unknown

2. video with correct extension (mp4).

[debug] Command-line config: ['-vU', 'http://vk.com/video-213978958_456240301']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.11.11 [8b64402] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19043-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 4.1.3, ffprobe 4.1.3
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.09.24, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1723 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.11.11, Current version: 2022.11.11
yt-dlp is up to date (2022.11.11)
[debug] [vk] Extracting URL: http://vk.com/video-213978958_456240301
[vk] -213978958_456240301: Downloading JSON metadata
[vk] -213978958_456240301: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] -213978958_456240301: Downloading 1 format(s): hls-5472
[debug] Invoking hlsnative downloader on "https://vkvd240.mycdn.me/expires/1670251108698/srcIp/104.28.224.99/pr/40/srcAg/CHROME/ms/45.136.20.153/type/5/sig/F834_6bUHfM/ct/8/urls/45.136.22.146/clientType/13/id/3177770453570/video/"
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 99
[download] Destination: Звёздный домен： Сороковое тысячелетие ⧸ Сорок тысяч лет культивации 11 серия [TV-1] [FHD 1080p] ⧸ Xing Yu Siwan Nian [-213978958_456240301].mp4
[debug] File locking is not supported. Proceeding without locking
[download]   3.0% of ~ 714.38MiB at    4.64MiB/s ETA 02:29 (frag 3/99)
ERROR: Interrupted by user

[debug] Command-line config: ['-vUF', 'http://vk.com/video-213978958_456240301']
[debug] Encodings: locale cp65001, fs utf-8, pref cp65001, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.11.11 [8b64402] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19043-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 4.1.3, ffprobe 4.1.3
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.09.24, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1723 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.11.11, Current version: 2022.11.11
yt-dlp is up to date (2022.11.11)
[debug] [vk] Extracting URL: http://vk.com/video-213978958_456240301
[vk] -213978958_456240301: Downloading JSON metadata
[vk] -213978958_456240301: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[info] Available formats for -213978958_456240301:
ID       EXT           RESOLUTION FPS │   FILESIZE   TBR PROTO │ VCODEC    VBR ACODEC  ABR
──────────────────────────────────────────────────────────────────────────────────────────
hls-195  mp4           256x144     24 │ ~ 23.38MiB  195k m3u8  │ unknown  195k unknown  0k
url240   unknown_video 240p           │                  https │ unknown       unknown
hls-376  mp4           426x240     24 │ ~ 45.16MiB  377k m3u8  │ unknown  377k unknown  0k
url360   unknown_video 360p           │                  https │ unknown       unknown
hls-756  mp4           640x360     24 │ ~ 90.69MiB  757k m3u8  │ unknown  757k unknown  0k
url480   unknown_video 480p           │                  https │ unknown       unknown
hls-1244 mp4           852x480     24 │ ~149.20MiB 1245k m3u8  │ unknown 1245k unknown  0k
url720   unknown_video 720p           │                  https │ unknown       unknown
hls-2613 mp4           1280x720    24 │ ~313.26MiB 2613k m3u8  │ unknown 2613k unknown  0k
url1080  unknown_video 1080p          │                  https │ unknown       unknown
hls-5472 mp4           1920x1080   24 │ ~656.06MiB 5473k m3u8  │ unknown 5473k unknown  0k

Is it possible to check previous format(s) in format-list and use extension from it if the best(last) one is unknown?

aslian · 2023-02-15T12:40:46Z

Vimeo extractor had the same problem #1402.

bazzilic · 2023-09-03T11:46:52Z

Would it make sense for yt-dlp to ffprobe the downloaded video and try to deduce the format from there?

dirkf · 2023-09-03T12:52:43Z

Too late by then: the extension is needed for format selection before downloading.

Maybe a HEAD, or other, request for the video would reveal a MIME type, as mentioned above.

bazzilic · 2023-09-03T15:47:46Z

I thought, the problem is that the downloaded file is video.unknown_video instead of video.mp4. That could be solved after the download by doing ffprobe on the file to determine the correct format, I think.

gamer191 · 2023-09-07T03:09:38Z

Too late by then: the extension is needed for format selection before downloading.

Maybe a HEAD, or other, request for the video would reveal a MIME type, as mentioned above.

making a head request to every single video url just for format selection wouldn't make sense, imo, unless the user passes --check-all-formats

I like the ffprobe idea, although it's not my decision

Or why not just write a proposal for .unknown_video to be standardised as a video format?

now that I think about it, a similar file extension likely already exists (UPDATE: apparently .vid exists, but nothing seems to support it)

dirkf · 2023-09-12T04:54:12Z

making a head request to every single video url just for format selection wouldn't make sense, imo, unless the user passes --check-all-formats

Only when a video/media URL is found that doesn't convey any format information, via determine_ext() or an accompanying MIME type, or similar.

Closes #5675 Authored by: seproDev

iG8R added site-bug Issue with a specific website triage Untriaged issue labels Nov 29, 2022

bashonly removed the triage Untriaged issue label Dec 1, 2022

iG8R mentioned this issue Dec 2, 2022

unknown_video extension while downloading VK video playlist axcore/tartube#472

Closed

seproDev mentioned this issue May 8, 2024

[ie/vk] Improve format extraction #9885

Merged

9 tasks

seproDev closed this as completed in #9885 May 8, 2024

seproDev added a commit that referenced this issue May 8, 2024

[ie/vk] Improve format extraction (#9885)

df5c9e7

Closes #5675 Authored by: seproDev

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[VK Video] unknown_video extension while downloading some VK video playlists #5675

[VK Video] unknown_video extension while downloading some VK video playlists #5675

iG8R commented Nov 29, 2022

bashonly commented Dec 1, 2022

iG8R commented Dec 1, 2022

iG8R commented Dec 2, 2022

aslian commented Feb 15, 2023

bazzilic commented Sep 3, 2023

dirkf commented Sep 3, 2023

bazzilic commented Sep 3, 2023

gamer191 commented Sep 7, 2023 •

edited

dirkf commented Sep 12, 2023

[VK Video] unknown_video extension while downloading some VK video playlists #5675

[VK Video] unknown_video extension while downloading some VK video playlists #5675

Comments

iG8R commented Nov 29, 2022

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

Checklist

Region

Provide a description that is worded well enough to be understood

Provide verbose output that clearly demonstrates the problem

Complete Verbose Output

bashonly commented Dec 1, 2022

iG8R commented Dec 1, 2022

iG8R commented Dec 2, 2022

aslian commented Feb 15, 2023

bazzilic commented Sep 3, 2023

dirkf commented Sep 3, 2023

bazzilic commented Sep 3, 2023

gamer191 commented Sep 7, 2023 • edited

dirkf commented Sep 12, 2023

gamer191 commented Sep 7, 2023 •

edited