[BBC] no video formats found

[basicmaster]

Checklist

• I'm reporting a broken site support
• I've verified that I'm running youtube-dl version 2019.11.28
• I've checked that all provided URLs are alive and playable in a browser
• I've checked that all URLs and arguments with special characters are properly quoted or escaped
• I've searched the bugtracker for similar issues including closed ones

Verbose log

$ youtube-dl -v https://www.bbc.com/news/av/embed/p07xmg48/50670843
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'https://www.bbc.com/news/av/embed/p07xmg48/50670843']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.11.28
[debug] Python version 2.7.12 (CPython) - Linux-4.15.0-72-generic-x86_64-with-Ubuntu-16.04-xenial
[debug] exe versions: avconv 2.8.15, avprobe 2.8.15, ffmpeg N-95296-g1cdb7c5, ffprobe N-95296-g1cdb7c5, rtmpdump 2.4
[debug] Proxy map: {}
[bbc] 50670843: Downloading webpage
[bbc] p07xmg48: Downloading media selection XML
ERROR: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/filme/yt/youtube-dl/youtube_dl/YoutubeDL.py", line 796, in extract_info
    ie_result = ie.extract(url)
  File "/filme/yt/youtube-dl/youtube_dl/extractor/common.py", line 530, in extract
    ie_result = self._real_extract(url)
  File "/filme/yt/youtube-dl/youtube_dl/extractor/bbc.py", line 937, in _real_extract
    self._sort_formats(formats)
  File "/filme/yt/youtube-dl/youtube_dl/extractor/common.py", line 1327, in _sort_formats
    raise ExtractorError('No video formats found')
ExtractorError: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

Description

A download of a BBC video doesn't work. The video is actually part of this page, from which it cannot be downloaded as well. With both links (this one and the embedded one), video playback works flawlessly in a browser, outside of UK (in Germany). Looking at the network traffic, I can also extract the DASH manifest by hand and download it using youtube-dl.

[dirkf]

In this case and some others, YTDL downloads a media selection XML containing only captions.

[BlohoJo]

Bump. Missed this post and reported as new issue.

Here's another URL with the issue... program ID workaround doesn't work either:

https://www.bbc.com/news/av/health-51637561/coronavirus-watch-how-germs-spread
Program ID: p084zgch - https://www.bbc.co.uk/programmes/p084zgch

Tried Jdownloader which also fails. Video Download Helper in Chrome only shows two small 640 byte mp4's. Video plays fine in Chrome browser.

Verbose log

E:\USER\temp_ytdl>youtube-dl -v https://www.bbc.com/news/av/health-51637561/coro
navirus-watch-how-germs-spread
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www.bbc.com/news/av/health-51637561/c
oronavirus-watch-how-germs-spread']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2020.03.01
[debug] Python version 3.4.4 (CPython) - Windows-2008ServerR2-6.1.7601-SP1
[debug] exe versions: ffmpeg 4.1.3, ffprobe N-89595-g40d4b13228, rtmpdump 2.4-20
151223-gfa8646d-OpenSSL_1.0.2n-x86_64-static
[debug] Proxy map: {}
[bbc] coronavirus-watch-how-germs-spread: Downloading webpage
[bbc] p084zgch: Downloading media selection XML
ERROR: No video formats found; please report this issue on https://yt-dl.org/bug
 . Make sure you are using the latest version; type  youtube-dl -U  to update. B
e sure to call youtube-dl with the --verbose flag and include its complete outpu
t.
Traceback (most recent call last):
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpavop71
lg\build\youtube_dl\YoutubeDL.py", line 797, in extract_info
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpavop71
lg\build\youtube_dl\extractor\common.py", line 530, in extract
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpavop71
lg\build\youtube_dl\extractor\bbc.py", line 937, in _real_extract
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpavop71
lg\build\youtube_dl\extractor\common.py", line 1327, in _sort_formats
youtube_dl.utils.ExtractorError: No video formats found; please report this issu
e on https://yt-dl.org/bug . Make sure you are using the latest version; type  y
outube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and
include its complete output.

Using Program ID workaround:

E:\USER\temp_ytdl>youtube-dl -v https://www.bbc.co.uk/programmes/p084zgch
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www.bbc.co.uk/programmes/p084zgch']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2020.03.01
[debug] Python version 3.4.4 (CPython) - Windows-2008ServerR2-6.1.7601-SP1
[debug] exe versions: ffmpeg 4.1.3, ffprobe N-89595-g40d4b13228, rtmpdump 2.4-20
151223-gfa8646d-OpenSSL_1.0.2n-x86_64-static
[debug] Proxy map: {}
[bbc.co.uk] p084zgch: Downloading video page
[bbc.co.uk] p084zgch: Downloading media selection XML
ERROR: No video formats found; please report this issue on https://yt-dl.org/bug
 . Make sure you are using the latest version; type  youtube-dl -U  to update. B
e sure to call youtube-dl with the --verbose flag and include its complete outpu
t.
Traceback (most recent call last):
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpavop71
lg\build\youtube_dl\YoutubeDL.py", line 797, in extract_info
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpavop71
lg\build\youtube_dl\extractor\common.py", line 530, in extract
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpavop71
lg\build\youtube_dl\extractor\bbc.py", line 592, in _real_extract
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpavop71
lg\build\youtube_dl\extractor\common.py", line 1327, in _sort_formats
youtube_dl.utils.ExtractorError: No video formats found; please report this issu
e on https://yt-dl.org/bug . Make sure you are using the latest version; type  y
outube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and
include its complete output.

[BlohoJo]

Workaround (for now): I used Firefox 54 with Flash installed and the site used the Flash Player with Adobe HDS. There is a message that this player will be going away soon so this workaround will stop working soon (probably). I was able to use Video Download Helper 7.3.6 and the Companion App 1.3.0 to download the FLV. Alternatively, HDS Link Detector 0.7.2 also works, along with AdobeHDS.php. However, the resulting FLV file download has all kinds of corrupt and invalid timestamps; this seems to be fixed by running ffmpeg on it and converting it to mp4.

For the non-flash, I even tried Replay Media Catcher which seems to be able to download just about anything and even this didn't work. Only way to get non Flash video on BBC now it seems is a screen capture. :(

[dirkf]

From the programme page https://www.bbc.co.uk/programmes/p084zgch, the playlist appeared to be loaded as https://open.live.bbc.co.uk/mediaselector/6/select/version/2.0/mediaset/pc/vpid/p084zgch/format/json/atk/c85a24e6703a7a5637970e73bd1a5c1383249c88/asn/1/jsfunc/JS_callbacks0. How this URL was created is a non-obvious exercise for the reader.

Catchup programmes (ie that have been broadcast) aren't served in this way and yt-dl handles them fine.

[BlohoJo]

Here's the HDS php link/command:

php AdobeHDS.php --manifest "https://vod-hds-ww-live.bbcfmt.hs.llnwd.net/usp/auth/vod/piff_abr_full_hd/e5a1fe-p084zgch/vf_p084zgch_695dda19-4766-4b5d-9654-6b2978073815.ism/pc_hd_abr_v2_hds_master.f4m?s=1583137477&e=1583180677&h=3a94794813521233d10ecead2a067db8" --delete

[BlohoJo]

From the playlist dirkf posted, I used the DASH link with youtube-dl:

youtube-dl "https://vod-dash-ww-live.akamaized.net/usp/auth/vod/piff_abr_full_hd/e5a1fe-p084zgch/vf_p084zgch_695dda19-4766-4b5d-9654-6b2978073815.ism/pc_hd_abr_v2_dash_master.mpd?__gda__=1583180890_21f38e90ee3f8d22cc12016b4617e940"

 

How this URL was created is a non-obvious exercise for the reader.

It appears, you only need to put the program ID in to the following URL:

https://open.live.bbc.co.uk/mediaselector/6/select/version/2.0/mediaset/pc/vpid/p084zgch/

So for other programs, just change the program ID. 😛

[dirkf]

OK, a winner! The "pc" path component appears to reflect the device that the BBC thinks you're using, and could be "iptv-all" as in the playlist path used by the yt-dl BBC site plugin.

Dear BBC site plugin maintainer, please support this type of programme page with playlist path "/mediaselector/6/select/version/2.0/mediaset/iptv-all/vpid/" in addition to the traditional type with playlist path "/mediaselector/5/select/version/2.0/mediaset/iptv-all/vpid/"

[ajj8]

This was fixed with #23415 (posted 3 months ago now! still not merged)
@dstftw @remitamine

[dirkf]

Alternative resolution is PR #26821 (less than a year and had no memory of having posted in this bug :-()