BBC iPlayer downloads failing

[methodphoto]

youtube-dl -f m4a --verbose "https://www.bbc.co.uk/programmes/b08hdpd9"
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-f', u'm4a', u'--verbose', u'https://www.bbc.co.uk/programmes/b08hdpd9']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.11.28
[debug] Python version 2.7.16 (CPython) - Darwin-19.2.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 4.0.2, ffprobe 4.0.2, rtmpdump 2.4
[debug] Proxy map: {}
[bbc.co.uk] b08hdpd9: Downloading video page
[bbc.co.uk] b08hdpd9: Downloading playlist JSON
[bbc.co.uk] b08hdpd9: Downloading legacy playlist XML
ERROR: Unable to download XML: HTTP Error 404: Not Found (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 627, in _request_webpage
return self._downloader.urlopen(url_or_request)
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2237, in urlopen
return self._opener.open(req, timeout=self._socket_timeout)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 435, in open
response = meth(req, response)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 548, in http_response
'http', request, response, code, msg, hdrs)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 473, in error
return self._call_chain(*args)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 407, in _call_chain
result = func(*args)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 556, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)

[Vangelis66]

@methodphoto :
Hi 😄 ; actually, the URL you tried with,
https://www.bbc.co.uk/programmes/b08hdpd9
corresponds to a brand/series PID (b08hdpd9), NOT to an individual radio episode!
AFAIAA, the bbc.co.uk extractor does not support series PIDs (i.e. iPlayer's/Sounds Box Sets, etc.)

Currently, there are five radio episodes of "Following the Martian Invasion" available for grabs:
https://www.bbc.co.uk/programmes/b08hdpd9/episodes/player
but you have to specify an individual episode PID to yt-dl (which, for the time being, has to be harvested manually from that page)

E.g. to fetch the last episode (5), I'd use the following command:

youtube-dl --console-title --hls-prefer-native --hls-use-mpegts -c --no-part "https://www.bbc.co.uk/programmes/b08h0cb0" -o "Following the Martian Invasion, Ep5 - The Martian Overthrow.m4a" --add-metadata --write-thumbnail --embed-thumbnail

[bbc.co.uk] b08h0cb0: Downloading video page
[bbc.co.uk] b08h0cb0: Downloading playlist JSON
[bbc.co.uk] b08h0c90: Downloading media selection XML
[bbc.co.uk] b08h0c90: Downloading MPD manifest
[bbc.co.uk] b08h0c90: Downloading m3u8 information
[bbc.co.uk] b08h0c90: Downloading m3u8 information
[bbc.co.uk] b08h0c90: Downloading MPD manifest
[bbc.co.uk] b08h0c90: Downloading m3u8 information
[bbc.co.uk] b08h0c90: Downloading m3u8 information
[bbc.co.uk] b08h0c90: Downloading MPD manifest
[bbc.co.uk] b08h0c90: Downloading m3u8 information
[bbc.co.uk] b08h0c90: Downloading m3u8 information
[bbc.co.uk] b08h0c90: Downloading MPD manifest
[bbc.co.uk] b08h0c90: Downloading m3u8 information
[bbc.co.uk] b08h0c90: Downloading m3u8 information
[bbc.co.uk] b08h0c90: Downloading thumbnail ...
[bbc.co.uk] b08h0c90: Writing thumbnail to: Following the Martian Invasion, Ep5
- The Martian Overthrow.jpg
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 127
[download] Destination: Following the Martian Invasion, Ep5 - The Martian Overth
row.m4a
[download] 100% of 32.34MiB in 01:11
[ffmpeg] Fixing malformed AAC bitstream in "Following the Martian Invasion, Ep5
- The Martian Overthrow.m4a"
[ffmpeg] Adding metadata to 'Following the Martian Invasion, Ep5 - The Martian O
verthrow.m4a'
[atomicparsley] Adding thumbnail to "Following the Martian Invasion, Ep5 - The M
artian Overthrow.m4a"

... The --embed-thumbnail switch requires atomicparsley binary to be present in your %PATH% (I'm on Windows here...).

If you want to fetch the whole series, then create a .txt file with all five episode URLs and feed that to yt-dl via the -a switch:

    -a, --batch-file FILE            File containing URLs to download ('-' for
                                     stdin), one URL per line. Lines starting
                                     with '#', ';' or ']' are considered as
                                     comments and ignored.

Kind regards

[djcsdy]

I've raised a PR (#23438) that makes it possible to download entire playlists from BBC iPlayer, which would allow you to download this series using the URL https://www.bbc.co.uk/programmes/b08hdpd9/episodes/player

However I raised my PR in December and it hasn't gotten any response. Maybe I did something wrong that has caused it to be ignored by the maintainers.

[dirkf]

The quoted episodes URL https://www.bbc.co.uk/programmes/b08hdpd9/episodes/player is correctly handled by the 2021.02.22 version. So is https://www.bbc.co.uk/programmes/b08hdpd9/episodes (same result).

The original series URL https://www.bbc.co.uk/programmes/b08hdpd9 still fails.

One could detect a series-format page and punt it by appending episodes/player. The best discriminator I could see is page contains a link to page's URL with /episodes appended, as there's no explicit series metadata.

However, the top-level failing page has other media items that might have been wanted.

So the answer is "always append /episodes" if you want a series.

I believe this issue can be closed.