[BBC Radio 4] "best" downloads "worst"?

[optimiz]

• I've verified and I assure that I'm running youtube-dl 2017.04.17
• At least skimmed through README and most notably FAQ and BUGS sections
• Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

• Bug report (encountered problems with youtube-dl)

---

Hello, first of all, thanks for this very useful program. When attempting to download the "best" version of a BBC Radio 4 show, youtube-dl ends up downloading the "worst" version instead. Using the following example, which has a 320k stream, specify "-f best", youtube-dl passes the parsed m3u8 to ffmpeg for download, but the only stream downloaded is the 48k stream, the other three streams (including the requested best 320k one) are dropped with a "No longer receiving playlist #" notice. (It is not geoblocked.)

$ youtube-dl -f best http://www.bbc.co.uk/programmes/b08mb1gb

[bbc.co.uk] b08mb1gb: Downloading video page
[bbc.co.uk] b08mb1fh: Downloading media selection XML
[bbc.co.uk] b08mb1fh: Downloading m3u8 information
[bbc.co.uk] b08mb1fh: Downloading m3u8 information
[bbc.co.uk] b08mb1fh: Downloading m3u8 information
[bbc.co.uk] b08mb1fh: Downloading m3u8 information
[bbc.co.uk] b08mb1fh: Downloading m3u8 information
[bbc.co.uk] b08mb1fh: Downloading m3u8 information
[bbc.co.uk] b08mb1fh: Downloading m3u8 information
[bbc.co.uk] b08mb1fh: Downloading m3u8 information
[download] Destination: b08mb1fh.mp4
ffmpeg version 2.8.10 Copyright (c) 2000-2016 the FFmpeg developers
built with gcc 5.3.1 (GCC) 20160406 (Red Hat 5.3.1-6)
Input #0, hls,applehttp, from 'https://bbcfmt-ic-63780400-0c0612-aodhlswwlive.s.loris.llnwd.net/usp/auth/vod/piff_abr_full_audio/87b25b-b08mb1fh/vf_b08mb1fh_bacf2f62-b726-4178-bddc-69df94351500.ism/vf_b08mb1fh_bacf2f62-b726-4178-bddc-69df94351500.m3u8':
Duration: 00:36:30.00, start: 10.000000, bitrate: 0 kb/s
Program 0
Metadata:
variant_bitrate : 51000
Stream #0:0: Audio: aac (HE-AAC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 67 kb/s
Metadata:
variant_bitrate : 51000
Program 1
Metadata:
variant_bitrate : 102000
Stream #0:1: Audio: aac (HE-AAC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 111 kb/s
Metadata:
variant_bitrate : 102000
Program 2
Metadata:
variant_bitrate : 136000
Stream #0:2: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 150 kb/s
Metadata:
variant_bitrate : 136000
Program 3
Metadata:
variant_bitrate : 340000
Stream #0:3: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 322 kb/s
Metadata:
variant_bitrate : 340000
[mp4 @ 0x555bb4e54320] Codec for stream 0 does not use global headers but container format requires global headers
Output #0, mp4, to 'file:b08mb1fh.mp4.part':
Metadata:
encoder : Lavf56.40.101
Stream #0:0: Audio: aac ([64][0][0][0] / 0x0040), 48000 Hz, stereo, 67 kb/s
Metadata:
variant_bitrate : 51000
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
[mp4 @ 0x555bb4e54320] Non-monotonous DTS in output stream 0:0; previous: 14336, current: 0; changing to 14337. This may result in incorrect timestamps in the output file.
[hls,applehttp @ 0x555bb480b640] No longer receiving playlist 1
[hls,applehttp @ 0x555bb480b640] No longer receiving playlist 2
[hls,applehttp @ 0x555bb480b640] No longer receiving playlist 3
size= 13030kB time=00:36:29.48 bitrate= 48.8kbits/s
video:0kB audio:12829kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.569317%
[ffmpeg] Downloaded 13343211 bytes
[download] 100% of 12.73MiB

If we look at the m3u8, we can manually obtain the non-geoblocked 320k stream variant string -audio_eng=320000...

$ curl 'https://bbcfmt-ic-63780400-0c0612-aodhlswwlive.s.loris.llnwd.net/usp/auth/vod/piff_abr_full_audio/87b25b-b08mb1fh/vf_b08mb1fh_bacf2f62-b726-4178-bddc-69df94351500.ism/vf_b08mb1fh_bacf2f62-b726-4178-bddc-69df94351500.m3u8'

#EXTM3U
#EXT-X-VERSION:2
#Created with Unified Streaming Platform(version=1.7.18)
#variants
#EXT-X-STREAM-INF:BANDWIDTH=51000,CODECS="mp4a.40.5"
vf_b08mb1fh_bacf2f62-b726-4178-bddc-69df94351500-audio_eng_1=48000.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=102000,CODECS="mp4a.40.5"
vf_b08mb1fh_bacf2f62-b726-4178-bddc-69df94351500-audio_eng_1=96000.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=136000,CODECS="mp4a.40.2"
vf_b08mb1fh_bacf2f62-b726-4178-bddc-69df94351500-audio_eng=128000.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=340000,CODECS="mp4a.40.2"
vf_b08mb1fh_bacf2f62-b726-4178-bddc-69df94351500-audio_eng=320000.m3u8

Adding the -audio_eng=320000 variant string to the m3u8 download url manually to a ffmpeg command performs the correct 320k download, so it's not a ffmpeg issue...

$ ffmpeg -i 'https://bbcfmt-ic-63780400-0c0612-aodhlswwlive.s.loris.llnwd.net/usp/auth/vod/piff_abr_full_audio/87b25b-b08mb1fh/vf_b08mb1fh_bacf2f62-b726-4178-bddc-69df94351500.ism/vf_b08mb1fh_bacf2f62-b726-4178-bddc-69df94351500-audio_eng=320000.m3u8' -codec copy b08mb1fh.aac

ffmpeg version 2.8.10 Copyright (c) 2000-2016 the FFmpeg developers
built with gcc 5.3.1 (GCC) 20160406 (Red Hat 5.3.1-6)
[https @ 0x55d5c5d4b6e0] No trailing CRLF found in HTTP header.
Input #0, hls,applehttp, from 'https://bbcfmt-ic-63780400-0c0612-aodhlswwlive.s.loris.llnwd.net/usp/auth/vod/piff_abr_full_audio/87b25b-b08mb1fh/vf_b08mb1fh_bacf2f62-b726-4178-bddc-69df94351500.ism/vf_b08mb1fh_bacf2f62-b726-4178-bddc-69df94351500-audio_eng=320000.m3u8':
Duration: 00:36:30.00, start: 10.000000, bitrate: 0 kb/s
Program 0
Metadata:
variant_bitrate : 0
Stream #0:0: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 322 kb/s
Output #0, adts, to 'b08mb1fh.aac':
Metadata:
encoder : Lavf56.40.101
Stream #0:0: Audio: aac ([15][0][0][0] / 0x000F), 48000 Hz, stereo, 322 kb/s
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
[adts @ 0x55d5c5f30740] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 2048 >= 0
[tls @ 0x55d5c5aeeb80] The TLS connection was non-properly terminated.
[https @ 0x55d5c5bc6080] No trailing CRLF found in HTTP header.
size= 86225kB time=00:36:29.39 bitrate= 322.6kbits/s
video:0kB audio:86225kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%

The file size and corresponding audio quality are significantly different...

-rw-rw-r-- 1 user user 13343211 Apr 23 23:45 b08mb1fh.mp4 -- 48k stream (youtube-dl -f best)
-rw-rw-r-- 1 user user 88294290 Apr 23 23:55 b08mb1fh.aac -- 320k stream (direct ffmpeg download)

This happens on every BBC Radio 4 show youtube-dl attempts to download. Can non-geoblocked "best" variants be included in the parsing for BBC Radio streams? Please advise. Thank you.

[0880]

If we look at the m3u8, we can manually obtain the non-geoblocked 320k stream variant string -audio_eng=320000...

Input #0, hls,applehttp, from 'https://bbcfmt-ic-63780400-0c0612-aodhlswwlive.s.loris.llnwd.net/usp/auth/vod/piff_abr_full_audio/87b25b-b08mb1fh/vf_b08mb1fh_bacf2f62-b726-4178-bddc-69df94351500.ism/vf_b08mb1fh_bacf2f62-b726-4178-bddc-69df94351500-audio_eng=320000.m3u8'

Specific format selection works too.

-f "stream-nonuk-audio_streaming_concrete_combined_af_limelight_nonuk_hls_https-340"

youtube-dl "http://www.bbc.co.uk/programmes/b08mb1gb" -f "stream-nonuk-audio_streaming_concrete_combined_af_limelight_nonuk_hls_https-340" -g
https://bbcfmt-ic-02f7f700-0c0612-aodhlswwlive.s.loris.llnwd.net/usp/auth/vod/piff_abr_full_audio/87b25b-b08mb1fh/vf_b08mb1fh_bacf2f62-b726-4178-bddc-69df94351500.ism/vf_b08mb1fh_bacf2f62-b726-4178-bddc-69df94351500-audio_eng=320000.m3u8