[RTBF.be] Extration problem for the "Live" streams

[IkiaeM]

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

• I understand that I will be blocked if I remove or skip any mandatory* field

Checklist

• I'm reporting a broken site
• I've verified that I'm running yt-dlp version 2022.08.08 (update instructions) or later (specify commit)
• I've checked that all provided URLs are playable in a browser with the same IP and same login details
• I've checked that all URLs and arguments with special characters are properly quoted or escaped
• I've searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
• I've read the guidelines for opening an issue
• I've read about sharing account credentials and I'm willing to share it if required

Region

Belgique

Provide a description that is worded well enough to be understood

Extration problem for the "Live" streams found here: https://www.rtbf.be/auvio/direct#/

It seems to be blocked because of Subtitle...

Provide verbose output that clearly demonstrates the problem

• Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
• Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

$ yt-dlp -vU -g --cookies-from-browser firefox https://www.rtbf.be/auvio/direct_le-beau-velo-de-ravel?lid=408218#/
[debug] Command-line config: ['-vU', '-g', '--cookies-from-browser', 'firefox', 'https://www.rtbf.be/auvio/direct_le-beau-velo-de-ravel?lid=408218#/']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.08.08 [3157158] (pip)
[debug] Python 3.9.13 (CPython 64bit) - macOS-10.15.7-x86_64-i386-64bit 
[debug] Checking exe version: ffmpeg -bsfs
[debug] Checking exe version: ffprobe -bsfs
[debug] exe versions: ffmpeg 5.1 (setts), ffprobe 5.1
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[Cookies] Extracting cookies from firefox
[debug] Extracting cookies from: "/Users/xxx/Library/Application Support/Firefox/Profiles/xc8x3893.default-release/cookies.sqlite"
[Cookies] Extracted 888 cookies from firefox
[debug] Proxy map: {}
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.08.08, Current version: 2022.08.08
yt-dlp is up to date (2022.08.08)
[debug] [RTBF] Extracting URL: https://www.rtbf.be/auvio/direct_le-beau-velo-de-ravel?lid=408218#/
[RTBF] 408218: Downloading webpage
ERROR: 408218: An extractor error has occurred. (caused by KeyError('subtitle')); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "/usr/local/Cellar/yt-dlp/2022.8.8/libexec/lib/python3.9/site-packages/yt_dlp/extractor/common.py", line 666, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/Cellar/yt-dlp/2022.8.8/libexec/lib/python3.9/site-packages/yt_dlp/extractor/redbee.py", line 283, in _real_extract
    title = data['subtitle']
KeyError: 'subtitle'

[elyse0]

My bad, I didn't know streams don't have a subtitle. It's a quick fix, we just need to fallback to 'title'

yt-dlp/yt_dlp/extractor/redbee.py

Lines 279 to 285 in e183bb8

	provider = data.get('provider')
	if provider in self._PROVIDERS:
	return self.url_result(data['url'], self._PROVIDERS[provider])
	title = data['subtitle']
	is_live = data.get('isLive')
	height_re = r'-(\d+)p\.'

I was testing it but it only downloads like 20s and then it gets stuck 🤔 https://pastebin.com/dBnQMf2z

@IkiaeM It used to work before? Or it's the first time you're trying it?

[IkiaeM]

This is the first time I've tried it. I discovered the repo while searching because YT-DL doesn't work at all for the site and this discovery seems to work very well for VOD which is great but I was using it mainly for the Live stream. And be careful because it throws fake "m3u8" streams that are fixed and always the same... Because that's what YT-DL does it downloads the fake streams and not the protected DASH stream. But this is the first time I'm trying it and I didn't see any info on the live stream so I took the liberty of opening a ticket. I will test tomorrow to see if I get the same thing as you, thanks for your help

[IkiaeM]

I just looked at your log and indeed it downloads the fake m3u8 stream (https://rtbf-live.fl.freecaster.net/live/rtbf/geo/drm/latrois_aes-audio_vf=128000-video=2000000.m3u8). This is not the real stream. It is a fake one that is fixed and indeed lasts about 20 seconds...

[IkiaeM]

It no longer uses "Freecaster" but a DRM protected DASH stream

[IkiaeM]

Youtube-dl also finds the same stream as you but I just tried, I updated yt-dlp and I still have the same error, here are the logs and it shows that I am up-to-date .... It's quite strange.

$ yt-dlp -g --cookies-from-browser firefox https://www.rtbf.be/auvio/direct_archives-sonuma?lid=408216#/ -vU
[debug] Command-line config: ['-g', '--cookies-from-browser', 'firefox', 'https://www.rtbf.be/auvio/direct_archives-sonuma?lid=408216#/', '-vU']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.08.14 [5593720] (pip)
[debug] Python 3.9.13 (CPython 64bit) - macOS-10.15.7-x86_64-i386-64bit
[debug] Checking exe version: ffmpeg -bsfs
[debug] Checking exe version: ffprobe -bsfs
[debug] exe versions: ffmpeg 5.1 (setts), ffprobe 5.1
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[Cookies] Extracting cookies from firefox
[debug] Extracting cookies from: "/Users/xxx/Library/Application Support/Firefox/Profiles/xc8x3893.default-release/cookies.sqlite"
[Cookies] Extracted 950 cookies from firefox
[debug] Proxy map: {}
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.08.14, Current version: 2022.08.14
yt-dlp is up to date (2022.08.14)
[debug] [RTBF] Extracting URL: https://www.rtbf.be/auvio/direct_archives-sonuma?lid=408216#/
[RTBF] 408216: Downloading webpage
ERROR: 408216: An extractor error has occurred. (caused by KeyError('subtitle')); please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
File "/usr/local/Cellar/yt-dlp/2022.8.14/libexec/lib/python3.9/site-packages/yt_dlp/extractor/common.py", line 666, in extract
ie_result = self._real_extract(url)
File "/usr/local/Cellar/yt-dlp/2022.8.14/libexec/lib/python3.9/site-packages/yt_dlp/extractor/redbee.py", line 283, in _real_extract
title = data['subtitle']
KeyError: 'subtitle'

[elyse0]

I haven't made a pull request with the fix, that's why it still doesn't work jeje

It gets 404 on a mpd manifest, probably related 🤔 It plays in firefox with DRM disabled, so it's not drm protected. Anyway, I'll look into it

[RTBF] 408212: Downloading webpage
[RTBF] 408212: Downloading m3u8 information
WARNING: [RTBF] Ignoring subtitle tracks found in the HLS manifest; if any subtitle tracks are missing, please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
[RTBF] 408212: Downloading MPD manifest
WARNING: [RTBF] Failed to download MPD manifest: HTTP Error 404: Not Found
[info] 408212: Downloading 1 format(s): hls-2320
[download] Destination: JT 19h30 2022-08-14 17_46 [408212].mp4
[hls @ 0x555bedbdaa00] Skip ('#EXT-X-VERSION:4')

[IkiaeM]

Okay, thank you very much. I must admit that DASH and DRM protection are subjects that still escape me a little... If you need, or if you have any questions I would be delighted to help you.

[IkiaeM]

I just tried to disable DRM streaming and it tells me there's an error... And Firefox tells me I have to enable it to play the audio and video stream.

[IkiaeM]

(The site and the live feeds are currently in trouble I think because they don't load properly from what I see)
I think I may have discovered something. I tried with Safari and I looked at what was going through the network and it's not a Dash feed but a M3U8 feed! So there is a way to get it and there is no token as far as I can see.

Here is what I could get: https://c9851ec-rbm-hilv.fsly.cdn.ebsd.ericsson.net/L25/3bd9567b/ee1747fe.isml/.m3u8

[IkiaeM]

The problem is the sound in the stream because the video without sound there is 0 problem it is fluid but as soon as you activate the sound then everything freezes but that is them the problem and not us but I say it anyway for your tests :)

Update : It was my computer the problem, sorry x)

[IkiaeM]

The stream I sent you changes every time the Program changes. So it's temporary... It's very annoying...
old : https://c9851ec-rbm-hilv.fsly.cdn.ebsd.ericsson.net/L25/3bd9567b/ee1747fe.isml/.m3u8
new : https://c9851ec-rbm-hilv.fsly.cdn.ebsd.ericsson.net/L25/3bd9567b/322fadb4.isml/.m3u8

[elyse0]

Thanks!

I've already found a fix using the new RedBee extractor (#4479), I'm just trying to figure out what to do with the 'fake' streams without affecting old videos 🤔

[Cookies] Extracting cookies from firefox
[Cookies] Extracted 2669 cookies from firefox
[RTBF] 408219: Downloading webpage
[RTBF] live_408219: Downloading JSON metadata
[RTBF] live_408219: Downloading JSON metadata
[RTBF] live_408219: Downloading JSON metadata
[RTBF] live_408219: Downloading MPD manifest
[RTBF] live_408219: Downloading m3u8 information
[RTBF] live_408219: Downloading ISM manifest
[info] Available formats for 408219:
ID                                          EXT  RESOLUTION FPS │   TBR PROTO │ VCODEC        VBR ACODEC      ABR ASR MORE INFO
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
audio_fre=64000                             m4a  audio only     │   64k dash  │ audio only        mp4a.40.2   64k 48k [fr] DASH audio, m4a_dash
audio_qaa=64000                             m4a  audio only     │   64k dash  │ audio only        mp4a.40.2   64k 48k [qaa] DASH audio, m4a_dash
audio_qad=64000                             m4a  audio only     │   64k dash  │ audio only        mp4a.40.2   64k 48k [qad] DASH audio, m4a_dash
audio_fre=128000                            m4a  audio only     │  128k dash  │ audio only        mp4a.40.2  128k 48k [fr] DASH audio, m4a_dash
audio_qaa=128000                            m4a  audio only     │  128k dash  │ audio only        mp4a.40.2  128k 48k [qaa] DASH audio, m4a_dash
audio_qad=128000                            m4a  audio only     │  128k dash  │ audio only        mp4a.40.2  128k 48k [qad] DASH audio, m4a_dash
audio_fre=192000                            m4a  audio only     │  192k dash  │ audio only        mp4a.40.2  192k 48k [fr] DASH audio, m4a_dash
audio_qaa=192000                            m4a  audio only     │  192k dash  │ audio only        mp4a.40.2  192k 48k [qaa] DASH audio, m4a_dash
audio_qad=192000                            m4a  audio only     │  192k dash  │ audio only        mp4a.40.2  192k 48k [qad] DASH audio, m4a_dash
audio-aacl-128-Français                     m3u8 audio only     │       m3u8  │ audio only        unknown             [fr] Français
audio-aacl-128-Français__Audio_description_ m3u8 audio only     │       m3u8  │ audio only        unknown             [qad] Français (Audio description)
audio-aacl-128-Version_originale            m3u8 audio only     │       m3u8  │ audio only        unknown             [qaa] Version originale
audio-aacl-192-Français                     m3u8 audio only     │       m3u8  │ audio only        unknown             [fr] Français
audio-aacl-192-Français__Audio_description_ m3u8 audio only     │       m3u8  │ audio only        unknown             [qad] Français (Audio description)
audio-aacl-192-Version_originale            m3u8 audio only     │       m3u8  │ audio only        unknown             [qaa] Version originale
audio-aacl-64-Français                      m3u8 audio only     │       m3u8  │ audio only        unknown             [fr] Français
audio-aacl-64-Français__Audio_description_  m3u8 audio only     │       m3u8  │ audio only        unknown             [qad] Français (Audio description)
audio-aacl-64-Version_originale             m3u8 audio only     │       m3u8  │ audio only        unknown             [qaa] Version originale
video=260000_mode=trik_                     mp4  416x234        │   91k dash  │ avc1.64000D   91k video only          DASH video, mp4_dash
video=260000                                mp4  416x234     25 │  260k dash  │ avc1.64000D  260k video only          DASH video, mp4_dash
341                                         mp4  416x234     25 │  341k m3u8  │ avc1.64000D  341k video only
video=450000_mode=trik_                     mp4  448x252        │  158k dash  │ avc1.640015  158k video only          DASH video, mp4_dash
video=450000                                mp4  448x252     25 │  450k dash  │ avc1.640015  450k video only          DASH video, mp4_dash
542                                         mp4  448x252     25 │  542k m3u8  │ avc1.640015  542k video only
video=800000_mode=trik_                     mp4  576x324        │  280k dash  │ avc1.640015  280k video only          DASH video, mp4_dash
video=800000                                mp4  576x324     25 │  800k dash  │ avc1.640015  800k video only          DASH video, mp4_dash
978                                         mp4  576x324     25 │  978k m3u8  │ avc1.640015  978k video only
video=1400000_mode=trik_                    mp4  704x396        │  490k dash  │ avc1.64001E  490k video only          DASH video, mp4_dash
video=1400000                               mp4  704x396     25 │ 1400k dash  │ avc1.64001E 1400k video only          DASH video, mp4_dash
1614                                        mp4  704x396     25 │ 1614k m3u8  │ avc1.64001E 1614k video only
video=2200000_mode=trik_                    mp4  896x504        │  770k dash  │ avc1.64001F  770k video only          DASH video, mp4_dash
video=2200000                               mp4  896x504     25 │ 2200k dash  │ avc1.64001F 2200k video only          DASH video, mp4_dash
2462                                        mp4  896x504     25 │ 2462k m3u8  │ avc1.64001F 2462k video only
video=3500000_mode=trik_                    mp4  1280x720       │ 1225k dash  │ avc1.64001F 1225k video only          DASH video, mp4_dash
video=3500000                               mp4  1280x720    25 │ 3500k dash  │ avc1.64001F 3500k video only          DASH video, mp4_dash
3904                                        mp4  1280x720    25 │ 3904k m3u8  │ avc1.64001F 3904k video only

[IkiaeM]

Here are the old feeds that don't work anymore and broadcast fake content (the numbers after audio-vf and video are for different quality so they can change but), they don't seem to be on the site anymore except the last one which seems to always have a trace so blocking the last one might be enough:

https://rtbf-live.fl.freecaster.net/live/rtbf/geo/drm/laune_aes-audio_vf=128000-video=2000000.m3u8
https://rtbf-live.fl.freecaster.net/live/rtbf/laune_aes-audio_vf=128000-video=2000000.m3u8

https://rtbf-live.fl.freecaster.net/live/rtbf/geo/drm/ladeux_aes-audio_vf=128000-video=2000000.m3u8
https://rtbf-live.fl.freecaster.net/live/rtbf/ladeux_aes-audio_vf=128000-video=2000000.m3u8

https://rtbf-live.fl.freecaster.net/live/rtbf/geo/drm/latrois-audio_vf=128000-video=2000000.m3u8
https://rtbf-live.fl.freecaster.net/live/rtbf/latrois-audio_vf=128000-video=2000000.m3u8
http://rtbf-live.fl.freecaster.net/live/rtbf/geo/open/2e54586bb99b57a87004a7cafba990a213361120/latrois.m3u8 (the one that always seems to be hanging around the site)

[jfou87]

Hello, i have a similar issue since RTBF has moved form Freecaster to RedBee.
When trying to download a file, using yt-dlp, it returns "ERROR: This video is DRM protected; Try selecting another format with --format or add --check-formats to automatically fallback to the next best format"

Is anyone has a clue or a way to solve it ?
Thank you

[elyse0]

If the video is DRM protected, there is no way to download it u-u