[extractors/rtvc] Add extractors #6578

elyse0 · 2023-03-20T20:45:06Z

IMPORTANT: PRs without the template will be CLOSED

Description of your pull request and other information

This PR add extractors for RTVC Play and some subdomains needed for embeded players like https://www.senalcolombia.tv/senal-en-vivo

Fixes #6457

SenalColombiaLiveIE

[debug] Command-line config: ['https://www.senalcolombia.tv/senal-en-vivo', '--verbose', '-F']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.03.04 [392389b7d] (source)
[debug] Lazy loading extractors is disabled
[debug] Git HEAD: 7de39a252
[debug] Python 3.10.7 (CPython x86_64 64bit) - Linux-5.19.0-35-generic-x86_64-with-glibc2.36 (OpenSSL 3.0.5 5 Jul 2022, glibc 2.36)
[debug] exe versions: ffmpeg N-108931-g4dda3b1653-20221104 (setts), ffprobe N-108931-g4dda3b1653-20221104, phantomjs broken, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.15.0, certifi-2022.09.24, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1792 extractors
[SenalColombiaLive] Extracting URL: https://www.senalcolombia.tv/senal-en-vivo
[SenalColombiaLive] senal-en-vivo: Downloading webpage
[RTVCKaltura] Extracting URL: https://media.rtvc.gov.co/kalturartvc/indexSC.html
[RTVCKaltura] indexSC: Downloading webpage
[RTVCKaltura] indexSC: Downloading m3u8 information
[RTVCKaltura] indexSC: Downloading JSON metadata
[RTVCKaltura] indexSC: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[info] Available formats for indexSC:
ID     EXT RESOLUTION │   TBR PROTO │ VCODEC    VBR ACODEC  ABR
───────────────────────────────────────────────────────────────
500-0  mp4 unknown    │  500k m3u8  │ unknown  500k unknown  0k
500-1  mp4 unknown    │  500k m3u8  │ unknown  500k unknown  0k
3500-0 mp4 unknown    │ 3500k m3u8  │ unknown 3500k unknown  0k
3500-1 mp4 unknown    │ 3500k m3u8  │ unknown 3500k unknown  0k

RTVCPlay

[debug] Command-line config: ['https://www.rtvcplay.co/peliculas-documentales/llinas-el-cerebro-y-el-universo', '--verbose', '-F']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.03.04 [392389b7d] (source)
[debug] Lazy loading extractors is disabled
[debug] Git HEAD: 7de39a252
[debug] Python 3.10.7 (CPython x86_64 64bit) - Linux-5.19.0-35-generic-x86_64-with-glibc2.36 (OpenSSL 3.0.5 5 Jul 2022, glibc 2.36)
[debug] exe versions: ffmpeg N-108931-g4dda3b1653-20221104 (setts), ffprobe N-108931-g4dda3b1653-20221104, phantomjs broken, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.15.0, certifi-2022.09.24, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Loaded 1792 extractors
[RTVCPlay] Extracting URL: https://www.rtvcplay.co/peliculas-documentales/llinas-el-cerebro-y-el-universo
[RTVCPlay] llinas-el-cerebro-y-el-universo: Downloading webpage
[download] Downloading playlist: Llinás, el cerebro y el universo
[RTVCPlay] Playlist Llinás, el cerebro y el universo: Downloading 3 items of 3
[download] Downloading item 1 of 3
[RTVCPlay] Extracting URL: https://www.rtvcplay.co/peliculas-documentales/llinas-el-cerebro-y-el-universo/llinas-cerebro-universo
[RTVCPlay] llinas-cerebro-universo: Downloading webpage
[RTVCPlay] llinas-cerebro-universo: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[info] Available formats for llinas-cerebro-universo:
ID  EXT RESOLUTION │  TBR PROTO │ VCODEC   VBR ACODEC  ABR
──────────────────────────────────────────────────────────
270 mp4 unknown    │ 270k m3u8  │ unknown 270k unknown  0k
360 mp4 unknown    │ 360k m3u8  │ unknown 360k unknown  0k
720 mp4 unknown    │ 720k m3u8  │ unknown 720k unknown  0k
[download] Downloading item 2 of 3
[RTVCPlay] Extracting URL: https://www.rtvcplay.co/peliculas-documentales/llinas-el-cerebro-y-el-universo/concierto-cerebro
[RTVCPlay] concierto-cerebro: Downloading webpage
[RTVCPlay] concierto-cerebro: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[info] Available formats for concierto-cerebro:
ID  EXT RESOLUTION │  TBR PROTO │ VCODEC   VBR ACODEC  ABR
──────────────────────────────────────────────────────────
270 mp4 unknown    │ 270k m3u8  │ unknown 270k unknown  0k
360 mp4 unknown    │ 360k m3u8  │ unknown 360k unknown  0k
720 mp4 unknown    │ 720k m3u8  │ unknown 720k unknown  0k
[download] Downloading item 3 of 3
[RTVCPlay] Extracting URL: https://www.rtvcplay.co/peliculas-documentales/llinas-el-cerebro-y-el-universo/llinas-audio-descriptivo-lenguaje-senas
[RTVCPlay] llinas-audio-descriptivo-lenguaje-senas: Downloading webpage

Template

Before submitting a pull request make sure you have:

At least skimmed through contributing guidelines including yt-dlp coding conventions
Searched the bugtracker for similar pull requests
Checked the code with flake8 and ran relevant tests

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check all of the following options that apply:

I am the original author of this code and I am willing to release it under Unlicense
I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

Fix or improvement to an extractor (Make sure to add/update tests)
New extractor (Piracy websites will not be accepted)
Core bug fix/improvement
New feature (It is strongly recommended to open an issue first)

yt_dlp/extractor/rtvcplay.py

bashonly · 2023-03-21T18:47:20Z

yt_dlp/extractor/rtvcplay.py

+                if not podcast_episodes:
+                    raise ExtractorError('Could not find asset_id nor program playlist nor podcast episodes')
+
+                return self.playlist_result([self.url_result(episode['file'], **traverse_obj(episode, {


url_transparent=True, right? or else the entry metadata being passed is only for --flat-playlist?

Also, the test URL https://www.rtvcplay.co/series-al-oido/diez-versiones has its playlist result handed off to the generic extractor because the URLs are direct mp3 links. Should we be returning info_dict entries instead of url_results here?

~~Yep we could, I'll fix this~~

Maybe it's better like this because we don't have an id for the entry, at least Generic uses the filename

yt_dlp/extractor/rtvcplay.py

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>

yt_dlp/extractor/rtvcplay.py

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>

denaunistic · 2023-04-23T21:28:24Z

@bashonly This can be merged or changes are still pending?

elyse0 · 2023-04-23T22:09:17Z

@denaunistic In the meantime you could load these extractors (rtvcplay.py, senalcolombia.py) as a plugin

https://github.com/yt-dlp/yt-dlp#plugins

bashonly · 2023-04-23T22:24:31Z

@bashonly This can be merged or changes are still pending?

needs a review from the maintainer-in-chief before it can be merged

yt_dlp/extractor/rtvcplay.py

* Add `RTVCPlay` extractor * Add `RTVCPlayEmbed` extractor * Add `RTVCKaltura` extractor * Add `SenalColombiaLive` extractor Closes yt-dlp#6457 Authored by: elyse0

elyse0 added 10 commits March 11, 2023 02:37

[senalcolombia] Add extractor

2a13bbe

Add RTVC extractor

a19d6c9

Update SenalColombia extractor

9f138f5

Simplify SenalColombiaLive extractor

15f29cf

Remove webpage tests

6290600

Fix linter

0ab4655

Extract program page as playlist

207a2d1

Add support for podcasts

eeaf0b4

Improve podcast support

863fdab

Don't match /embed/ in RTVCPlay extractor

7de39a2

bashonly requested changes Mar 21, 2023

View reviewed changes

bashonly added site-request Request to support a new website pending-fixes PR has had changes requested labels Mar 21, 2023

elyse0 and others added 10 commits March 21, 2023 13:22

Fix base class typo

cd57dca

Access current content on hydration

f7c10c3

Fix base class typo

fae6f6a

Fix possible html error

123b1df

Improve entries

aa380af

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>

Use url_transparent in url_result

aa0287a

Improve metadata extraction

2f8ac26

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>

Improve format extraction

afcc9c0

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>

Fix seasons extraction

acf3447

Fix seasons and episodes extraction

dfb7137

bashonly reviewed Mar 22, 2023

View reviewed changes

yt_dlp/extractor/rtvcplay.py Outdated Show resolved Hide resolved

bashonly reviewed Mar 22, 2023

View reviewed changes

yt_dlp/extractor/rtvcplay.py Outdated Show resolved Hide resolved

elyse0 and others added 2 commits March 21, 2023 21:03

Apply suggested changes

30f0b36

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>

Apply suggested changes

0081a50

Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>

bashonly removed the pending-fixes PR has had changes requested label Mar 22, 2023

bashonly requested a review from pukkandan April 23, 2023 22:22

pukkandan reviewed Apr 24, 2023

View reviewed changes

yt_dlp/extractor/rtvcplay.py Outdated Show resolved Hide resolved

Update yt_dlp/extractor/rtvcplay.py

edc9c72

pukkandan assigned bashonly Apr 24, 2023

bashonly merged commit 9b30cd3 into yt-dlp:master Apr 24, 2023
11 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[extractors/rtvc] Add extractors #6578

[extractors/rtvc] Add extractors #6578

elyse0 commented Mar 20, 2023

bashonly Mar 21, 2023 •

edited

elyse0 Mar 22, 2023 •

edited

denaunistic commented Apr 23, 2023

elyse0 commented Apr 23, 2023

bashonly commented Apr 23, 2023

[extractors/rtvc] Add extractors #6578

[extractors/rtvc] Add extractors #6578

Conversation

elyse0 commented Mar 20, 2023

Description of your pull request and other information

SenalColombiaLiveIE

RTVCPlay

Before submitting a pull request make sure you have:

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check all of the following options that apply:

What is the purpose of your pull request?

bashonly Mar 21, 2023 • edited

Choose a reason for hiding this comment

elyse0 Mar 22, 2023 • edited

Choose a reason for hiding this comment

denaunistic commented Apr 23, 2023

elyse0 commented Apr 23, 2023

bashonly commented Apr 23, 2023

bashonly Mar 21, 2023 •

edited

elyse0 Mar 22, 2023 •

edited