Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

add funker530 extractor #7291

Merged
merged 13 commits into from Jun 13, 2023
Merged

add funker530 extractor #7291

merged 13 commits into from Jun 13, 2023

Conversation

Cyberes
Copy link
Contributor

@Cyberes Cyberes commented Jun 12, 2023

IMPORTANT: PRs without the template will be CLOSED

Description of your pull request and other information

Added extractor for funker530.com. Extracts thumbnail, description, and metadata. Passes tests.

Template

Before submitting a pull request make sure you have:

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check all of the following options that apply:

  • I am the original author of this code and I am willing to release it under Unlicense
  • I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

Copilot Summary

馃 Generated by Copilot at 1177d8e

Summary

馃帴馃啎馃悵

Add support for Funker530 video extraction. Create a new funker530.py module with a Funker530IE class that handles the extraction logic and imports it in _extractors.py.

Funker530IE
Extracts combat videos
From Rumble platform

Walkthrough

  • Add support for extracting videos from Funker530 website (link, link)
    • Import Funker530IE class from funker530.py module in _extractors.py (link)
    • Define Funker530IE class in funker530.py module (link)

@Cyberes
Copy link
Contributor Author

Cyberes commented Jun 12, 2023

Ok, we're good to go!

@Cyberes
Copy link
Contributor Author

Cyberes commented Jun 12, 2023

Funker530 is a little weird because YouTube videos can be embedded. For example, https://funker530.com/video/theres-far-more-than-meets-the-eye-territory-shifts/
This extractor will fail in this situation. Anyone have any suggestions on how to fix it?

@bashonly
Copy link
Member

bashonly commented Jun 12, 2023

So it looks like the site either embeds Rumble videos or embeds Youtube videos?

Generic (YoutubeIE._extract_embed_urls()) can detect the youtube embeds, and If we tweak RumbleEmbedIE._extract_embed_urls(), the Rumble embeds are detected without a dedicated funker530 extractor:

diff --git a/yt_dlp/extractor/rumble.py b/yt_dlp/extractor/rumble.py
index 98f660f8b..82f3f0f8c 100644
--- a/yt_dlp/extractor/rumble.py
+++ b/yt_dlp/extractor/rumble.py
@@ -144,7 +144,7 @@ def _extract_embed_urls(cls, url, webpage):
         if embeds:
             return embeds
         return [f'https://rumble.com/embed/{mobj.group("id")}' for mobj in re.finditer(
-            r'<script>\s*Rumble\(\s*"play"\s*,\s*{\s*[\'"]video[\'"]\s*:\s*[\'"](?P<id>[0-9a-z]+)[\'"]', webpage)]
+            r'<script>[^<]*\bRumble\(\s*"play"\s*,\s*{\s*[\'"]?video[\'"]?\s*:\s*[\'"](?P<id>[0-9a-z]+)[\'"]', webpage)]
 
     def _real_extract(self, url):
         video_id = self._match_id(url)

So there's no need to reinvent the RumbleEmbed extractor inside the funker530 extractor.

However, If there is valuable metadata in the funker530 webpage that is missing from Rumble or Youtube (e.g. description), we could still use this extractor to get that metadata and return a url_transparent result.

P.S. please review this again
https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#yt-dlp-coding-conventions

@bashonly bashonly added site-request Request to support a new website pending-fixes PR has had changes requested labels Jun 12, 2023
@Cyberes
Copy link
Contributor Author

Cyberes commented Jun 12, 2023

Will do.

@Cyberes
Copy link
Contributor Author

Cyberes commented Jun 12, 2023

However, If there is valuable metadata in the funker530 webpage that is missing from Rumble or Youtube (e.g. description), we could still use this extractor to get that metadata and return a url_transparent result.

I just called the individual Rumble and YouTube extractors and return their output rather than returning a url_transparent. Looking at examples, I'm not sure how url_transparent is different from what an extractor normally returns.

@bashonly
Copy link
Member

Looking at examples, I'm not sure how url_transparent is different

If you return only a url_result, any extra metadata you return with it will be discarded by the next extractor. Returning an info_dict with '_type': 'url_transparent' or a url_result() with url_transparent=True will give your extractor's metadata priority & it will be returned in the final info_dict even if the next extractor extracts those same fields.

yt_dlp/extractor/funker530.py Outdated Show resolved Hide resolved
yt_dlp/extractor/funker530.py Outdated Show resolved Hide resolved
yt_dlp/extractor/funker530.py Outdated Show resolved Hide resolved
yt_dlp/extractor/funker530.py Outdated Show resolved Hide resolved
yt_dlp/extractor/funker530.py Outdated Show resolved Hide resolved
Copy link
Contributor Author

@Cyberes Cyberes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Committed changes

yt_dlp/extractor/funker530.py Outdated Show resolved Hide resolved
yt_dlp/extractor/funker530.py Outdated Show resolved Hide resolved
yt_dlp/extractor/funker530.py Outdated Show resolved Hide resolved
yt_dlp/extractor/funker530.py Outdated Show resolved Hide resolved
@bashonly bashonly removed the pending-fixes PR has had changes requested label Jun 12, 2023
yt_dlp/extractor/funker530.py Outdated Show resolved Hide resolved
yt_dlp/extractor/funker530.py Outdated Show resolved Hide resolved
yt_dlp/extractor/funker530.py Outdated Show resolved Hide resolved
yt_dlp/extractor/funker530.py Outdated Show resolved Hide resolved
Cyberes and others added 2 commits June 12, 2023 20:49
Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
Co-authored-by: pukkandan <pukkandan.ytdlp@gmail.com>
@Cyberes Cyberes requested a review from pukkandan June 13, 2023 02:55
@bashonly bashonly merged commit cab94a0 into yt-dlp:master Jun 13, 2023
11 checks passed
aalsuwaidi pushed a commit to aalsuwaidi/yt-dlp that referenced this pull request Apr 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
site-request Request to support a new website
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants