Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[cleanup] Remove dead extractors #8604

Merged
merged 32 commits into from Nov 26, 2023
Merged

[cleanup] Remove dead extractors #8604

merged 32 commits into from Nov 26, 2023

Conversation

seproDev
Copy link
Collaborator

@seproDev seproDev commented Nov 17, 2023

IMPORTANT: PRs without the template will be CLOSED

Description of your pull request and other information

Cleanup and removal of dead/broken extractors.

Files removed:

  • airmozilla (Now handled by Panopto)
  • atttechchannel (Shut down and moved to YouTube)
  • biqle (Shut down piracy site, redirects to redtube) Closes [NSFW] Broken site BIQLE #4763
  • bitwave (Down for 2 years)
  • breakcom (Original site shutdown 2018)
  • camwithher (Domain dead. .com seems like clone/entirely different backend)
  • carambatv (Shut down 2020)
  • channel9 (Shut down in 2021)
  • chirbit (Dead since January 2023)
  • cinchcast (Dead since 2018)
  • clipsyndicate (Site redirects to some 2-year-old landing page. Looking on IA seems to have been down since 2021)
  • cloudy (Domain expired)
  • daftsex (Shut down piracy site, redirects to redtube)
  • defense (Site now uses YouTube)
  • dotsub (Shut down)
  • echomsk (Shut down)
  • ehow (No longer has videos)
  • elevensports (Purchased by DAZN)
  • ellentube (Moved to YouTube)
  • engadget (Shut down video section of site and moved to YouTube)
  • escapist (Video section from site removed, uses YouTube)
  • esri (Shut down and redirects to YouTube. New site mediaspace.esri.com has different IDs)
  • expotv (Video section gone, now does influencer marketing?)
  • extremetube (Dead. Redirects to YouPorn)
  • fourzerostudio (Shut down)
  • foxgay (Dead)
  • fusion (Dead since 2021)
  • gfycat (Dead)
  • helsinki (Subdomain dead, new site uses different IDs)
  • hitbox (Dead since 2020)
  • howcast (Shut down video section and only uses YouTube embeds in articles, which weren't supported.)
  • howstuffworks (redirect to iHeart radio)
  • keezmovies (Dead. Redirects to YouPorn. Since Tube8 inherits from keezmovies, _extract_info function was moved over.)
  • laola1tv (Main site shut down, EHFTV/ITTF moved to different provider with different IDs.)
  • linuxacademy (Dead)
  • m6 (Was urL_result to 6play before. Site now also redirects)
  • meta (Shut down)
  • metacafe (Shut down)
  • mgoon (Dead since 2020)
  • miomio (Dead since 2020)
  • mnet (Moved to mnetplus.world and TVING. IDs are different)
  • moevideo (Partially dead/rebranded. Potentially pirate hoster)
  • mofosex (Dead. Redirects to YouPorn)
  • mojvideo (Shut down in March)
  • movieclips (Site has shut down and only redirects to YouTube Channel)
  • mwave (Dead)
  • mychannels (Heroku Error for past year)
  • myvi (Dead since August)
  • newstube (dead, redirects to loan comparison site?)
  • normalboots (Site no longer has videos)
  • nosvideo (Dead piracy hoster)
  • ooyala (Sold to Brightcove with service shut down)
  • pandoratv (Shut down) Closes can't download from pandora.tv #6026
  • people (Removed videos section from site)
  • playfm ("on holiday" since 2022)
  • plays (Merged(?) with medal.tv, which has its own extractor)
  • playvid (Shut down in 2020)
  • porncom (Now a content aggregator, linking to other sites)
  • pornez (Dead pirate site)
  • pornhd (Now a content aggregator, linking to other sites) Closes Pornhd.com grabber not working #3232
  • radiobremen (merged in to ARD Mediathek)
  • recurbate (Domain was seized) Closes [Recurbate] HTTP Error 403 / "This video is only available for registered users" #7912
  • rice (mediahub is dead)
  • rtvnh (new site nhnieuws has different api and ids)
  • ruhd (Back soon since 2018)
  • shared (Dead piracy video hoster)
  • spankwire (Dead. Redirects to YouPorn)
  • streamcloud (Dead piracy site)
  • swrmediathek (Now part of ARD Mediathek)
  • techtalks (Moved to YouTube)
  • tinypic (Shut down in 2019)
  • tokentube (Dead since April)
  • trilulilu (Shut down in 2020)
  • tunepk (Shut down)
  • tvnet (Dead)
  • tvnow (Relaunched as RTL+)
  • twentyfourvideo (Dead)
  • unscripted (Dead)
  • veehd (Dead piracy site)
  • vimple (Dead)
  • vodlocker (Dead piracy site)
  • voicerepublic (Dead)
  • vrak (Shut down in October)
  • vrv (Replaced by CrunchyRoll) Closes vrv.co has been absorbed by Crunchyroll #6322
  • vshare (Dead piracy site)
  • vupload (Dead)
  • vyborymos (Dead since 2021)
  • vzaar (Site shut down with api dead. Direct download links still work, but handled by generic)
  • wakanim (Shut down in November) Closes [Wakanim]  #1609
  • watchbox (Dead since 2019)
  • watchindianporn (Dead)
  • willow (Only TV channel now)
  • xbef (Dead)
  • xtube (Shut down)
  • xuite (Shut down)
  • yesjapan (Relaunched as From Zero. Different IDs)
  • yinyuetai (Relaunched on different subdomain. New IDs)
  • ynet (Subdomain dead. Videos on main site have different ID format)

Extractors removed:

Extractors marked broken:

  • AolIE
  • BehindKinkIE (moved to kink.com/blog)
  • BFIPlayerIE (Only supported ooyala based videos)
  • BetIE
  • BYUtvIE
  • Clubic (Uses ancient m6 endpoint)
  • CMTIE
  • DHMIE
  • NRLTVIE (Only supported ooyala based videos)
  • SCTEIE (Subdomain dead. IDs on new site might be the same. Authwalled)
  • SCTECourseIE (Subdomain dead. IDs on new site might be the same. Authwalled)
  • SRMediathekIE (Partially moved to ARD Mediathek. Other parts broken)
  • ToypicsIE (new site no longer on videos subdomain)
  • ToypicsUserIE (new site no longer on videos subdomain)
  • Tube8IE (Modified to no longer inherit from KeezMoviesIE. Not deleted, since site is similar enough to before)
  • VidbitIE (Currently invite only, site has new URL structure)
  • VootIE (Merged with JioCinema)
  • VootSeriesIE (Merged with JioCinema)
Template

Before submitting a pull request make sure you have:

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check all of the following options that apply:

  • I am the original author of this code and I am willing to release it under Unlicense
  • I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

@seproDev seproDev added the docs/meta/cleanup related to docs, code cleanup, templates, devscripts etc label Nov 17, 2023
@bashonly bashonly self-requested a review November 17, 2023 21:16
Copy link
Member

@bashonly bashonly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you checked if there are open issues for any of these sites?

yt_dlp/extractor/wat.py Outdated Show resolved Hide resolved
yt_dlp/extractor/voot.py Outdated Show resolved Hide resolved
yt_dlp/extractor/nintendo.py Outdated Show resolved Hide resolved
yt_dlp/extractor/cmt.py Outdated Show resolved Hide resolved
yt_dlp/extractor/bfi.py Outdated Show resolved Hide resolved
yt_dlp/extractor/nrl.py Outdated Show resolved Hide resolved
seproDev and others added 3 commits November 18, 2023 01:47
Co-authored-by: bashonly <88596187+bashonly@users.noreply.github.com>
Uses ancient m6 endpoint
@seproDev seproDev marked this pull request as draft November 18, 2023 01:43
yt_dlp/extractor/generic.py Show resolved Hide resolved
yt_dlp/extractor/generic.py Show resolved Hide resolved
Domain was seized
@seproDev seproDev marked this pull request as ready for review November 18, 2023 22:16
Site shut down with api dead. Direct download links still work, but handled by generic.
@LoneFox78
Copy link

Another one that should be removed is hitbox, which has been dead since 2020.

Site redirects to some 2 year old landing page. Looking on IA seems to have been down since 2021.
Moved to mnetplus.world and TVING. IDs different.
@seproDev
Copy link
Collaborator Author

seproDev commented Nov 20, 2023

@LoneFox78 Thanks for the hint.
Discovered a mistake in my testing method, where network errors would cause a skip, which meant the test was counted as a pass. Re-ran all tests and found these additional dead sites:

  • clipsyndicate (Site redirects to some 2-year-old landing page. Looking on IA seems to have been down since 2021)
  • fusion (Dead since 2021)
  • hitbox (Dead since 2020)
  • mnet (Moved to mnetplus.world and TVING. IDs are different)
  • watchbox (Dead since 2019)

@bashonly bashonly merged commit 9751a45 into yt-dlp:master Nov 26, 2023
15 checks passed
@seproDev seproDev deleted the dead branch November 26, 2023 04:58
@glensc
Copy link
Contributor

glensc commented Nov 28, 2023

wow. this is some heavy work.

aalsuwaidi pushed a commit to aalsuwaidi/yt-dlp that referenced this pull request Apr 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
docs/meta/cleanup related to docs, code cleanup, templates, devscripts etc
Projects
Archived in project
6 participants