Support for third-party information extractors

[goggle]

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2017.10.29. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

• I've verified and I assure that I'm running youtube-dl 2017.10.29

Before submitting an issue make sure you have:

• At least skimmed through the README, most notably the FAQ and BUGS sections
• Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

• Bug report (encountered problems with youtube-dl)
• Site support request (request for adding support for a new site)
• Feature request (request for a new functionality)
• Question
• Other

---

Note that youtube-dl does not support sites dedicated to copyright infringement. In order for site support request to be accepted all provided example URLs should not violate any copyrights.

---

Description of your issue, suggested solution and other information

Would it be possible to add support for third-party information extractors to youtube-dl?

youtube-dl will never be able to cover all the wishes for site supports and often it takes a long time to get a new information extractor into the youtube-dl core. Wouldn't it be great, if users could easily add support for third-party information extractors?

I have something like this in my mind: A developer writes an information extractor and publishes it as a git repository. The user who wants to use these third-party information extractors, clones them all into a directory (e.g. to /home/goggle/.youtube-dl/extractors/). Now, if the user wants to use these information extractors, they simply needs to call youtube-dl with an additional option (e.g. youtube-dl --third-party /home/goggle/.youtube-dl/extractors). youtube-dl would make sure, that third-pary information extractors would be prioritized if they cover the same _VALID_URL regex as a core information extractor. Also the error messages for third-party information extractors would need to be adapted, so that failures in third-party information extractors would not land on the official youtube-dl bug tracker.

Would something like that be technically doable? What's the opinion of the main developers on this feature request?

[yan12125]

FYI This was rejected #1185

[goggle]

Thanks for mentioning that. It has been more than 4 years since the previous discussion about this topic, maybe we can initiate it again.

In the previous discussion the head developers were afraid, that allowing third-party extractors would lead to a lot of forks of youtube-dl. I don't see this to be an issue at all, it's rather an issue right now. If I absolutely need an information extractor for my purposes in youtube-dl, I have two options:

1. I write the information extractor, make a pull request, and hope that it well be included in youtube-dl one day.
2. I fork the whole project, write the information extractor and maintain the fork (i.e. regularly merging upstream commits into my fork). Furthermore, I always have to create my local install of youtube-dl from this fork and cannot use my system package manager.

I understand that option 1 is absolutely the preferred way of handling this problem, but sometimes it just doesn't work out. If my pull request does not get accepted, I have no other possibility than to go for option 2. This is not really an enjoyable situation, since I always have to keep my fork of youtube-dl updated with upstream youtube-dl.
There are many reasons, why new information extractors don't get included into youtube-dl:

• The code does not satisfy the upstream youtube-dl coding standards.
• The developer does not respond to the requested changes in the code review process.
• The maintainers of youtube-dl do not have access to the video website, since one needs an account (often a paid account). So the information extractor cannot be tested.
• The maintainers of youtube-dl do not have access to the video website, because of geo restriction.
• Often pull requests simply are forgotten in the huge queue of pull requests. If i remember correctly, the size of the pull request queue has been doubled within a year. Now there are around 250 open pull requests!

Support for third-party information extractors would mean, that pull requests for new extractors could coexist with external plugins, and users of youtube-dl could easily test these new extractors without having to know anything about the git toolchain and installing their own version of youtube-dl from the sources. Wouldn't this be beneficial for everyone?