Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Add a way to register custom (external) extractors. #1185

Closed
ravishi opened this Issue · 12 comments

3 participants

@ravishi

I wonder if this was not requested before.

I think that there should be a way to easily register custom extractors into youtube-dl.

It could be done through cli arguments or (even better) setuptools entry-points.

@yasoob
@phihag
Collaborator

I'm not sure what this would be useful for. Why wouldn't you just publish your extractor? While youtube-dl is licensed under Unlicense (basically Public Domain), I don't see any reason why we should encourage forking, which this would amount to. Of course, you could argue that we're the bottleneck, but new extractors are typically the fastest changes accepted.

@ravishi

@phihag,

It would be useful to allow third-party Python packages to register their own custom extractors.

I think that it would, on the contrary, discourage forking, since people would be able to just plug-in their own extractors.

Also, there are complex and specific extractors, like the ones that require the user to request for API keys and stuff from the service provider. I don't know your policy for extractor inclusion, but seems to me that higly specific or complex extractors are good to avoid.

@phihag
Collaborator

Our policy for extractor inclusion is simple: If it is extracting audio or video, we'll include it. API keys can just go into the configuration if they need to stay secret. I'd like all users of youtube-dl benefit from an extractor.

@ravishi

But how about complex extractors?

I have an extractor that depends on pyAMF and does some pretty nasty stuff with keys, secrets and salts. pyAMF must be compiled and depends on some amf libraries.

That would increase the complexity of youtube-dl (and it's installation process). But I could release it as a separate package and register an entry-point, so it would integrate into youtube-dl.

Anyone interested in this specific extractor could assume the trouble of compiling and setting it up, while the great majority of youtube-dl users wouldn't be bothered.

If it ever gets widely used, or if by some reason the complexity comes to be removed, it can be later merged into youtube-dl either by the original author or by another contributor.

@phihag
Collaborator

@ravishi You're right, that is indeed a sensible reason for having an interface for external extractors. Are you willing to release your code under Unlicense? If so, can you give me a link to a repository?

@ravishi

I don't have the code yet. I have some unreleased tools that download songs from some paid services that i'm willing to integrate into youtube-dl.

Since I know that this could be useful for you I can try to come up with something to show you.

@yasoob

Yes @ravishi , it would be really helpful for us. Every little contribution counts. If you face any difficulty while making it compatible with youtube-dl then feel free to open another issue.

@ravishi

I updated the PR at #1198, could you guys check it out?

@phihag
Collaborator

Reconsidering, I do not think that this is a good idea. We regularly manage to go from suggested extractor to release with the new code within a day, and nearly always do within a week. Therefore, I do not think that turnaround time is a problem.

I fear this change would enable a wild proliferation of extractor code (and extractors-for-pay and the like). We are already seeing wildly grown tutorials which advise users to pass in crazy options. Imagine how much worse the situation would be if users started using locally-patched versions of youtube-dl! Inevitably, while the patches may have been useful at a specific date in a specific situation, mainline youtube-dl may add support in the meantime, or change its interfaces.

Furthermore, I do not want to ease it for developers to write code that will never be integrated into mainline youtube-dl - while we have an extremely lenient license, encouraging people to not contribute to the project is against the very spirit of open source software development.

rdio-dl seems to be the poster child for this PR. Looking at the code, the only reason for not plain integrating it into mainline youtube-dl is the lack of a license. I do not want to enable developers to write plugins for youtube-dl under different (potentially proprietary) licenses.

I apologize for the delay, but I sadly have to close this issue as wontfix. It's easy enough to create a private branch of youtube-dl with a private extractors. The vast majority of extractors would be accepted just fine into youtube-dl, where we can ensure that our users get a consistent experience and where we can update the code if helper functions change.

@phihag phihag closed this
@ravishi
@phihag
Collaborator

Well, if an external library is truly necessary, than we can just make the import in the relevant method (and fail gracefully if the library is not there). We already do that for xattrs, libc extensions for terminal title setting, and probably others.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.