Add a way to not follow redirects

[Rudloff]

Checklist

• I'm reporting a feature request
• I've verified that I'm running youtube-dl version 2021.12.17
• I've searched the bugtracker for similar feature requests including closed ones

Description

The generic extractor currently follows redirects blindly. This is not an issue when running youtube-dl locally, however when it is used in a web app this can allow an attacker to run a SSRF attack.
Even if the web app does not allow calling youtube-dl on internal URLs, an attacker could still craft an external URL redirecting to an internal resource.
Because of this, apps sending requests to arbitrary external URLs should disable redirects in the HTTP client but youtube-dl provides no easy way to do this.

I see two solutions to this:

• Add a way to disable the generic extractor
• Add a flag that would make this extractor not follow redirects

[dirkf]

• Add a way to disable the generic extractor

Eg, new option --disable-generic-extractor opposite to --force-generic-extractor.

• Add a flag that would make this extractor not follow redirects

• Configure the server to block redirect responses to the yt-dl-based app (using a local proxy or similar).

Or disabling the generic extractor is a simple code change in extractor/extractors.py.

[Rudloff]

It turns out disabling redirects is not enough to prevent SSRF attacks, because an attacker could craft an HTML page containing something like this:

<video><source src="http://localhost/" /></video>

And youtube-dl will automatically make a request to this URL.

IMHO, web apps running youtube-dl should disable the generic extractor, but it seems there is currently no way to do this without patching: #28377