[Request] Don't download webpage if video identifier matches in archive file

[mjolnir870]

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2018.09.08. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

• I've verified and I assure that I'm running youtube-dl 2018.09.08

Before submitting an issue make sure you have:

• At least skimmed through the README, most notably the FAQ and BUGS sections
• Searched the bugtracker for similar issues including closed ones
• Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser

What is the purpose of your issue?

• Bug report (encountered problems with youtube-dl)
• Site support request (request for adding support for a new site)
• Feature request (request for a new functionality)
• Question
• Other

---

Description of your issue, suggested solution and other information

When using the --download-archive feature youtube-dl still downloads each video's webpage. After downloading, the video is skipped based off the identifier that youtube-dl already has from the initial link.

Can this process be changed so that youtube-dl will discard the video as soon as the identifier is matched from the URL? Doing it this way will allow downloads to complete much faster since youtube-dl doesn't have to download potentially hundreds of webpages for a large playlist. The bigger return on this is that it also avoids slamming a video site with hundreds of page requests that are eventually discarded.

Current process:

1. youtube-dl has video with identifier ABCD
2. youtube-dl downloads webpage for video ABCD
3. youtube-dl aborts download of video ABCD since it matches in archive file

Requested process:

1. youtube-dl has video with identifier ABCD
2. youtube-dl aborts download of video ABCD since it matches in archive file

[dstftw]

This depends on a particular extractor. It does discard as soon as possible if video id is known beforehand and extractor supports it.

[mjolnir870]

Can this be reopened and then specifically be a request for the pornhub extractor to support this then? The video URL provides the identifier but the pornhub extractor is still downloading the webpage.

[mjolnir870]

Should I open a new issue instead of requesting that this one be reopened?

[dstftw]

Open a new issue with complete explanation regarding concrete extractor.