--verify-archive

[ZizzyDizzyMC]

---

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2016.04.19. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

• I've verified and I assure that I'm running youtube-dl 2016.04.19

Before submitting an issue make sure you have:

• At least skimmed through README and most notably FAQ and BUGS sections
• Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

• Bug report (encountered problems with youtube-dl)
• Site support request (request for adding support for a new site)
• Feature request (request for a new functionality)
• Question
• Other

Description of your issue, suggested solution and other information

1. I have a very large archive of YouTube videos that I downloaded by loading a list of channel URLs. I used the --download-archive flag so I did not have to re-download any videos when I went over the archive again to update it.
2. I currently do not have a way to "re-run" the archive so I can use --write-info-json and or --write-descriptions without redownloading everything because I used --download-archive
3. I am proposing a new flag --verify-archive which would allow youtube-dl to check and make sure that it does indeed have all content associated with a video. If and only if the user used the appropriate flags to download such content.
4. Using --verify-archive with --write-info-json and or --write-descriptions while having used --download-archive in the past will check the specified download directory contents for such video to verify that the entire set of files exists.
5. --verify-archive will not affect new video downloads as they happen. --verify-archive should also work with all other flags that specify downloading a file and storing it in directory.
6. I have 74 thousand videos totaling 5TB. It's not economically feasible to re-download them in full again.

[ZizzyDizzyMC]

I found a work around using --skip-download flag. I changed the location of the --download-archive files and added --write-info-json and --write-description flag along with --skip-download and removed --all-subs and --write-all-thumbnails. This allows me to go over the archive and download what I need, and only what I need without any extra stuff that I already have. PERFECT. Bit messy maybe but it does work.

I would consider this request as low priority, though in full it could still be useful to use as a single flag in the future.

[vxbinaca]

Why not use --no-overwrites? Or am I misunderstanding something?

[ZizzyDizzyMC]

I had not saw --no-overwrites under the filesystem options flag list. Though I need to test it now, I still don't think it would have worked without having first changed the location of the --download-archive files.
--download-archive flag makes a list of VideoID's that are skipped upon a second run. Being skipped --no-overwrites probably has no effect at this time. (NEED TO TEST.)

I'll test this theory when my archive finishes downloading the descriptions and json's. Would rather not disturb it at this time because I don't want to get the 429 too many requests error from YouTube.

--verify-archive would in theory force file checking when ran, to make sure that --download-archive's video list contains all possible content. Content that we instructed it to download via other flags. (--write-description,--all-subs,--write-all-thumbnails, ect.)