Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upGitHub is where the world builds software
Millions of developers and companies build, ship, and maintain their software on GitHub — the largest and most advanced development platform in the world.
Feature request: writing video datetime to cache #8188
Comments
|
Why not just use |
|
I do. But download-archive does not record files outside the date limits set. |
|
I have implemented caching of extractor results for video pages on my fork here. Once it builds up a cache of the videos, on subsequent executions it blazes through the list in seconds. However, the cache file grows fairly quickly as it caches a lot more information than just video datetime. Hopefully this is still something that can be useful regarding this issue. |
|
I can see this issue and #858 exists already so I'm going to post here. Please add some ability to cache SOMETHING when youtube-dl goes through all the trouble of downloading DASH manifests. I have quite a few channels now with over 3000 videos so youtube-dl invocations are taking more then one hour to finish. |
|
I still have this problem. Is there a fix? Maybe an alternative to --date-after ? |
There are a couple of youtube channels that I keep up-to-date with youtube-dl. For the most part, it works.
Unfortunately, there are a couple of cases where it's (obnoxiously) slow. In particular, when I want everything of a particular series on a channel (easily accomplished with --match-title) only after the date I switched over to youtube-dl for the channel (again, easily accomplished with --date-after).
It then proceeds to download the entire history of the user (regardless of date), and download the DASH manifest of every single video matching the title before checking the date. Every time. After reading #1502, I understand that dates of user history are sometimes out of order, so the first time doing all that downloading of info files and history is kind of unvoidable.
But given that this is in a script that's run on a regular basis, and (as far as I know) the upload time of a video doesn't change, I don't suppose that you could add an option to cache it? A simple key-value (video id -> datetime) would do.
(Alternatively, a way to specify "I know these videos are in order" would do.)
I saw --cache-dir, but "At the moment, only YouTube player files (for videos with obfuscated signatures) are cached".
I also tried working around it with --write-info-json and --load-info, but could not figure out how to get youtube-dl to load said files when downloading more than a single file.