Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upGitHub is where the world builds software
Millions of developers and companies build, ship, and maintain their software on GitHub — the largest and most advanced development platform in the world.
Improvement Suggestion: faster dateafter (and datebefore) parameters #5922
Comments
|
If I'm correct, in case of youtube, user pages do not give accurate upload date. (Only inaccurate descriptions like "2 weeks ago"). As a result, it's necessary to extract all videos and compare one by one. Would you like to provide a concrete example URL? |
|
While the exact upload dates of the videos are not given, the videos on this page are sorted by upload date. As a consequence, youtube-dl could be optimized by checking the upload date of each video (and the other tasks assigned - download, encode, tag...) only until a single video is out of the date range as all the other videos will not make it. It's valid in three cases:
I hope that I was clear enough... An example URL: https://www.youtube.com/user/MrSuicideSheep/videos. Sorry for the inconvenience, I couldn't think of a youtube channel on top of my head, so I wrote SOME_YOUTUBER so you could swap it with whoever you want. |
|
Duplicate of #1502. |
I regularly download all the videos from youtube channels. But I only need the most recent videos, I keep the old ones on a server at home. For that, I use the following line in my bash script:
youtube-dl -c --dateafter $date_after --yes-playlist -o '~/Youtube-DL/%(uploader)s/%(title)s.%(ext)s' -f 141/140 --external-downloader aria2c --external-downloader-args '-s 8' $chaineIn
$chaine, I usually put URLs along the lines of this:https://youtube.com/user/SOME_YOUTUBER/videos.Of course, I specify the last time I downloaded the channel in
$date_after. By the way, don't hesitate to suggest a cleaner way to eliminate the videos I have already downloaded.However, since in these URLs the videos are classed by date by youtube, much time is wasted on checking wether or not a video has been posted recently enough, particularly when a channel has several thousands videos.
Maybe this time could be spared by assuming that as soon as a video is found to be out of date, all the remaining are, too.
On a side note, did I setup aria2c right for establishing 8 simultaneous connections? EDIT: I have CN:1 during the downloads.
If needed, I can provide you with my entire script - 23 lines.