-
-
Notifications
You must be signed in to change notification settings - Fork 5.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Question] How are videos uploaded during a channel playlist download handled? #56
Comments
Yes. Here's a trick to handle this: (My example code is for windows, but you can easily adapt it for unix) :start
set has_new_dl=0
youtube-dl URL --download-archive archivefile --exec "set has_new_dl=1 && echo.Downloaded: {}"`
if ["%has_new_dl%"]==["1"] goto start This will loop until youtube-dl runs once without downloading anything |
Another trick would be to use |
Thanks, let me work on this today before I close things out but super helpful! |
Check out my main file https://github.com/undaunt/youtube-metadata-parse/blob/master/bin/yt-dlc-execute.sh |
I see you are using a file rather than a variable, but that's fine.
You should probably use Edit: Upon testing, it seems touch takes multiple files as args. It is touching both |
I had issues setting the variable between the various bash shells. Touching a file and then checking for its existence cleared that up. Thanks for testing, I just woke up and was going to re-test to confirm. |
Checklist
Question
Just wondering what the behavior is if a channel uploads a video midway through my download process. Eg: I'm downloading all videos from a channel via playlist, then running a second job to grab all remaining non-playlist videos. If a channel uploads a new video to a playlist during the playlist snatching, will it get discovered if that playlist hasn't already been scanned?
What I'm trying to determine is, if my batch file has a lot of content to crawl through, it's possible a video that belongs in a playlist could get missed during a long first run and then inadvertently added to the archive.log during the second, channel based job? I'm going to write a bash script to iterate through both as one single cron job, wondering if to cover my bases I need to trigger the playlist job a second time during the initial load of all backdated content so I don't miss anything?
If a video is added during the initial channel download, I imagine it will just be skipped as I'd be running that job second and the pages of all existing videos on the channel are scanned at the start on /videos based jobs, right?
Thanks!
The text was updated successfully, but these errors were encountered: