Deleting a file from HDD, but Tartube redownloads it

[DankMemeGuy]

I deleted a lot of videos from folders directly from the HDD (on purpose). However, to my surprise when Tartube ran next, it had downloaded all those channels again. Is that an intended feature to automatically check if a video was downloaded before in Tartube's database vs what's actually on the HDD?

Would be nice to have the ability to delete the video. I usually upload them to the cloud and delete the local copy. I'd rather not have it keep redownloading them!

[axcore]

youtube-dl doesn't know anything about Tartube's database. It also doesn't know anything about videos you have deleted. It simply downloads videos if they don't already exist on your hard drive.

There are two solutions, choose whichever one you like:

1. Use custom downloads (Edit > System preferences... > Operations > Custom > In custom downloads, download each video independently of its channel or playlist). Then click the Check all button to get a list of videos. Then click Operations > Custom download all to download each video individually.

2. Turn on youtube-dl archive files, Edit > System preferences > youtube-dl > Preferences > Allow youtube-dl to create is own archive file

[DankMemeGuy]

Would #1 still work with the automatic scheduling of Tartube?

#2 was already on and it still does this

[axcore]

Would #1 still work with the automatic scheduling of Tartube?

No, but I will add that functionality.

#2 was already on and it still does this

Perhaps you can test this. Make sure archive files are turned on. Add a channel with few videos. (I use this random channel for testing, but you can use any channel). Download all the videos. Now examine the archive file, ytdl-archive.txt. Do you see this?

youtube --kKtFhEWzk
youtube 0RfIn-LrZ3w
youtube wMrHAyX30TU

Now delete everything in the channel's folder, except the archive file. Now download the channel again. Do you see this?

[DankMemeGuy]

The archive said:

youtube --kKtFhEWzk
youtube 0RfIn-LrZ3w
youtube wMrHAyX30TU

I deleted the files but didn't touch the textfile, and redownloaded:

youtube-dl --newline -i --geo-bypass --geo-bypass-country US --hls-prefer-native -o K:/YouTube/Temporary Videos/Test/%(title)s-%(id)s.%(ext)s --write-description --write-info-json --write-annotations --write-thumbnail --no-check-certificate -f bestvideo[height<=?1440][fps<=?60]+bestaudio/best[height<=?1440] --user-agent Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.89 Safari/537.36 --cookies=K:\YouTube\cookies.txt https://www.youtube.com/channel/UCQqM9nXKbGaYFfl0mh6ShRA/videos
[youtube:channel] UCQqM9nXKbGaYFfl0mh6ShRA: Downloading channel page
[youtube:playlist] UUQqM9nXKbGaYFfl0mh6ShRA: Downloading webpage
[download] Downloading playlist: Uploads from Tallest to Shortest
[youtube:playlist] playlist Uploads from Tallest to Shortest: Downloading 3 videos
[download] Downloading video 1 of 3
[youtube] --kKtFhEWzk: Downloading webpage
[youtube] --kKtFhEWzk: Downloading MPD manifest
[info] Writing video description to: K:/YouTube/Temporary Videos/Test/Tallest To Shortest - Rabbit Hole---kKtFhEWzk.description
[info] Writing video description metadata as JSON to: K:/YouTube/Temporary Videos/Test/Tallest To Shortest - Rabbit Hole---kKtFhEWzk.info.json
[youtube] --kKtFhEWzk: Downloading thumbnail ...
[youtube] --kKtFhEWzk: Writing thumbnail to: K:/YouTube/Temporary Videos/Test/Tallest To Shortest - Rabbit Hole---kKtFhEWzk.webp
[dashsegments] Total fragments: 94
[download] Destination: K:/YouTube/Temporary Videos/Test/Tallest To Shortest - Rabbit Hole---kKtFhEWzk.f248.webm
[download] 100% of 19.26MiB in 00:16
[dashsegments] Total fragments: 49
[download] Destination: K:/YouTube/Temporary Videos/Test/Tallest To Shortest - Rabbit Hole---kKtFhEWzk.f251.webm
[download] 100% of 7.55MiB in 00:05
[ffmpeg] Merging formats into "K:/YouTube/Temporary Videos/Test/Tallest To Shortest - Rabbit Hole---kKtFhEWzk.webm"
Deleting original file K:/YouTube/Temporary Videos/Test/Tallest To Shortest - Rabbit Hole---kKtFhEWzk.f248.webm (pass -k to keep)
Deleting original file K:/YouTube/Temporary Videos/Test/Tallest To Shortest - Rabbit Hole---kKtFhEWzk.f251.webm (pass -k to keep)
[download] Downloading video 2 of 3
[youtube] 0RfIn-LrZ3w: Downloading webpage
[youtube] 0RfIn-LrZ3w: Downloading MPD manifest
[info] Writing video description to: K:/YouTube/Temporary Videos/Test/Tallest to Shortest - Internet Troll-0RfIn-LrZ3w.description
[info] Writing video description metadata as JSON to: K:/YouTube/Temporary Videos/Test/Tallest to Shortest - Internet Troll-0RfIn-LrZ3w.info.json
[youtube] 0RfIn-LrZ3w: Downloading thumbnail ...
[youtube] 0RfIn-LrZ3w: Writing thumbnail to: K:/YouTube/Temporary Videos/Test/Tallest to Shortest - Internet Troll-0RfIn-LrZ3w.webp
[dashsegments] Total fragments: 57
[download] Destination: K:/YouTube/Temporary Videos/Test/Tallest to Shortest - Internet Troll-0RfIn-LrZ3w.f137.mp4
[download] 100% of 108.84MiB in 00:27
[dashsegments] Total fragments: 29
[download] Destination: K:/YouTube/Temporary Videos/Test/Tallest to Shortest - Internet Troll-0RfIn-LrZ3w.f251.webm
[download] 100% of 3.97MiB in 00:03
[ffmpeg] Merging formats into "K:/YouTube/Temporary Videos/Test/Tallest to Shortest - Internet Troll-0RfIn-LrZ3w.mkv"
Deleting original file K:/YouTube/Temporary Videos/Test/Tallest to Shortest - Internet Troll-0RfIn-LrZ3w.f137.mp4 (pass -k to keep)
Deleting original file K:/YouTube/Temporary Videos/Test/Tallest to Shortest - Internet Troll-0RfIn-LrZ3w.f251.webm (pass -k to keep)
[download] Downloading video 3 of 3
[youtube] wMrHAyX30TU: Downloading webpage
[youtube] wMrHAyX30TU: Downloading MPD manifest
[info] Writing video description to: K:/YouTube/Temporary Videos/Test/Tallest to Shortest - Heart Is A Muscle (Work It Out)-wMrHAyX30TU.description
[info] Writing video description metadata as JSON to: K:/YouTube/Temporary Videos/Test/Tallest to Shortest - Heart Is A Muscle (Work It Out)-wMrHAyX30TU.info.json
[youtube] wMrHAyX30TU: Downloading thumbnail ...
[youtube] wMrHAyX30TU: Writing thumbnail to: K:/YouTube/Temporary Videos/Test/Tallest to Shortest - Heart Is A Muscle (Work It Out)-wMrHAyX30TU.webp
[dashsegments] Total fragments: 37
[download] Destination: K:/YouTube/Temporary Videos/Test/Tallest to Shortest - Heart Is A Muscle (Work It Out)-wMrHAyX30TU.f137.mp4
[download] 100% of 67.98MiB in 00:17
[dashsegments] Total fragments: 46
[download] Destination: K:/YouTube/Temporary Videos/Test/Tallest to Shortest - Heart Is A Muscle (Work It Out)-wMrHAyX30TU.f140.m4a
[download] 100% of 3.40MiB in 00:04
[ffmpeg] Merging formats into "K:/YouTube/Temporary Videos/Test/Tallest to Shortest - Heart Is A Muscle (Work It Out)-wMrHAyX30TU.mp4"
Deleting original file K:/YouTube/Temporary Videos/Test/Tallest to Shortest - Heart Is A Muscle (Work It Out)-wMrHAyX30TU.f137.mp4 (pass -k to keep)
Deleting original file K:/YouTube/Temporary Videos/Test/Tallest to Shortest - Heart Is A Muscle (Work It Out)-wMrHAyX30TU.f140.m4a (pass -k to keep)
[download] Finished downloading playlist: Uploads from Tallest to Shortest
WARNING: There are no annotations to write.
WARNING: There are no annotations to write.
WARNING: Requested formats are incompatible for merge and will be merged into mkv.
WARNING: There are no annotations to write.

Update:

I disabled the "'add videos which have been removed from a channel' in the youtube-dl preference panel and got:

youtube-dl --newline -i --geo-bypass --geo-bypass-country US --hls-prefer-native -o K:/YouTube/Temporary Videos/Test/%(title)s-%(id)s.%(ext)s --write-description --write-info-json --write-annotations --write-thumbnail --no-check-certificate -f bestvideo[height<=?1440][fps<=?60]+bestaudio/best[height<=?1440] --user-agent Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.89 Safari/537.36 --cookies=K:\YouTube\cookies.txt --download-archive K:/YouTube/Temporary Videos/Test/ytdl-archive.txt https://www.youtube.com/channel/UCQqM9nXKbGaYFfl0mh6ShRA/videos
[youtube:channel] UCQqM9nXKbGaYFfl0mh6ShRA: Downloading channel page
[youtube:playlist] UUQqM9nXKbGaYFfl0mh6ShRA: Downloading webpage
[download] Downloading playlist: Uploads from Tallest to Shortest
[youtube:playlist] playlist Uploads from Tallest to Shortest: Downloading 3 videos
[download] Downloading video 1 of 3
[download] Tallest To Shortest - Rabbit Hole has already been recorded in archive
[download] Downloading video 2 of 3
[download] Tallest to Shortest - Internet Troll has already been recorded in archive
[download] Downloading video 3 of 3
[download] Tallest to Shortest - Heart Is A Muscle (Work It Out) has already been recorded in archive
[download] Finished downloading playlist: Uploads from Tallest to Shortest

So it would appear that the missing video feature is breaking the archive teztdfile feature?

[axcore]

I tested this, and there are a few weird things going on. For example, if I right-click a channel and download just the channel, the archive file isn't used at all (but should be).

It might be a few days before I can test and fix this properly.

[DankMemeGuy]

No worries! Thank you for all the time you put into this. Really great software!

[axcore]

v2.2.0 fixes the archive file issue. Tartube uses the archive file in the same way, regardless of whether missing file detection is enabled, or not.

v2.2.0 adds the ability to schedule custom downloads. In a future release I will make the scheduled downloads completely customisable (so scheduled downloads will apply to all channels, or just a few channels, and you can add as many scheduled downloads as you like); but I ran out of time during this release.