Download filter from text file

[shakeyourbunny]

Currently, youtube-dl has a great capability for filtering out unwanted videos from the command line with --reject-title and --match-title.

It would be great to have a possibility to filter unwanted stuff from a text file, like --reject-title-from=filename.txt or/and --match-title-from=filename.txt

[phihag]

I'm not sure what's the use case for this feature. How many filters are you passing in at the moment, and why?

[shakeyourbunny]

The filter depend on the (youtube) channels I'm archiving ... some have only 2 or 5, but there are also some extreme cases of about 50 on big youtube channels like gamestarde or inecomcompany where they don't have proper playlists to skip unwanted stuff.

It's not really an issue but I'm dreading that putting them (programmatically) as command line arguments creates more problems.

[phihag]

Sorry, I'm still not quite sure why title matching or rejecting is useful here in the first place. Can you elaborate on the archiving process you are running, and/or give an example of the patterns you use?

The usecase for these features that I'd always assumed is wanting to, say, download all videos of a game channel except for the ones that feature "Call of Duty", or only the ones that feature "GTA". It sounds like you're using these options for a totally different purpose.

[shakeyourbunny]

Well, I regularily do scan a bunch of Youtube channels in order to grab the videos once a day. There are some channels which have the habit of trimming off their oldest ones (eg youdagames) or if the marketing campaign is over they remove then or close off the channel. Latter example would be the youtube channel "dragonage" which closed after the marketing for Dragon Age 2 was over. This is why I started to archive interesting (for me) stuff from video sites.

On topic of filter, my current script does a channel listing (youtube-dl --get-id stops retrieving IDs after a blocked / copyright error, I have written a custom replacement bash script), filters out already downloaded IDs. Next step is going through each title and throwing out every match (substring) from a text file.

The length of the filter file varies between some 2-3 lines, but for some channels it can very long due to inconsistent naming of the videos. They are simple substrings to weed out.

Example in .cleanup.dat (filter file name):
lets_fail
angespielt
10_dinge_die_schlimmer
10_jahre_gamepro_geburtstag
gewinner
22_great_videogame
kane_and_lynch
force_unleashed
[ ... cut due length ]
entwicklervideo
developer_diary
unreal_engine
highlights_der_gamestar_dvd
war_thunder
watch_dogs
you_are_minecraft

[phihag]

Sorry for the delay. With youtube-dl 2013.11.22.2 and newer, there is now a --download-archive option which does what you want much easier. By the way, you can (I think since one of the very first versions) pass in the -i option to skip over blocked / copyright error videos, so there is absolutely no reason to write a custom shell script for that.

In summary, consider running

youtube-dl --download-archive ~/.my_video_log -i video_URL_here..

If you always want to use the archive, and always want -i, simply put

--download-archive ~/.my_video_log -i

into ~/.config/youtube-dl.conf.