Question about downloading mutiple playlists, which might contain same track, but named differently by uploader and by youtube.

[jd1008]

Hi,
Is there a way to tell youtube-dl not to download duplicates of a file?
For example, an artist's works are uploaded by many different people,
which results in youtube appending a unique string to the track names
to distinguish them from other tracks with the same original titles.

Thanx!!

[phihag]

If multiple playlists contain the same video object (i.e. the same video URL), youtube-dl will only download them once. If multiple people upload a video that's bitwise-identical, the resulting videos may still be different - for example, due to being encoded at different times, edited, cut, etc. .

Therefore, it is not feasible to detect videos that look similar to a human, but are not bitwise-identical when downloaded. In any case, even if a video service would always return the same file for the same upload, we would still need to download the entire file before being sure that it is indeed identical (or get a checksum or so from the server).

If your filename only contains the video title, there may be a chance that videos have the same name. I could imagine a local database of downloaded videos by title and then a fuzzy search for the title, but that's outside the scope of the youtube-dl project. We'd be happy to accept code or maybe even add a --match-title-with-external-program option ourselves.