-j --flat-playlist duplicate/missing urls

[TheHardew]

Checklist

• I'm reporting a broken site support issue
• I've verified that I'm running youtube-dl version 2020.01.24
• I've checked that all provided URLs are alive and playable in a browser
• I've checked that all URLs and arguments with special characters are properly quoted or escaped
• I've searched the bugtracker for similar bug reports including closed ones
• I've read bugs section in FAQ

Verbose log

Command:

youtube-dl -vj --flat-playlist "https://www.youtube.com/user/EthosLab" | grep wdv

Output:

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-vj', '--flat-playlist', 'https://www.youtube.com/user/EthosLab']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2020.01.24
[debug] Python version 3.8.1 (CPython) - Linux-5.4.15-zen1-1-zen-x86_64-with-glibc2.2.5
[debug] exe versions: ffmpeg 4.2.2, ffprobe 4.2.2, rtmpdump 2.4
[debug] Proxy map: {}
{"_type": "url", "url": "wdvjnwNYrxo", "ie_key": "Youtube", "id": "wdvjnwNYrxo", "title": "Minecraft - Simulation Protocol: Episode 14"}
{"_type": "url", "url": "wdvjnwNYrxo", "ie_key": "Youtube", "id": "wdvjnwNYrxo", "title": "Minecraft - Simulation Protocol: Episode 14"}

Description

I was trying to dump urls to videos from a channel to a text file. One url was printed twice. The channel has currently 2318 videos, 2318 lines were printed, which means that one video was also skipped, "https://youtu.be/CfosiqM_XyI". None of the videos should be printed twice. In case this is not a problem with youtube-dl, if youtube.com did send the same url twice, the second instance should be filtered out and youtube-dl should try to download urls until it has 2318 (in this case) unique urls.

[dstftw]

Video is duplicated in playlist itself.