Pages saves different versions with same video name

[latot]

Hi, i found this from iqiyi, i was curious because when we try get all the formats from the webpage all formats was skiped, this can be extended out of this site, the problem is, some pages have multiples versions of the files with for example different qualities, but like the iqiyi case, all the files are saved with the same file name, but with different url...

So with this we are unable to download all formats, maybe we need other way to store this, i try for example the "resolution" option for the filename and can save all the formats, but there value in iqiyi is "NA" for example...

I think would be better, store a filename and the data used to know the quality of the file, if don't match just add a number or something to the file to avoid this.

notice, first i apply this pr #18640 to do this works, there is a bug, so we need download the video and then use --all-formats

~/.local/bin/youtube-dl "https://www.iqiyi.com/v_19rrl6lnds.html" -v --proxy 221.6.201.18:9999
~/.local/bin/youtube-dl "https://www.iqiyi.com/v_19rrl6lnds.html" -v --proxy 221.6.201.18:9999 --all-subs --all-formats
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['https://www.iqiyi.com/v_19rrl6lnds.html', '-v', '--proxy', '221.6.201.18:9999', '--all-subs', '--all-formats']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2018.12.17
[debug] Python version 3.6.5 (CPython) - Linux-4.14.83-gentoo-x86_64-Intel-R-_Core-TM-_i7-8550U_CPU_@_1.80GHz-with-gentoo-2.6
[debug] exe versions: ffmpeg 3.3.6, ffprobe 3.3.6, rtmpdump 2.4
[debug] Proxy map: {'http': '221.6.201.18:9999', 'https': '221.6.201.18:9999'}
[iqiyi] temp_id: download video page
[iqiyi] 38943e07b32ce17fa106830e41008574: Downloading JSON metadata
[info] 38943e07b32ce17fa106830e41008574: downloading video in 6 formats
[debug] Invoking downloader on 'http://cache.m.iqiyi.com/mus/text/202775501/1e85091ad50aea3143ad616a149b2ca4/afbe8fd3d73448c9//20170301/18/4a/285267c23b2011f86b50ac9759381e3a.m3u8?qd_originate=tmts_py&tvid=439121100&bossStatus=0&qd_vip=0&px=&src=3_31_312&prv=&previewType=&previewTime=&from=&qd_time=1550001677125&qd_p=dd06c912&qd_asc=c87d4264172fa3bc435eadca26ad6a1a&qypid=439121100_04022000001000000000_96&qd_k=e8d008ffab6d1d6c3a21e8686cb02980&isdol=0&code=2&ff=f4v&iswb=0&qd_s=otv&vf=d90f2f99c0bc0e16ce4f7b97ad9de821&np_tag=nginx_part_tag'
[download] 狐妖小红娘  - 第14集 狐妖小红娘第14集-38943e07b32ce17fa106830e41008574.mp4 has already been downloaded
[download] 100% of 96.33MiB
[debug] ffmpeg command line: ffprobe -show_streams 'file:狐妖小红娘  - 第14集 狐妖小红娘第14集-38943e07b32ce17fa106830e41008574.mp4'
[ffmpeg] Fixing malformed AAC bitstream in "狐妖小红娘  - 第14集 狐妖小红娘第14集-38943e07b32ce17fa106830e41008574.mp4"
[debug] ffmpeg command line: ffmpeg -y -i 'file:狐妖小红娘  - 第14集 狐妖小红娘第14集-38943e07b32ce17fa106830e41008574.mp4' -c copy -f mp4 -bsf:a aac_adtstoasc 'file:狐妖小红娘  - 第14集 狐妖小红娘第14集-38943e07b32ce17fa106830e41008574.temp.mp4'
[debug] Invoking downloader on 'http://cache.m.iqiyi.com/mus/text/202775501/66f9b3675c015fa92518b07ba75b43b2/afbe8fd3d73448c9//20170301/1e/35/e9a102d643f584edb40718cafe6cc1f8.m3u8?qd_originate=tmts_py&tvid=439121100&bossStatus=0&qd_vip=0&px=&src=3_31_312&prv=&previewType=&previewTime=&from=&qd_time=1550001677125&qd_p=dd06c912&qd_asc=6124c2e8c001aa1bf56fd8795885621b&qypid=439121100_04022000001000000000_1&qd_k=e8d008ffab6d1d6c3a21e8686cb02980&isdol=0&code=2&ff=f4v&iswb=0&qd_s=otv&vf=ba13e08471c4449e951dd2cd0734a09a&np_tag=nginx_part_tag'
[download] 狐妖小红娘  - 第14集 狐妖小红娘第14集-38943e07b32ce17fa106830e41008574.mp4 has already been downloaded
[download] 100% of 96.33MiB
[debug] ffmpeg command line: ffprobe -show_streams 'file:狐妖小红娘  - 第14集 狐妖小红娘第14集-38943e07b32ce17fa106830e41008574.mp4'
[ffmpeg] Fixing malformed AAC bitstream in "狐妖小红娘  - 第14集 狐妖小红娘第14集-38943e07b32ce17fa106830e41008574.mp4"
[debug] ffmpeg command line: ffmpeg -y -i 'file:狐妖小红娘  - 第14集 狐妖小红娘第14集-38943e07b32ce17fa106830e41008574.mp4' -c copy -f mp4 -bsf:a aac_adtstoasc 'file:狐妖小红娘  - 第14集 狐妖小红娘第14集-38943e07b32ce17fa106830e41008574.temp.mp4'
[debug] Invoking downloader on 'http://cache.m.iqiyi.com/mus/text/202775501/38943e07b32ce17fa106830e41008574/afbe8fd3d73448c9//20170301/ee/89/22ab72a295ff1a0ca29b092ad40366d9.m3u8?qd_originate=tmts_py&tvid=439121100&bossStatus=0&qd_vip=0&px=&src=3_31_312&prv=&previewType=&previewTime=&from=&qd_time=1550001677125&qd_p=dd06c912&qd_asc=7b96483ecbd994b0d321976e7899976c&qypid=439121100_04022000001000000000_2&qd_k=e8d008ffab6d1d6c3a21e8686cb02980&isdol=0&code=2&ff=f4v&iswb=0&qd_s=otv&vf=93995fc620aa402808f1f134bc285bb6&np_tag=nginx_part_tag'
[download] 狐妖小红娘  - 第14集 狐妖小红娘第14集-38943e07b32ce17fa106830e41008574.mp4 has already been downloaded
[download] 100% of 96.33MiB
[debug] ffmpeg command line: ffprobe -show_streams 'file:狐妖小红娘  - 第14集 狐妖小红娘第14集-38943e07b32ce17fa106830e41008574.mp4'
[ffmpeg] Fixing malformed AAC bitstream in "狐妖小红娘  - 第14集 狐妖小红娘第14集-38943e07b32ce17fa106830e41008574.mp4"
[debug] ffmpeg command line: ffmpeg -y -i 'file:狐妖小红娘  - 第14集 狐妖小红娘第14集-38943e07b32ce17fa106830e41008574.mp4' -c copy -f mp4 -bsf:a aac_adtstoasc 'file:狐妖小红娘  - 第14集 狐妖小红娘第14集-38943e07b32ce17fa106830e41008574.temp.mp4'
[debug] Invoking downloader on 'http://cache.m.iqiyi.com/mus/text/202775501/a77fdc27211fd0cf2d96bc54714d60a8/afbe8fd3d73448c9//20181019/88/d1/c85c8fae78531987e3d785e3cffe6416.m3u8?qd_originate=tmts_py&tvid=439121100&bossStatus=0&qd_vip=0&px=&src=3_31_312&prv=&previewType=&previewTime=&from=&qd_time=1550001677125&qd_p=dd06c912&qd_asc=6c174ed050136e264eeb39a4228f3a3a&qypid=439121100_04022000001000000000_21&qd_k=e8d008ffab6d1d6c3a21e8686cb02980&isdol=0&code=1&ff=265ts&iswb=0&qd_s=otv&vf=724f0c39240e7123cb12541789a43a64&np_tag=nginx_part_tag'
[download] 狐妖小红娘  - 第14集 狐妖小红娘第14集-38943e07b32ce17fa106830e41008574.mp4 has already been downloaded
[download] 100% of 96.33MiB
[debug] ffmpeg command line: ffprobe -show_streams 'file:狐妖小红娘  - 第14集 狐妖小红娘第14集-38943e07b32ce17fa106830e41008574.mp4'
[ffmpeg] Fixing malformed AAC bitstream in "狐妖小红娘  - 第14集 狐妖小红娘第14集-38943e07b32ce17fa106830e41008574.mp4"
[debug] ffmpeg command line: ffmpeg -y -i 'file:狐妖小红娘  - 第14集 狐妖小红娘第14集-38943e07b32ce17fa106830e41008574.mp4' -c copy -f mp4 -bsf:a aac_adtstoasc 'file:狐妖小红娘  - 第14集 狐妖小红娘第14集-38943e07b32ce17fa106830e41008574.temp.mp4'
[debug] Invoking downloader on 'http://cache.m.iqiyi.com/mus/text/202775501/ab39b39a74433b623ece9374bc792912/afbe8fd3d73448c9//20181019/09/2d/9f8c7d81782fa95d35c074eefa3bb230.m3u8?qd_originate=tmts_py&tvid=439121100&bossStatus=0&qd_vip=0&px=&src=3_31_312&prv=&previewType=&previewTime=&from=&qd_time=1550001677125&qd_p=dd06c912&qd_asc=c4bf270e9381319740dc675c0347ac7e&qypid=439121100_04022000001000000000_17&qd_k=e8d008ffab6d1d6c3a21e8686cb02980&isdol=0&code=1&ff=265ts&iswb=0&qd_s=otv&vf=255a1fadca974eaa85ed522912e74d78&np_tag=nginx_part_tag'
[download] 狐妖小红娘  - 第14集 狐妖小红娘第14集-38943e07b32ce17fa106830e41008574.mp4 has already been downloaded
[download] 100% of 96.33MiB
[debug] ffmpeg command line: ffprobe -show_streams 'file:狐妖小红娘  - 第14集 狐妖小红娘第14集-38943e07b32ce17fa106830e41008574.mp4'
[ffmpeg] Fixing malformed AAC bitstream in "狐妖小红娘  - 第14集 狐妖小红娘第14集-38943e07b32ce17fa106830e41008574.mp4"
[debug] ffmpeg command line: ffmpeg -y -i 'file:狐妖小红娘  - 第14集 狐妖小红娘第14集-38943e07b32ce17fa106830e41008574.mp4' -c copy -f mp4 -bsf:a aac_adtstoasc 'file:狐妖小红娘  - 第14集 狐妖小红娘第14集-38943e07b32ce17fa106830e41008574.temp.mp4'
[debug] Invoking downloader on 'http://cache.m.iqiyi.com/mus/text/202775501/c984fd7f451d2458a6016732420897fa/afbe8fd3d73448c9//20170301/df/81/f3615d89c41e9c0ffac94e32c86c8aa8.m3u8?qd_originate=tmts_py&tvid=439121100&bossStatus=0&qd_vip=0&px=&src=3_31_312&prv=&previewType=&previewTime=&from=&qd_time=1550001677125&qd_p=dd06c912&qd_asc=116465442f2fa324f00f27040704a4c8&qypid=439121100_04022000001000000000_4&qd_k=e8d008ffab6d1d6c3a21e8686cb02980&isdol=0&code=2&ff=f4v&iswb=0&qd_s=otv&vf=839138be572dc6b9e073dcd70e7080d1&np_tag=nginx_part_tag'
[download] 狐妖小红娘  - 第14集 狐妖小红娘第14集-38943e07b32ce17fa106830e41008574.mp4 has already been downloaded
[download] 100% of 96.33MiB
[debug] ffmpeg command line: ffprobe -show_streams 'file:狐妖小红娘  - 第14集 狐妖小红娘第14集-38943e07b32ce17fa106830e41008574.mp4'
[ffmpeg] Fixing malformed AAC bitstream in "狐妖小红娘  - 第14集 狐妖小红娘第14集-38943e07b32ce17fa106830e41008574.mp4"
[debug] ffmpeg command line: ffmpeg -y -i 'file:狐妖小红娘  - 第14集 狐妖小红娘第14集-38943e07b32ce17fa106830e41008574.mp4' -c copy -f mp4 -bsf:a aac_adtstoasc 'file:狐妖小红娘  - 第14集 狐妖小红娘第14集-38943e07b32ce17fa106830e41008574.temp.mp4'

At the end, this is not for iqiyi, just for pages that saves the files with the same name and different qualities.

Bye.

[dstftw]

As already pointed out when downloading multiple formats it's you responsibility to provide unique output template.

[latot]

Hi yes, but is not working how should, there is not info about, so for example we can use epoch to generate "something" about the files, but that means youtube-dl will loose the ability to recognize downloaded videos because epoch will do every file with a different name every time.

Even out of how the file is named i think would be useful if youtube-dl can store and compare not only the filename the quality too.

[latot]

@dstftw notice for example in iqiyi, all the files are in the same format, but different qualities.....

[dstftw]

Do you even read my messages? You don't need to generate "something". You must use format_id sequence as already pointed out.

[latot]

Hi, i was confused u.u

format_id is the numeric value and not the extension/format of the file (mp4, webm, etc).

Thanks for the patience and help.

Thx.