I did this
# initial download
curl --create-dirs --output 'test/123_#1.json' --url 'https://httpbin.org/get?[1-3]'
sleep 60
# added `--skip-existing`
curl --create-dirs --output 'test/123_#1.json' --url 'https://httpbin.org/get?[1-3]' --skip-existing
sleep 60
# added `--parallel`
curl --create-dirs --output 'test/123_#1.json' --url 'https://httpbin.org/get?[1-3]' --skip-existing --parallel
I expected the following
I expected curl to skip existing files in --parallel mode, like in sequential mode, but parallel overwrites the files instead of skipping them.
curl/libcurl version
curl 8.10.1 (x86_64-pc-linux-musl) libcurl/8.10.1 OpenSSL/3.1.7 zlib/1.3.1 libssh2/1.11.0 nghttp2/1.58.0
Release-Date: 2024-09-18
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS HSTS HTTP2 HTTPS-proxy IPv6 Largefile libz NTLM SSL threadsafe TLS-SRP UnixSockets
operating system
Linux 6.10.9-1-liquorix-amd64 #1 ZEN SMP PREEMPT liquorix 6.10-11.1~bookworm (2024-09-08) x86_64 GNU/Linux
I did this
I expected the following
I expected curl to skip existing files in
--parallelmode, like in sequential mode, butparalleloverwrites the files instead of skipping them.curl/libcurl version
operating system