Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

--etag-save and --etag-compare are not compatible with multiple URLs #15729

Closed
tamird opened this issue Dec 12, 2024 · 3 comments
Closed

--etag-save and --etag-compare are not compatible with multiple URLs #15729

tamird opened this issue Dec 12, 2024 · 3 comments
Assignees

Comments

@tamird
Copy link

tamird commented Dec 12, 2024

I did this

$ curl -sfSL --create-dirs --output-dir test/.tmp/debian-kernels/arm64 --parallel --remote-name-all --etag-compare .etags --etag-save .etags https://mirrors.wikimedia.org/debian/pool/main/l/linux/linux-image-6.1.0-28-cloud-arm64-unsigned_6.1.119-1_arm64.deb https://mirrors.wikimedia.org/debian/pool/main/l/linux/linux-image-6.10.12-cloud-arm64-unsigned_6.10.12-1_arm64.deb
$ cat .etags
"13542e4-6279763772481"
$ curl -sfSL --create-dirs --output-dir test/.tmp/debian-kernels/arm64 --parallel --remote-name-all --etag-compare .etags --etag-save .etags https://mirrors.wikimedia.org/debian/pool/main/l/linux/linux-image-6.1.0-28-cloud-arm64-unsigned_6.1.119-1_arm64.deb https://mirrors.wikimedia.org/debian/pool/main/l/linux/linux-image-6.10.12-cloud-arm64-unsigned_6.10.12-1_arm64.deb
$ cat .etags
"19aab48-623740fdc9977"

It's clear that there's trampling going on here - only one of the etags makes it into the file. In fact the design of these options is inherently incompatible with multiple files, but there's also no validation that rejects the combination.

I expected the following

An error or otherwise correct handling of etags for multiple files.

curl/libcurl version

curl 8.7.1 (x86_64-apple-darwin24.0) libcurl/8.7.1 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.12 nghttp2/1.62.0
Release-Date: 2024-03-27
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IPv6 Kerberos Largefile libz MultiSSL NTLM SPNEGO SSL threadsafe UnixSockets

operating system

Darwin tamird-mac 24.1.0 Darwin Kernel Version 24.1.0: Thu Oct 10 21:02:45 PDT 2024; root:xnu-11215.41.3~2/RELEASE_ARM64_T8112 arm64

@bagder bagder self-assigned this Dec 12, 2024
@tamird tamird changed the title --etags-save and --etags-compare are not compatible with multiple URs --etag-save and --etag-compare are not compatible with multiple URs Dec 12, 2024
@bagder bagder changed the title --etag-save and --etag-compare are not compatible with multiple URs --etag-save and --etag-compare are not compatible with multiple URLs Dec 12, 2024
@bagder
Copy link
Member

bagder commented Dec 12, 2024

I think the most sensible thing is to error when trying this on multiple URLs, as it would need something much more advanced to work correctly for such cases.

@bagder
Copy link
Member

bagder commented Dec 12, 2024

Alternatively, we make etags work on the first provided URL

@tamird
Copy link
Author

tamird commented Dec 12, 2024

In my opinion that would be worse. You could potentially conceive of a backwards-compatible change to the etags file format, but I wouldn't do something (like the current behavior) that surprises users.

bagder added a commit that referenced this issue Dec 12, 2024
And document it.

Add tests 484 and 485

Fixes #15729
Reported-by: Tamir Duberstein
Closes #
@bagder bagder closed this as completed in a300879 Dec 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging a pull request may close this issue.

2 participants