Using --etag-compare with --output truncate existing output

[kwon-young]

I did this

$ curl --etag-save test.etag -o test.ext -v 'http://myurl.com/test.ext'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                                                                                                                                                              
                                 Dload  Upload   Total   Spent    Left  Speed                                                                                                                                                                                
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying ip:80...                                                                                                                                                  
* Connected to myurl.com (ip) port 80 (#0)                                                                                                                                                                                                     
> GET /test.ext HTTP/1.1                                                                                                                    
> Host: myurl.com                                                                                                                                                                                                                                    
> User-Agent: curl/7.69.1                                                                                                                                                                                                                                    
> Accept: */*                                                                                                                                                                                                                                                
>                                                                                                                                                                                                                                                            
* Mark bundle as not supporting multiuse                                                                                                                                                                                                                     
< HTTP/1.1 200 OK                                                                                                                                                                                                                                            < Date: Sat, 04 Apr 2020 00:21:28 GMT                                                                                                                                                                                                                        
< Content-Type: application/ext                                                                                                                                                                                                                             
< Content-Length: 3840872                                                                                                                                                                                                                                    
< Connection: keep-alive                                                                                                                                                                                                                                     
< Set-Cookie: __cfduid=d55f833dcabfc0685a31a6437ba68ad091585959688; expires=Mon, 04-May-20 00:21:28 GMT; path=/; domain=.myurl.com; HttpOnly; SameSite=Lax                                                                                                   
< Last-Modified: Tue, 25 Feb 2014 02:26:16 GMT                                                                                                                                                                                                               
< ETag: "530bff48-3a9b68"                                                                                                                                                                                                                                    
< Cache-Control: max-age=14400                                                                                                                                                                                                                               
< CF-Cache-Status: HIT                                                                                                                                                                                                                                       
< Age: 6791                                                                                                                                                                                                                                                  
< Accept-Ranges: bytes                                                                                                                                                                                                                                       
< Server: cloudflare                                                                                                                                                                                                                                         
< CF-RAY: 57e6de961b60ee4c-CDG                                                                                                                                                                                                                               
<                                                                                                                                                                                                                                                            
{ [964 bytes data]                                                                                                                                                                                                                                           
100 3750k  100 3750k    0     0  5138k      0 --:--:-- --:--:-- --:--:-- 5131k                                                                                                                                                                               
* Connection #0 to host myurl.com left intact

This download test.ext as expected.

If we try to update test.ext but only if the file changed on the server using --etag-compare and the server respond that the file has not been updated, test.ext is truncated:

$ curl --etag-compare test.etag -o test.ext -v 'http://myurl.com/test.ext'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                                                                                                                                                              
                                 Dload  Upload   Total   Spent    Left  Speed                                                                                                                                                                                
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying ip:80...                                                                                                                                                  
* Connected to myurl.com (ip) port 80 (#0)                                                                                                                                                                                                     
> GET /test.ext HTTP/1.1                                                                                                                    
> Host: myurl.com                                                                                                                                                                                                                                    
> User-Agent: curl/7.69.1                                                                                                                                                                                                                                    
> Accept: */*                                                                                                                                                                                                                                                
> If-None-Match: "530bff48-3a9b68"                                                                                                                                                                                                                           
>                                                                                                                                                                                                                                                            
* Mark bundle as not supporting multiuse                                                                                                                                                                                                                     
< HTTP/1.1 304 Not Modified                                                                                                                                                                                                                                  
< Date: Sat, 04 Apr 2020 00:23:30 GMT                                                                                                                                                                                                                        < Connection: keep-alive                                                                                                                                                                                                                                     
< Set-Cookie: __cfduid=d582f0a38586003e65053440888ac938f1585959810; expires=Mon, 04-May-20 00:23:30 GMT; path=/; domain=.myurl.com; HttpOnly; SameSite=Lax                                                                                                   
< Last-Modified: Tue, 25 Feb 2014 02:26:16 GMT                                                                                                                                                                                                               
< ETag: "530bff48-3a9b68"                                                                                                                                                                                                                                    
< Cache-Control: max-age=14400                                                                                                                                                                                                                               
< CF-Cache-Status: HIT                                                                                                                                                                                                                                       
< Age: 6913                                                                                                                                                                                                                                                  
< Server: cloudflare                                                                                                                                                                                                                                         
< CF-RAY: 57e6e18f2f0da84f-CDG                                                                                                                                                                                                                               
<                                                                                                                                                                                                                                                              0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0                                                                                                                                                                               
* Connection #0 to host myurl.com left intact
$ file test.ext
test.ext: empty

curl/libcurl version

curl 7.69.1 (x86_64-pc-linux-gnu) libcurl/7.69.1 OpenSSL/1.1.1e zlib/1.2.11 libidn2/2.3.0 libpsl/0.21.0 (+libidn2/2.2.0) libssh2/1.9.0 nghttp2/1.40.0
Release-Date: 2020-03-11
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS GSS-API HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets

operating system

Linux edwin 5.5.13-1-MANJARO #1 SMP PREEMPT Wed Mar 25 17:14:28 UTC 2020 x86_64 GNU/Linux

Discussion

I would like to help resolve this issue, but I am having a hard writing a test case of this.

I would need the ability to specify existing files like test.etag and test.ext prior running the curl command.
Then assert that test.ext is still the same after running the curl command.

Is there some documentation on the language used for the test cases?

[bagder]

I would need the ability to specify existing files like test.etag and test.ext prior running the curl command.

You can create files for the test in <client><file> (where file can also have 1,2,3,4 suffixes I believe).

Then assert that test.ext is still the same after running the curl command.

<verify><file>, also with suffixes if you need more than one

Is there some documentation on the language used for the test cases?

Yes tests/FILEFORMAT but I will also recommend grepping existing test cases in case the language in the docs isn't clear!

[kwon-young]

I saw that you wrote the test case for this.

Thank you for your help.