--verbose prints CR+LF (\r\n) instead of just \n for header lines (started with > or <) on Linux

[andrei-korshikov]

I did this

$ curl 'https://curl.se' --head --verbose 2>output
Look in output at lines started with < or > (headers-related): they have ^M at the end, i.e. they ends with 0x0D0A.
Other lines (started with *, { or }) are not affected (don't contain \r before \n).

I expected the following

All --verbose output lines should end with \n—not \r\n—on Linux.

curl/libcurl version

curl 8.12.0 (x86_64-pc-linux-gnu) libcurl/8.12.0 OpenSSL/3.4.0 zlib/1.3.1 brotli/1.1.0 zstd/1.5.6 libidn2/2.3.7 libpsl/0.21.5 libssh2/1.11.1 nghttp2/1.64.0 nghttp3/1.7.0
Release-Date: 2025-02-05
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTP3 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM PSL SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd

operating system

Linux laptop 6.7.8-zen1-1-zen #1 ZEN SMP PREEMPT_DYNAMIC Sun, 03 Mar 2024 00:30:23 +0000 x86_64 GNU/Linux
(Arch Linux)

[bagder]

I believe curl has done this since its first release.

[bagder]

I fear that changing this now will cause problems to users who through the decades have adapted to this behavior.

The output is quite simply exactly what the server sends, and as curl also supports plain LR line endings for HTTP/1 transfers they are not always stored as CRLF.

I think we should consider documenting this properly to be the fix.

[andrei-korshikov]

I believe curl has done this since its first release.

I think so too:)

The output is quite simply exactly what the server sends, and as curl also supports plain LR line endings for HTTP/1 transfers they are not always stored as CRLF.

Hmm. The problem is: there is no CR or LF or CRLF on the wire (I've checked with Wireshark), but ^M is present in curl's output. Moreover, curl prints CR for empty strings (> in the end of the header block). And CR is present in request headers, which are generated by curl (again, it is in the command output, but not on the wire).

So, I believe #99903b5 is very confusing and even misleading. While it is technically true (curl does show the exact content), users might believe that their ^M are always from a server. As I see from Wireshark captures, in my case ^Ms are pure representation things generated by curl itself.

Maybe my original description was not clear enough, I'm sorry if so. CR is present before LF at the end of each and every header line, both for request and response headers. Both client and server are Linux boxes. There is no CR in network packets—so CR is printing (representation) "feature" of --verbose, not a real byte from network.

[bagder]

Hmm. The problem is: there is no CR or LF or CRLF on the wire (I've checked with Wireshark

If you ask for HTTP/1.1, you do. When curl speaks HTTP/2 or HTTP/3 it converts and shows the headers to look the same: then the CRLF pair is actually put there by curl itself. But the purpose is then to make them look like they do when HTTP/1.x is used. I'm not sure we need to elaborate on this exact process in the documentation.

Moreover, curl prints CR for empty strings (> in the end of the header block).

Like for received that's because the protocol includes CRLF.

[andrei-korshikov]

But the purpose is then to make them look like they do when HTTP/1.x is used.

Aha! Now I see. Thank you for the explanation. And, of course, thank you for curl:)