User-Agent not set when proxying SSL request

[coffeegist]

The issue

The man page specifies that the user-agent set from using -A can also be set by using -H. This leads to misconceptions, because -A will set proxy-header user-agents, and -H will not. I'm proposing a documentation change, based on the following outcomes.

The Test Setup

Simulate a proxy with socat so we can see the requests:

kali# socat - TCP-LISTEN:80,crlf,reuseaddr,fork 

Make an https request using curl with a special user-agent:

user@kali:~# curl -v --proxy http://127.0.0.1:80 -H "User-Agent: Mozilla/5.0 (Curl Proxy Test)" https://www.google.com/
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 80 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to www.google.com:443
> CONNECT www.google.com:443 HTTP/1.1
> Host: www.google.com:443
> User-Agent: curl/7.58.0
> Proxy-Connection: Keep-Alive
>
^C

user@kali:~# curl -v --proxy http://127.0.0.1:80 -A "Mozilla/5.0 (Curl Proxy Test)" https://www.google.com/
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 80 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to www.google.com:443
> CONNECT www.google.com:443 HTTP/1.1
> Host: www.google.com:443
> User-Agent: Mozilla/5.0 (Curl Proxy Test)
> Proxy-Connection: Keep-Alive
>
^C

I expected the following

I expected the user-agent's to be the same, since the man page specifies that the UA can be set with either the -A or -H flags.

Expected output below:

user@kali:~# curl -v --proxy http://127.0.0.1:80 -H "User-Agent: Mozilla/5.0 (Curl Proxy Test)" https://www.google.com/
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 80 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to www.google.com:443
> CONNECT www.google.com:443 HTTP/1.1
> Host: www.google.com:443
> User-Agent: Mozilla/5.0 (Curl Proxy Test)
> Proxy-Connection: Keep-Alive
>
^C

user@kali:~# curl -v --proxy http://127.0.0.1:80 -A "Mozilla/5.0 (Curl Proxy Test)" https://www.google.com/
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 80 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to www.google.com:443
> CONNECT www.google.com:443 HTTP/1.1
> Host: www.google.com:443
> User-Agent: Mozilla/5.0 (Curl Proxy Test)
> Proxy-Connection: Keep-Alive
>
^C

curl/libcurl version

curl 7.58.0 (x86_64-pc-linux-gnu) libcurl/7.58.0 OpenSSL/1.0.2n zlib/1.2.8 libidn2/2.0.4 libpsl/0.19.1 (+libidn2/2.0.4) libssh2/1.8.0 nghttp2/1.29.0 librtmp/2.3
Release-Date: 2018-01-24
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP HTTP2 UnixSockets HTTPS-proxy PSL

operating system

Linux Kali 4.14.0-kali3-amd64 #1 SMP Debian 4.14.12-2kali2 (2018-01-11) x86_64 GNU/Linux

[bagder]

The man page section for -H says:

Starting in 7.37.0, you need --proxy-header to send custom headers intended for a proxy.

[coffeegist]

@bagder thanks for the response. My confusion came from this line:

-A, --user-agent
(HTTP) Specify the User-Agent string to send to the HTTP server. To encode blanks
in the string, surround the string with single quote marks. This can also be set
with the -H, --header option of course.

That made it sound like -A and -H do the same thing, which I realize now is obviously not the case.

EDIT: If this isn't really seen as an issue by the community, feel free to close it. Personally, it just didn't make sense to a few of us that -A and -H had different effects.

[bagder]

They once had the exact same behavior, until we realized that sending -H headers unconditionally to both the server and the proxy has security implications and you may very well decide to send headers to just one of the parties...

[bagder]

I added a mention of --proxy-header to the man page section for -A.