User-Agent not set when proxying SSL request #2381

Closed
audrummer15 opened this Issue Mar 14, 2018 · 4 comments

Comments

Projects
None yet
3 participants
@audrummer15

audrummer15 commented Mar 14, 2018

The issue

The man page specifies that the user-agent set from using -A can also be set by using -H. This leads to misconceptions, because -A will set proxy-header user-agents, and -H will not. I'm proposing a documentation change, based on the following outcomes.

The Test Setup

Simulate a proxy with socat so we can see the requests:

kali# socat - TCP-LISTEN:80,crlf,reuseaddr,fork 

Make an https request using curl with a special user-agent:

user@kali:~# curl -v --proxy http://127.0.0.1:80 -H "User-Agent: Mozilla/5.0 (Curl Proxy Test)" https://www.google.com/
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 80 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to www.google.com:443
> CONNECT www.google.com:443 HTTP/1.1
> Host: www.google.com:443
> User-Agent: curl/7.58.0
> Proxy-Connection: Keep-Alive
>
^C

user@kali:~# curl -v --proxy http://127.0.0.1:80 -A "Mozilla/5.0 (Curl Proxy Test)" https://www.google.com/
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 80 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to www.google.com:443
> CONNECT www.google.com:443 HTTP/1.1
> Host: www.google.com:443
> User-Agent: Mozilla/5.0 (Curl Proxy Test)
> Proxy-Connection: Keep-Alive
>
^C

I expected the following

I expected the user-agent's to be the same, since the man page specifies that the UA can be set with either the -A or -H flags.

Expected output below:

user@kali:~# curl -v --proxy http://127.0.0.1:80 -H "User-Agent: Mozilla/5.0 (Curl Proxy Test)" https://www.google.com/
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 80 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to www.google.com:443
> CONNECT www.google.com:443 HTTP/1.1
> Host: www.google.com:443
> User-Agent: Mozilla/5.0 (Curl Proxy Test)
> Proxy-Connection: Keep-Alive
>
^C

user@kali:~# curl -v --proxy http://127.0.0.1:80 -A "Mozilla/5.0 (Curl Proxy Test)" https://www.google.com/
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 80 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to www.google.com:443
> CONNECT www.google.com:443 HTTP/1.1
> Host: www.google.com:443
> User-Agent: Mozilla/5.0 (Curl Proxy Test)
> Proxy-Connection: Keep-Alive
>
^C

curl/libcurl version

curl 7.58.0 (x86_64-pc-linux-gnu) libcurl/7.58.0 OpenSSL/1.0.2n zlib/1.2.8 libidn2/2.0.4 libpsl/0.19.1 (+libidn2/2.0.4) libssh2/1.8.0 nghttp2/1.29.0 librtmp/2.3
Release-Date: 2018-01-24
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP HTTP2 UnixSockets HTTPS-proxy PSL

operating system

Linux Kali 4.14.0-kali3-amd64 #1 SMP Debian 4.14.12-2kali2 (2018-01-11) x86_64 GNU/Linux

@bagder

This comment has been minimized.

Show comment Hide comment
@bagder

bagder Mar 14, 2018

Member

The man page section for -H says:

Starting in 7.37.0, you need --proxy-header to send custom headers intended for a proxy.

Member

bagder commented Mar 14, 2018

The man page section for -H says:

Starting in 7.37.0, you need --proxy-header to send custom headers intended for a proxy.

@jay jay added the documentation label Mar 15, 2018

@audrummer15

This comment has been minimized.

Show comment Hide comment
@audrummer15

audrummer15 Mar 15, 2018

@bagder thanks for the response. My confusion came from this line:

-A, --user-agent
(HTTP) Specify the User-Agent string to send to the HTTP server. To encode blanks
in the string, surround the string with single quote marks. This can also be set
with the -H, --header option of course.

That made it sound like -A and -H do the same thing, which I realize now is obviously not the case.

EDIT: If this isn't really seen as an issue by the community, feel free to close it. Personally, it just didn't make sense to a few of us that -A and -H had different effects.

audrummer15 commented Mar 15, 2018

@bagder thanks for the response. My confusion came from this line:

-A, --user-agent
(HTTP) Specify the User-Agent string to send to the HTTP server. To encode blanks
in the string, surround the string with single quote marks. This can also be set
with the -H, --header option of course.

That made it sound like -A and -H do the same thing, which I realize now is obviously not the case.

EDIT: If this isn't really seen as an issue by the community, feel free to close it. Personally, it just didn't make sense to a few of us that -A and -H had different effects.

@bagder

This comment has been minimized.

Show comment Hide comment
@bagder

bagder Mar 15, 2018

Member

They once had the exact same behavior, until we realized that sending -H headers unconditionally to both the server and the proxy has security implications and you may very well decide to send headers to just one of the parties...

Member

bagder commented Mar 15, 2018

They once had the exact same behavior, until we realized that sending -H headers unconditionally to both the server and the proxy has security implications and you may very well decide to send headers to just one of the parties...

bagder added a commit that referenced this issue Mar 15, 2018

@bagder

This comment has been minimized.

Show comment Hide comment
@bagder

bagder Mar 15, 2018

Member

I added a mention of --proxy-header to the man page section for -A.

Member

bagder commented Mar 15, 2018

I added a mention of --proxy-header to the man page section for -A.

@bagder bagder closed this Mar 15, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment