Using an HTTPS proxy hangs curl #1853

Closed
ljwagerfield opened this Issue Sep 2, 2017 · 6 comments

Comments

Projects
None yet
3 participants
@ljwagerfield

ljwagerfield commented Sep 2, 2017

I did this

curl -v --proxy-insecure -x https://localhost:8080 http://google.com/

I expected the following

<response from http://google.com>

What actually happened

*   Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8080 (#0)
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /usr/local/etc/openssl/cert.pem
  CApath: /usr/local/etc/openssl/certs
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* Proxy certificate:
*  subject: C=AU; ST=Some-State; O=Internet Widgits Pty Ltd
*  start date: Dec  7 19:02:52 2016 GMT
*  expire date: Dec  7 19:02:52 2017 GMT
*  issuer: C=AU; ST=Some-State; O=Internet Widgits Pty Ltd
*  SSL certificate verify result: self signed certificate (18), continuing anyway.

...hangs at this point

curl/libcurl version

curl 7.55.1 (x86_64-apple-darwin16.7.0) libcurl/7.55.1 OpenSSL/1.0.2l zlib/1.2.8 nghttp2/1.25.0
Release-Date: 2017-08-14
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP HTTP2 UnixSockets HTTPS-proxy

operating system

macOS Sierra 10.12.6


This is the same behaviour as: #1156

I am using the same gist to run a basic HTTPS proxy.

@bagder

This comment has been minimized.

Show comment Hide comment
@bagder

bagder Sep 2, 2017

Member

So you truly want HTTPS to the proxy? (The title says HTTP...)

Member

bagder commented Sep 2, 2017

So you truly want HTTPS to the proxy? (The title says HTTP...)

@jay

This comment has been minimized.

Show comment Hide comment
@jay

jay Sep 2, 2017

Member

Bisected to 5113ad0, only affects http without proxytunnel. I didn't use the gist script for reasons noted below, instead I manually monitored socat output to see when the CONNECT GET lines would show (if they did bisect good, if they didn't bisect bad).
socat openssl-listen:4433,reuseaddr,fork,cert=yourcerthere.pem,verify=0 -

The reason I didn't use the gist script is because that was a very narrow example written to help us reproduce an issue with a proxytunnel to google's server. if you look in the script you'll see that. Technically you could repurpose it by changing the line in the script to net.Dial("tcp", "www.google.com:80") and then add curl option --proxytunnel since curl by default won't tunnel http connections, but the bug is not reproducible then.

Member

jay commented Sep 2, 2017

Bisected to 5113ad0, only affects http without proxytunnel. I didn't use the gist script for reasons noted below, instead I manually monitored socat output to see when the CONNECT GET lines would show (if they did bisect good, if they didn't bisect bad).
socat openssl-listen:4433,reuseaddr,fork,cert=yourcerthere.pem,verify=0 -

The reason I didn't use the gist script is because that was a very narrow example written to help us reproduce an issue with a proxytunnel to google's server. if you look in the script you'll see that. Technically you could repurpose it by changing the line in the script to net.Dial("tcp", "www.google.com:80") and then add curl option --proxytunnel since curl by default won't tunnel http connections, but the bug is not reproducible then.

@ljwagerfield ljwagerfield changed the title from Using an HTTP proxy hangs curl (again) to Using an HTTPS proxy hangs curl (again) Sep 3, 2017

@ljwagerfield

This comment has been minimized.

Show comment Hide comment
@ljwagerfield

ljwagerfield Sep 3, 2017

@bagder yes I mean't HTTPS proxy (title updated, sorry!).

In this very specific case, the gist script represents the problem quite well. I'm actually hitting the issue with a different HTTP proxy implementation, so it's not specific to the gist script. More specifically, it seems to be:

  • HTTPS proxy
  • HTTP origin
  • GET instead of CONNECT (i.e. no --proxytunnel option)

@bagder yes I mean't HTTPS proxy (title updated, sorry!).

In this very specific case, the gist script represents the problem quite well. I'm actually hitting the issue with a different HTTP proxy implementation, so it's not specific to the gist script. More specifically, it seems to be:

  • HTTPS proxy
  • HTTP origin
  • GET instead of CONNECT (i.e. no --proxytunnel option)

@bagder bagder changed the title from Using an HTTPS proxy hangs curl (again) to Using an HTTPS proxy hangs curl Sep 3, 2017

bagder added a commit that referenced this issue Sep 4, 2017

http-proxy: when not doing CONNECT, that phase is done immediately
`conn->connect_state` is NULL when doing a regular non-CONNECT request
over the proxy and should therefor be considered complete at once.

Fixes #1853
Reported-by: Lawrence Wagerfield
@bagder

This comment has been minimized.

Show comment Hide comment
@bagder

bagder Sep 4, 2017

Member

Please try the fix in #1862 and see if it helps your case. I could reproduce and this change made the problem go away for me.

Member

bagder commented Sep 4, 2017

Please try the fix in #1862 and see if it helps your case. I could reproduce and this change made the problem go away for me.

@ljwagerfield

This comment has been minimized.

Show comment Hide comment
@ljwagerfield

ljwagerfield Sep 5, 2017

Have tried building curl for the first time using --with-darwinssl -- but HTTPS Proxy does not get listed in the feature set.

Does it only work with --with-ssl?

Have tried building curl for the first time using --with-darwinssl -- but HTTPS Proxy does not get listed in the feature set.

Does it only work with --with-ssl?

@bagder

This comment has been minimized.

Show comment Hide comment
@bagder

bagder Sep 5, 2017

Member

Only with OpenSSL, GnuTLS and NSS.

I think the limitation for darwinssl is mostly that nobody has worked on actually adapting that backend for HTTPS proxy.

Member

bagder commented Sep 5, 2017

Only with OpenSSL, GnuTLS and NSS.

I think the limitation for darwinssl is mostly that nobody has worked on actually adapting that backend for HTTPS proxy.

@bagder bagder closed this in 9ef50ee Sep 7, 2017

@lock lock bot locked as resolved and limited conversation to collaborators May 6, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.