HTTPS proxy failures #148

Closed
geemus opened this Issue Sep 11, 2012 · 28 comments

Projects

None yet

8 participants

@geemus
Contributor
geemus commented Sep 11, 2012

Seems that https proxy stuff isn't working correctly, but I've had a hell of a time figuring out what I'm missing. My google fu is weak and/or there isn't much specific information to be found. I tried to the best of my ability to follow what little info I could and emulate Net::HTTP 1.9 functionality but the requests always fail through https proxy anyway. I tried today to make more progress by running Charles locally, but wasn't really able to get anywhere.

/cc @nextmat - any chance you could take a look and see if anything sticks out to you that I may have missed?

See also:

heroku/legacy-cli#441
heroku/legacy-cli#503

Contributor
nextmat commented Sep 11, 2012

Happy to take a look. It may be a week or more before I really have time to dig into this, unfortunately.

Contributor
geemus commented Sep 11, 2012

@nextmat - understood, I'll appreciate any help you can give when you get a chance. I've been banging my head against this off and on for far too long, decided it was time to admit defeat.

Any luck on this issue, I am also facing the same issue behind the proxy,
Or pls suggest any alternative method to deploy heroku app.

C:\Program Files\Heroku\lib\heroku>heroku login
Enter your Heroku credentials.
Email: xxxxxx@xxx.com
Password (typing will be hidden):
! Heroku client internal error.
! Search for help at: https://help.heroku.com
! Or report a bug at: https://github.com/heroku/heroku/issues/new

Error:       A non-blocking socket operation could not be completed immediately. (Errno::EWOULDBLOCK) (Excon::Errors::SocketError)
Backtrace:   C:/Program Files/ruby-1.9.2/lib/ruby/1.9.1/openssl/buffering.rb:36:in `sysread'
             C:/Program Files/ruby-1.9.2/lib/ruby/1.9.1/openssl/buffering.rb:36:in `sysread'
             C:/Program Files/ruby-1.9.2/lib/ruby/1.9.1/openssl/buffering.rb:36:in `fill_rbuff'
             C:/Program Files/ruby-1.9.2/lib/ruby/1.9.1/openssl/buffering.rb:68:in `read'
             C:/Program Files/Heroku/vendor/gems/excon-0.16.2/lib/excon/response.rb:21:in `parse'
             C:/Program Files/Heroku/vendor/gems/excon-0.16.2/lib/excon/ssl_socket.rb:56:in `initialize'
             C:/Program Files/Heroku/vendor/gems/excon-0.16.2/lib/excon/connection.rb:357:in `new'
             C:/Program Files/Heroku/vendor/gems/excon-0.16.2/lib/excon/connection.rb:357:in `socket'
             C:/Program Files/Heroku/vendor/gems/excon-0.16.2/lib/excon/connection.rb:188:in `request_kernel'
             C:/Program Files/Heroku/vendor/gems/excon-0.16.2/lib/excon/connection.rb:101:in `request'                 
     C:/Program Files/Heroku/vendor/gems/heroku-api-0.3.5/lib/heroku/api.rb:62:in `request'                 
     C:/Program Files/Heroku/vendor/gems/heroku-api-.3.5/lib/heroku/api/login.rb:9:in `post_login'
             C:/Program Files/Heroku/lib/heroku/auth.rb:80:in `api_key'
             C:/Program Files/Heroku/lib/heroku/auth.rb:189:in `ask_for_credentials'
             C:/Program Files/Heroku/lib/heroku/auth.rb:221:in `ask_for_and_save_credentials'
             C:/Program Files/Heroku/lib/heroku/auth.rb:84:in `get_credentials'
             C:/Program Files/Heroku/lib/heroku/auth.rb:41:in `login'
             C:/Program Files/Heroku/lib/heroku/command/auth.rb:31:in `login'
             C:/Program Files/Heroku/lib/heroku/command.rb:206:in `run'
             C:/Program Files/Heroku/lib/heroku/cli.rb:28:in `start'
             C:/Program Files/Heroku/bin/heroku:23:in `<main>'

Command:     heroku login
HTTP Proxy:  http://proxy.1232.com:8080
HTTPS Proxy: http://proxy.1232.com:8080
Version:     heroku-toolbelt/2.32.8 (i386-mingw32) ruby/1.9.2 autoupdate
Contributor
geemus commented Oct 16, 2012

@srinivasanraju - I would love to say I had a breakthrough, but unfortunately I'm just as stuck as ever right now.

Contributor
geemus commented Oct 16, 2012

FWIW, here is the document that I could somehow never find via searching, but that I suspect may include the solution: http://www.ietf.org/rfc/rfc2817.txt

@geemus geemus closed this in 0d7331e Oct 16, 2012
Contributor
geemus commented Oct 16, 2012

NOTE: I've had mixed luck with testing/verifying. I did my best to make sure the behavior mapped closely to that of curl when using Charles debugging proxy (I updated until both gave me certificate/handshake errors). This should be closer to the right solution at least, but since I don't have a proper https proxy setup it is hard to completely test/verify.

still have issues here - i am using ubuntu (12.04) behind a proxy server.

Heroku is the only application has the issue so far: http/ftp all works fine, I can login CloudFoundry behind the firewall - they had problem there as well in the past.

heroku login
Enter your Heroku credentials.
Email: xxxxxxx
Password (typing will be hidden):
! Heroku client internal error.
! Search for help at: https://help.heroku.com
! Or report a bug at: https://github.com/heroku/heroku/issues/new
Error: SSL_connect SYSCALL returned=5 errno=0 state=SSLv3 read server hello A (OpenSSL::SSL::SSLError) (Excon::Errors::SocketError)
Backtrace: /usr/local/heroku/vendor/gems/excon-0.16.4/lib/excon/ssl_socket.rb:60:in connect' /usr/local/heroku/vendor/gems/excon-0.16.4/lib/excon/ssl_socket.rb:60:ininitialize'
/usr/local/heroku/vendor/gems/excon-0.16.4/lib/excon/connection.rb:362:in new' /usr/local/heroku/vendor/gems/excon-0.16.4/lib/excon/connection.rb:362:insocket'
/usr/local/heroku/vendor/gems/excon-0.16.4/lib/excon/connection.rb:193:in request_kernel' /usr/local/heroku/vendor/gems/excon-0.16.4/lib/excon/connection.rb:101:inrequest'
/usr/local/heroku/vendor/gems/heroku-api-0.3.5/lib/heroku/api.rb:62:in request' /usr/local/heroku/vendor/gems/heroku-api-0.3.5/lib/heroku/api/login.rb:9:inpost_login'
/usr/local/heroku/lib/heroku/auth.rb:80:in api_key' /usr/local/heroku/lib/heroku/auth.rb:189:inask_for_credentials'
/usr/local/heroku/lib/heroku/auth.rb:221:in ask_for_and_save_credentials' /usr/local/heroku/lib/heroku/auth.rb:84:inget_credentials'
/usr/local/heroku/lib/heroku/auth.rb:41:in login' /usr/local/heroku/lib/heroku/command/auth.rb:31:inlogin'
/usr/local/heroku/lib/heroku/command.rb:206:in run' /usr/local/heroku/lib/heroku/cli.rb:28:instart'
/usr/bin/heroku:25:in `

'

Command:     heroku login
HTTP Proxy:  http://web-proxy.mycom.com:8080/
HTTPS Proxy: https://web-proxy.mycom.com:8080/
Version:     heroku-toolbelt/2.33.0 (x86_64-linux) ruby/1.9.3
Contributor
nextmat commented Nov 2, 2012

I unfortunately don't have access to a real https proxy anymore either but Charles should be pretty consistent. The last fix from @geemus is available in 0.16.5 and later, looks like the toolbelt is still using 0.16.4

Contributor
nextmat commented Nov 2, 2012

If you do upgrade the toolbelt and this fixes the issue please let us know. Thanks!

Hi Nexmat,

What i used is heroku-toolbelt/2.33.0. what is the latest version? where to get it.

Actually in uninstall the heroku-toobelt: apt-get uninstall heroku... and then try to install gem instead (not the toolbelt, i guess i may get a latest version by doing that) - but i found out the heroku --pre version is even older then what the one in heroku-toolbelt.

thank you for your response, please help :-)

Contributor
geemus commented Nov 5, 2012

@ethan-jiang-1 - heroku has decided to deprecate the gem, so new versions will no longer be released. So the latest and greatest will only be available via toolbelt. I'll let the maintainer know that a bump should happen there to get these changes.

Contributor
nextmat commented Nov 5, 2012

@ethan-jiang-1 The best source for heroku toolbelt is here: https://toolbelt.heroku.com

Hi there,

I uninstalled the toolbelt i had (installed not long ago) and install the latest one from : https://toolbelt.heroku.com

I noticed the information below during installation ,

Setting up foreman (0.60.0) ...
Setting up heroku (2.33.0) ...
Setting up heroku-toolbelt (2.33.0) ...

It is exactly same version of heroku-toolbelt i had last time, 2.33.0. so i got exactly the same error when i "heroku login...". the problem is still there,

Do you have some pre-release version somewhere so that i can give it a try?

Appericate your help

Contributor
geemus commented Nov 12, 2012

@ddollar - I think this is up in heroku.rb where it would need to be for toolbelt to get at it. When do newer heroku.rb versions get bumped up into toolbelt?

ddollar commented Nov 12, 2012

I'll do my best to get to it this week. If it's easy, I'd also be hugely grateful for a pull request to heroku/heroku.

Contributor
geemus commented Nov 12, 2012

@ddollar - I don't think any changes are required outside of pulling the new heroku.rb in, here is the pull request: heroku/legacy-cli#627. Just let me know on the pull request if there are additional steps I need to take there.

Contributor

I also facing ethan-jiang-1's issue. OS is Ubuntu12.10. Reading this topic, I applied related patches to installed excon sources manually. But still failed.

I observed handshake protocol then found issue. 0d7331e postpones SSL handshake but SSL Client Hello is sent before proxy connection is established. a part of proxy cannot be passed this strange order. At least my MS proxy... So SSL socket connection should be wait until proxy connection is established.

I try to revert eating response code, but it is not returned until timeout(?). Next, I replace it to simple "@socket.gets". It's success.

Contributor

Sorry, I reverted another line when success. "ssl_context.ssl_version = 'SSLv3' ". I'm not sure this is needed. So I will check tomorrow.

Contributor
geemus commented Nov 13, 2012

@rinrinne - thanks for the update. A diff or pull request showing what you got working would be very helpful. Glad to know that it sounds like we are close to something working at least.

Contributor

I found 2 issues.

  • SSL Client Hello is sent before proxy connection is established.
  • In Excon::Response#parse, IO.read is blocked when getting 200 status response without body.

Maybe IO blocking is the root cause. So I create patches. Could you review them?

Contributor
geemus commented Nov 14, 2012

Happy to review.

@geemus and @rinrinne,

Thanks both, I manually replace the excon gem 0.16.8 with your latest 0.16.10 in my current heroku-toolbelt/2.33.1. I am able to login in through our company's proxy now.

Cheers

Contributor
geemus commented Nov 26, 2012

@ethan-jiang-1 - great, thanks for the update! Hoping that this means the next version of the toolbelt will no longer have this issue.

Not sure if this helps anyone but was getting the exact error when attempting to set my HTTP_PROXY to my companies autoproxy url (which returns an automatic configuration script). When switching to an actual IP address of a proxy server the problem was resolved.

same problem here

root@amr:~# heroku login
Enter your Heroku credentials.
Email: dr.hacker.py@gmail.com
Password (typing will be hidden):
! Heroku client internal error.
! Search for help at: https://help.heroku.com
! Or report a bug at: https://github.com/heroku/heroku/issues/new

Error:       Connection reset by peer - SSL_connect (Errno::ECONNRESET) (Excon::Errors::SocketError)
Backtrace:   /usr/local/heroku/vendor/gems/excon-0.20.1/lib/excon/ssl_socket.rb:55:in `connect'
             /usr/local/heroku/vendor/gems/excon-0.20.1/lib/excon/ssl_socket.rb:55:in `initialize'
             /usr/local/heroku/vendor/gems/excon-0.20.1/lib/excon/connection.rb:344:in `new'
             /usr/local/heroku/vendor/gems/excon-0.20.1/lib/excon/connection.rb:344:in `socket'
             /usr/local/heroku/vendor/gems/excon-0.20.1/lib/excon/connection.rb:90:in `request_call'
             /usr/local/heroku/vendor/gems/excon-0.20.1/lib/excon/middlewares/mock.rb:79:in `request_call'
             /usr/local/heroku/vendor/gems/excon-0.20.1/lib/excon/middlewares/instrumentor.rb:22:in `request_call'
             /usr/local/heroku/vendor/gems/excon-0.20.1/lib/excon/middlewares/base.rb:15:in `request_call'
             /usr/local/heroku/vendor/gems/excon-0.20.1/lib/excon/middlewares/base.rb:15:in `request_call'
             /usr/local/heroku/vendor/gems/excon-0.20.1/lib/excon/connection.rb:223:in `request'
             /usr/local/heroku/vendor/gems/heroku-api-0.3.9/lib/heroku/api.rb:74:in `request'
             /usr/local/heroku/vendor/gems/heroku-api-0.3.9/lib/heroku/api/login.rb:9:in `post_login'
             /usr/local/heroku/lib/heroku/auth.rb:80:in `api_key'
             /usr/local/heroku/lib/heroku/auth.rb:189:in `ask_for_credentials'
             /usr/local/heroku/lib/heroku/auth.rb:221:in `ask_for_and_save_credentials'
             /usr/local/heroku/lib/heroku/auth.rb:84:in `get_credentials'
             /usr/local/heroku/lib/heroku/auth.rb:41:in `login'
             /usr/local/heroku/lib/heroku/command/auth.rb:31:in `login'
             /usr/local/heroku/lib/heroku/command.rb:206:in `run'
             /usr/local/heroku/lib/heroku/cli.rb:28:in `start'
             /usr/bin/heroku:25:in `<main>'

Command:     heroku login
HTTP Proxy:  http://dr_hacker%40students.mans.edu.eg:password@muproxy.mans.edu.eg:8080
HTTPS Proxy: http://dr_hacker%40students.mans.edu.eg:password@muproxy.mans.edu.eg:8080
Version:     heroku-toolbelt/2.39.0 (i686-linux) ruby/1.9.3
Contributor
geemus commented Jul 1, 2013

@AmrFouad - that version of excon did not yet decode URI encoded values, which would cause problems with the @ encoded as %40 in your proxy values.

You should have better luck if you change your proxy to read: http://dr_hacker@students.mans.edu.eg:password@muproxy.mans.edu.eg:8080

That was subsequently fixed in excon, but toolbelt has not yet been updated to take advantage of this. For now I believe the above should allow you to work around the problem, but do let me know if you continue to have problems.

AmrFouad commented Jul 2, 2013

I've already tried that and it doesn't even give me the "give your credentials" prompt.

root@amr:~# heroku login
! Heroku client internal error.
! Search for help at: https://help.heroku.com
! Or report a bug at: https://github.com/heroku/heroku/issues/new

Error:       the scheme http does not accept registry part: dr_hacker@students.mans.edu.eg:password@muproxy.mans.edu.eg:8080 (or bad hostname?) (URI::InvalidURIError)
Backtrace:   /usr/lib/ruby/1.9.1/uri/generic.rb:213:in `initialize'
             /usr/lib/ruby/1.9.1/uri/http.rb:84:in `initialize'
             /usr/lib/ruby/1.9.1/uri/common.rb:214:in `new'
             /usr/lib/ruby/1.9.1/uri/common.rb:214:in `parse'
             /usr/lib/ruby/1.9.1/uri/common.rb:747:in `parse'
             /usr/local/heroku/vendor/gems/excon-0.20.1/lib/excon/connection.rb:357:in `setup_proxy'
             /usr/local/heroku/vendor/gems/excon-0.20.1/lib/excon/connection.rb:49:in `initialize'
             /usr/local/heroku/vendor/gems/excon-0.20.1/lib/excon.rb:123:in `new'
             /usr/local/heroku/vendor/gems/excon-0.20.1/lib/excon.rb:123:in `new'
             /usr/local/heroku/lib/heroku/command/certs.rb:7:in `<class:Certs>'
             /usr/local/heroku/lib/heroku/command/certs.rb:6:in `<top (required)>'
             /usr/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
             /usr/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
             /usr/local/heroku/lib/heroku/command.rb:14:in `block in load'
             /usr/local/heroku/lib/heroku/command.rb:13:in `each'
             /usr/local/heroku/lib/heroku/command.rb:13:in `load'
             /usr/local/heroku/lib/heroku/cli.rb:27:in `start'
             /usr/bin/heroku:25:in `<main>'

Command:     heroku login
HTTP Proxy:  http://dr_hacker@students.mans.edu.eg:password@muproxy.mans.edu.eg:8080
HTTPS Proxy: http://dr_hacker@students.mans.edu.eg:password@muproxy.mans.edu.eg:8080
Version:     heroku-toolbelt/2.39.0 (i686-linux) ruby/1.9.3
Contributor
rinrinne commented Jul 4, 2013

Available characterset for userinfo is defined in RFC2396:

userinfo = ( unreserved | escaped | ";" | ":" | "&" | "=" | "+" | "$" | "," )
unreserved = alphanum | mark
mark = "-" | "_" | "." | "!" | "~" | "
" | "'" | "(" | ")"

So first proxy configuration by @AmrFouad is correct.

I think @AmrFouad facing issue is not proxy issue but SSL connection issue since SSL connection reset is sent to client after SSL handshake(it is started after proxy connection is established).
I'm not sure why connection was reset unexpectedly but you should try again without proxy (also proxy configuration)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment