proxy-ntlm requires Proxy-Connection: keep-alive #954

Closed
fajan opened this Issue Aug 11, 2016 · 7 comments

Projects

None yet

2 participants

@fajan
fajan commented Aug 11, 2016

I did this

In version 7.48.0 changelog: "Proxy-Connection: stop sending this header by default"
Source issue #633

"NTLM authentication cannot be used if an intervening proxy does not support keep-alive connections." (msdn)

Without the 'Proxy-Connection' header, --proxy-ntlm fails.

Maybe related issue: #876

curl 7.50.1 command:

$ curl --proxy-ntlm --proxy-user "MyDomain\ntlmtest:test" --proxy proxy.estest.intra:3128 -v https://example.com/file
*   Trying 101.102.103.104...
* Connected to proxy.estest.intra (101.102.103.104) port 3128 (#0)
* Establish HTTP proxy tunnel to example.com:443
* Proxy auth using NTLM with user 'MyDomain\ntlmtest'
> CONNECT example.com:443 HTTP/1.1
> Host: example.com:443
> Proxy-Authorization: NTLM TlRMTVNTUAA...AAAAAAAA=
> User-Agent: curl/7.50.1
>
< HTTP/1.0 407 Proxy Authentication Required
< Server: squid/2.7.STABLE5
< Date: Thu, 11 Aug 2016 08:17:15 GMT
< Content-Type: text/html
< Content-Length: 1330
< X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
< Proxy-Authenticate: Basic realm="Squid proxy-caching web server"
< X-Cache: MISS from intra.my.domain.com
< X-Cache-Lookup: NONE from intra.my.domain.com:3128
< Via: 1.0 intra.my.domain.com:3128 (squid/2.7.STABLE5)
< Connection: close
<
* Ignore 1330 bytes of response-body
* Received HTTP code 407 from proxy after CONNECT
* Closing connection 0
curl: (56) Received HTTP code 407 from proxy after CONNECT

I expected the following

same command with 7.47.1

$c:/programs/curl/7.47.1/curl --proxy-ntlm --proxy-user "MyDomain\ntlmtest:test" --proxy proxy.estest.intra:3128 -v https://example.com/file
*   Trying 101.102.103.104...
* Connected to proxy.estest.intra (101.102.103.104) port 3128 (#0)
* Establish HTTP proxy tunnel to example.com:443
* Proxy auth using NTLM with user 'MyDomain\ntlmtest'
> CONNECT example.com:443 HTTP/1.1
> Host: example.com:443
> Proxy-Authorization: NTLM TlRMTVNTUAA...AAAAAAAA=
> User-Agent: curl/7.47.1
> Proxy-Connection: Keep-Alive
>
< HTTP/1.0 407 Proxy Authentication Required
< Server: squid/2.7.STABLE5
< Date: Thu, 11 Aug 2016 09:18:33 GMT
< Content-Type: text/html
< Content-Length: 1330
< X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
< Proxy-Authenticate: NTLM TlRMTVNTUAA.....AuAGgAdQAAAAAA
< X-Cache: MISS from intra.my.domain.com
< X-Cache-Lookup: NONE from intra.my.domain.com:3128
< Via: 1.0 intra.my.domain.com:3128 (squid/2.7.STABLE5)
< Connection: keep-alive
< Proxy-Connection: keep-alive
<
* Ignore 1330 bytes of response-body
* TUNNEL_STATE switched to: 0
* Establish HTTP proxy tunnel to example.com:443
* Proxy auth using NTLM with user 'MyDomain\ntlmtest'
> CONNECT example.com:443 HTTP/1.1
> Host: example.com:443
> Proxy-Authorization: NTLM TlRMTVNTUAA........1GRUpMMTM=
> User-Agent: curl/7.47.1
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 Connection established
<
* Proxy replied OK to CONNECT request
....

command on 7.50.1. with --proxy-header proxy-connection:keep-alive extra options

$ curl --proxy-ntlm --proxy-user "MyDomain\ntlmtest:test" --proxy proxy.estest.intra:3128 --proxy-header proxy-connection:keep-alive -v https://example.com/file
*   Trying 101.102.103.104...
* Connected to proxy.estest.intra (101.102.103.104) port 3128 (#0)
* Establish HTTP proxy tunnel to example.com:443
* Proxy auth using NTLM with user 'MyDomain\ntlmtest'
> CONNECT example.com:443 HTTP/1.1
> Host: example.com:443
> Proxy-Authorization: NTLM TlRMTVNTUAA...AAAAAAA=
> User-Agent: curl/7.50.1
> proxy-connection:keep-alive
>
< HTTP/1.0 407 Proxy Authentication Required
< Server: squid/2.7.STABLE5
< Date: Thu, 11 Aug 2016 08:18:55 GMT
< Content-Type: text/html
< Content-Length: 1330
< X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
< Proxy-Authenticate: NTLM TlRMTVNTUAA.....AuAGgAdQAAAAAA
< X-Cache: MISS from intra.my.domain.com
< X-Cache-Lookup: NONE from intra.my.domain.com:3128
< Via: 1.0 intra.my.domain.com:3128 (squid/2.7.STABLE5)
< Connection: keep-alive
< Proxy-Connection: keep-alive
<
* Ignore 1330 bytes of response-body
* TUNNEL_STATE switched to: 0
* Establish HTTP proxy tunnel to example.com:443
* Proxy auth using NTLM with user 'MyDomain\ntlmtest'
> CONNECT example.com:443 HTTP/1.1
> Host: example.com:443
> Proxy-Authorization: NTLM TlRMTVNTUAA........1GRUpMMTM=
> User-Agent: curl/7.50.1
> proxy-connection:keep-alive
>
< HTTP/1.1 200 Connection established
<
* Proxy replied OK to CONNECT request
....

curl/libcurl version

$ curl -V
curl 7.50.1 (x86_64-pc-win32) libcurl/7.50.1 OpenSSL/1.0.2g
Protocols: dict file ftp ftps gopher http https imap imaps ldap pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IPv6 Largefile NTLM SSL

$ c:/programs/curl/7.47.1/curl -V
curl 7.47.1 (x86_64-pc-win32) libcurl/7.47.1 OpenSSL/1.0.2g
Protocols: dict file ftp ftps gopher http https imap imaps ldap pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IPv6 Largefile NTLM SSL

The same with ENABLE_SSPI=yes versions of above.
@bagder
Member
bagder commented Aug 11, 2016

So if you revert 113f04e from git master, it works again?

@fajan
fajan commented Aug 12, 2016

Yes.

$ curl --proxy-ntlm --proxy proxy.estest.intra:3128 --proxy-user MyDomain\ntlmtest:test -v https://example.com/file
*   Trying 101.102.103.104...
* TCP_NODELAY set
* Connected to proxy.estest.intra (101.102.103.104) port 3128 (#0)
* Establish HTTP proxy tunnel to example.com:443
* Proxy auth using NTLM with user 'MyDomain\ntlmtest'
> CONNECT example.com:443 HTTP/1.1
> Host: example.com:443
> Proxy-Authorization: NTLM TlRMTVNTU...AAAAAAAA=
> User-Agent: curl/7.50.2-DEV
> Proxy-Connection: Keep-Alive
>
< HTTP/1.0 407 Proxy Authentication Required
< Server: squid/2.7.STABLE5
< Date: Fri, 12 Aug 2016 06:06:00 GMT
< Content-Type: text/html
< Content-Length: 1333
< X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
< Proxy-Authenticate: NTLM TlRMTVNTUAA.....QAAAAAA
< X-Cache: MISS from intra.my.domain.com
< X-Cache-Lookup: NONE from intra.my.domain.com:3128
< Via: 1.0 intra.my.domain.com:3128 (squid/2.7.STABLE5)
< Connection: keep-alive
< Proxy-Connection: keep-alive
<
* Ignore 1333 bytes of response-body
* TUNNEL_STATE switched to: 0
* Establish HTTP proxy tunnel to example.com:443
* Proxy auth using NTLM with user 'MyDomain\ntlmtest'
> CONNECT example.com:443 HTTP/1.1
> Host: example.com:443
> Proxy-Authorization: NTLM TlRMTVNTU....SS1GRUpMMTM=
> User-Agent: curl/7.50.2-DEV
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 Connection established
<
* Proxy replied OK to CONNECT request
@bagder
Member
bagder commented Aug 12, 2016

Any idea what proxy/version this is that's causing this trouble? As mentioned in the issue, the header is deprecated and the advice is not to use...

@fajan
fajan commented Aug 12, 2016

Well, ntlm isn't today's tech, but still used.
I would suggest to only set 'Proxy-Connection' header if the auth mode is ntlm.
Systems that support this auth won't refuse the header especially if paired with 'Proxy-Authorization: NTLM ...'.

@bagder
Member
bagder commented Aug 12, 2016

Well, this isn't strictly because of NTLM as NTLM over proxy works perfectly well without this header. This problem is because of the proxy you're using is relying on this header to mean something more than what it actually does. But sure, we could limit the use of the header to NTLM only to limit the scope of the use.

But again, for documentation and curiosity, do you know which proxy/version this is that causes us this problem?

@fajan
fajan commented Aug 12, 2016

Okay, I missunderstood.
Squid Cache: Version 2.7.5 as the header said.
Meanwhile I found that it also accepts the --proxy-header Connection:keep-alive, which should solve the other issue's as well. "Go's included ReverseProxy removes hop-by-hop HTTP headers, but forgot to remove the ancient and never-standardized "Proxy-Connection" hop-by-hop header."
I don't really know how other proxies handle to this.

 curl --proxy-ntlm --proxy proxy.estest.intra:3128 --proxy-header Connection:keep-alive --proxy-user MyDomain/ntlmtest:test -v https://example.com/file
*   Trying 101.102.103.104...
* Connected to proxy.estest.intra (101.102.103.104) port 3128 (#0)
* Establish HTTP proxy tunnel to example.com:443
* Proxy auth using NTLM with user 'MyDomain/ntlmtest'
> CONNECT example.com:443 HTTP/1.1
> Host: example.com:443
> Proxy-Authorization: NTLM TlRMTVNT...AAAAAA=
> User-Agent: curl/7.50.1
> Connection:keep-alive
>
< HTTP/1.0 407 Proxy Authentication Required
< Server: squid/2.7.STABLE5
< Date: Fri, 12 Aug 2016 09:27:51 GMT
< Content-Type: text/html
< Content-Length: 1330
< X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
< Proxy-Authenticate: NTLM TlRMTVNTU......gAdQAAAAAA
< X-Cache: MISS from intra.my.domain.com
< X-Cache-Lookup: NONE from intra.my.domain.com:3128
< Via: 1.0 intra.my.domain.com:3128 (squid/2.7.STABLE5)
< Connection: keep-alive
< Proxy-Connection: keep-alive
<
* Ignore 1330 bytes of response-body
* TUNNEL_STATE switched to: 0
* Establish HTTP proxy tunnel to example.com:443
* Proxy auth using NTLM with user 'MyDomain/ntlmtest'
> CONNECT example.com:443 HTTP/1.1
> Host: example.com:443
> Proxy-Authorization: NTLM TlRMTVNTUAA......ISS1GRUpMMTM=
> User-Agent: curl/7.50.1
> Connection:keep-alive
>
< HTTP/1.1 200 Connection established
<
* Proxy replied OK to CONNECT request
@bagder bagder added a commit that closed this issue Aug 16, 2016
@bagder bagder proxy: fix tests as follow-up to 93b0d90
This fixes tests that were added after 113f04e as the tests would
fail otherwise.

We bring back "Proxy-Connection: Keep-Alive" now unconditionally to fix
regressions with old and stupid proxies, but we could possibly switch to
using it only for CONNECT or only for NTLM in a future if we want to
gradually reduce it.

Fixes #954

Reported-by: János Fekete
fa6b6f1
@bagder bagder closed this in fa6b6f1 Aug 16, 2016
@bagder
Member
bagder commented Aug 16, 2016

Thanks for this, should be back and functional in git master now!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment