/curl

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proxy-ntlm requires Proxy-Connection: keep-alive #954

Closed
fajan opened this Issue Aug 11, 2016 · 7 comments

Comments

Assignees
No one assigned
Labels
Projects
None yet
Milestone
No milestone
2 participants
@fajan @bagder
@fajan

fajan commented Aug 11, 2016

I did this

In version 7.48.0 changelog: "Proxy-Connection: stop sending this header by default"
Source issue #633

"NTLM authentication cannot be used if an intervening proxy does not support keep-alive connections." (msdn)

Without the 'Proxy-Connection' header, --proxy-ntlm fails.

Maybe related issue: #876

curl 7.50.1 command:

$ curl --proxy-ntlm --proxy-user "MyDomain\ntlmtest:test" --proxy proxy.estest.intra:3128 -v https://example.com/file
*   Trying 101.102.103.104...
* Connected to proxy.estest.intra (101.102.103.104) port 3128 (#0)
* Establish HTTP proxy tunnel to example.com:443
* Proxy auth using NTLM with user 'MyDomain\ntlmtest'
> CONNECT example.com:443 HTTP/1.1
> Host: example.com:443
> Proxy-Authorization: NTLM TlRMTVNTUAA...AAAAAAAA=
> User-Agent: curl/7.50.1
>
< HTTP/1.0 407 Proxy Authentication Required
< Server: squid/2.7.STABLE5
< Date: Thu, 11 Aug 2016 08:17:15 GMT
< Content-Type: text/html
< Content-Length: 1330
< X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
< Proxy-Authenticate: Basic realm="Squid proxy-caching web server"
< X-Cache: MISS from intra.my.domain.com
< X-Cache-Lookup: NONE from intra.my.domain.com:3128
< Via: 1.0 intra.my.domain.com:3128 (squid/2.7.STABLE5)
< Connection: close
<
* Ignore 1330 bytes of response-body
* Received HTTP code 407 from proxy after CONNECT
* Closing connection 0
curl: (56) Received HTTP code 407 from proxy after CONNECT

I expected the following

same command with 7.47.1

$c:/programs/curl/7.47.1/curl --proxy-ntlm --proxy-user "MyDomain\ntlmtest:test" --proxy proxy.estest.intra:3128 -v https://example.com/file
*   Trying 101.102.103.104...
* Connected to proxy.estest.intra (101.102.103.104) port 3128 (#0)
* Establish HTTP proxy tunnel to example.com:443
* Proxy auth using NTLM with user 'MyDomain\ntlmtest'
> CONNECT example.com:443 HTTP/1.1
> Host: example.com:443
> Proxy-Authorization: NTLM TlRMTVNTUAA...AAAAAAAA=
> User-Agent: curl/7.47.1
> Proxy-Connection: Keep-Alive
>
< HTTP/1.0 407 Proxy Authentication Required
< Server: squid/2.7.STABLE5
< Date: Thu, 11 Aug 2016 09:18:33 GMT
< Content-Type: text/html
< Content-Length: 1330
< X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
< Proxy-Authenticate: NTLM TlRMTVNTUAA.....AuAGgAdQAAAAAA
< X-Cache: MISS from intra.my.domain.com
< X-Cache-Lookup: NONE from intra.my.domain.com:3128
< Via: 1.0 intra.my.domain.com:3128 (squid/2.7.STABLE5)
< Connection: keep-alive
< Proxy-Connection: keep-alive
<
* Ignore 1330 bytes of response-body
* TUNNEL_STATE switched to: 0
* Establish HTTP proxy tunnel to example.com:443
* Proxy auth using NTLM with user 'MyDomain\ntlmtest'
> CONNECT example.com:443 HTTP/1.1
> Host: example.com:443
> Proxy-Authorization: NTLM TlRMTVNTUAA........1GRUpMMTM=
> User-Agent: curl/7.47.1
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 Connection established
<
* Proxy replied OK to CONNECT request
....

command on 7.50.1. with --proxy-header proxy-connection:keep-alive extra options

$ curl --proxy-ntlm --proxy-user "MyDomain\ntlmtest:test" --proxy proxy.estest.intra:3128 --proxy-header proxy-connection:keep-alive -v https://example.com/file
*   Trying 101.102.103.104...
* Connected to proxy.estest.intra (101.102.103.104) port 3128 (#0)
* Establish HTTP proxy tunnel to example.com:443
* Proxy auth using NTLM with user 'MyDomain\ntlmtest'
> CONNECT example.com:443 HTTP/1.1
> Host: example.com:443
> Proxy-Authorization: NTLM TlRMTVNTUAA...AAAAAAA=
> User-Agent: curl/7.50.1
> proxy-connection:keep-alive
>
< HTTP/1.0 407 Proxy Authentication Required
< Server: squid/2.7.STABLE5
< Date: Thu, 11 Aug 2016 08:18:55 GMT
< Content-Type: text/html
< Content-Length: 1330
< X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
< Proxy-Authenticate: NTLM TlRMTVNTUAA.....AuAGgAdQAAAAAA
< X-Cache: MISS from intra.my.domain.com
< X-Cache-Lookup: NONE from intra.my.domain.com:3128
< Via: 1.0 intra.my.domain.com:3128 (squid/2.7.STABLE5)
< Connection: keep-alive
< Proxy-Connection: keep-alive
<
* Ignore 1330 bytes of response-body
* TUNNEL_STATE switched to: 0
* Establish HTTP proxy tunnel to example.com:443
* Proxy auth using NTLM with user 'MyDomain\ntlmtest'
> CONNECT example.com:443 HTTP/1.1
> Host: example.com:443
> Proxy-Authorization: NTLM TlRMTVNTUAA........1GRUpMMTM=
> User-Agent: curl/7.50.1
> proxy-connection:keep-alive
>
< HTTP/1.1 200 Connection established
<
* Proxy replied OK to CONNECT request
....

curl/libcurl version

$ curl -V
curl 7.50.1 (x86_64-pc-win32) libcurl/7.50.1 OpenSSL/1.0.2g
Protocols: dict file ftp ftps gopher http https imap imaps ldap pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IPv6 Largefile NTLM SSL

$ c:/programs/curl/7.47.1/curl -V
curl 7.47.1 (x86_64-pc-win32) libcurl/7.47.1 OpenSSL/1.0.2g
Protocols: dict file ftp ftps gopher http https imap imaps ldap pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IPv6 Largefile NTLM SSL

The same with ENABLE_SSPI=yes versions of above.

@bagder bagder added HTTP connecting & proxies labels Aug 11, 2016

@bagder

This comment has been minimized.

Show comment
Hide comment
@bagder

bagder Aug 11, 2016

Member

So if you revert 113f04e from git master, it works again?
Member

bagder commented Aug 11, 2016

So if you revert 113f04e from git master, it works again?
@fajan

This comment has been minimized.

Show comment
Hide comment
@fajan

fajan Aug 12, 2016

Yes.

$ curl --proxy-ntlm --proxy proxy.estest.intra:3128 --proxy-user MyDomain\ntlmtest:test -v https://example.com/file
*   Trying 101.102.103.104...
* TCP_NODELAY set
* Connected to proxy.estest.intra (101.102.103.104) port 3128 (#0)
* Establish HTTP proxy tunnel to example.com:443
* Proxy auth using NTLM with user 'MyDomain\ntlmtest'
> CONNECT example.com:443 HTTP/1.1
> Host: example.com:443
> Proxy-Authorization: NTLM TlRMTVNTU...AAAAAAAA=
> User-Agent: curl/7.50.2-DEV
> Proxy-Connection: Keep-Alive
>
< HTTP/1.0 407 Proxy Authentication Required
< Server: squid/2.7.STABLE5
< Date: Fri, 12 Aug 2016 06:06:00 GMT
< Content-Type: text/html
< Content-Length: 1333
< X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
< Proxy-Authenticate: NTLM TlRMTVNTUAA.....QAAAAAA
< X-Cache: MISS from intra.my.domain.com
< X-Cache-Lookup: NONE from intra.my.domain.com:3128
< Via: 1.0 intra.my.domain.com:3128 (squid/2.7.STABLE5)
< Connection: keep-alive
< Proxy-Connection: keep-alive
<
* Ignore 1333 bytes of response-body
* TUNNEL_STATE switched to: 0
* Establish HTTP proxy tunnel to example.com:443
* Proxy auth using NTLM with user 'MyDomain\ntlmtest'
> CONNECT example.com:443 HTTP/1.1
> Host: example.com:443
> Proxy-Authorization: NTLM TlRMTVNTU....SS1GRUpMMTM=
> User-Agent: curl/7.50.2-DEV
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 Connection established
<
* Proxy replied OK to CONNECT request

fajan commented Aug 12, 2016

Yes.

$ curl --proxy-ntlm --proxy proxy.estest.intra:3128 --proxy-user MyDomain\ntlmtest:test -v https://example.com/file
*   Trying 101.102.103.104...
* TCP_NODELAY set
* Connected to proxy.estest.intra (101.102.103.104) port 3128 (#0)
* Establish HTTP proxy tunnel to example.com:443
* Proxy auth using NTLM with user 'MyDomain\ntlmtest'
> CONNECT example.com:443 HTTP/1.1
> Host: example.com:443
> Proxy-Authorization: NTLM TlRMTVNTU...AAAAAAAA=
> User-Agent: curl/7.50.2-DEV
> Proxy-Connection: Keep-Alive
>
< HTTP/1.0 407 Proxy Authentication Required
< Server: squid/2.7.STABLE5
< Date: Fri, 12 Aug 2016 06:06:00 GMT
< Content-Type: text/html
< Content-Length: 1333
< X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
< Proxy-Authenticate: NTLM TlRMTVNTUAA.....QAAAAAA
< X-Cache: MISS from intra.my.domain.com
< X-Cache-Lookup: NONE from intra.my.domain.com:3128
< Via: 1.0 intra.my.domain.com:3128 (squid/2.7.STABLE5)
< Connection: keep-alive
< Proxy-Connection: keep-alive
<
* Ignore 1333 bytes of response-body
* TUNNEL_STATE switched to: 0
* Establish HTTP proxy tunnel to example.com:443
* Proxy auth using NTLM with user 'MyDomain\ntlmtest'
> CONNECT example.com:443 HTTP/1.1
> Host: example.com:443
> Proxy-Authorization: NTLM TlRMTVNTU....SS1GRUpMMTM=
> User-Agent: curl/7.50.2-DEV
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 Connection established
<
* Proxy replied OK to CONNECT request
@bagder

This comment has been minimized.

Show comment
Hide comment
@bagder

bagder Aug 12, 2016

Member

Any idea what proxy/version this is that's causing this trouble? As mentioned in the issue, the header is deprecated and the advice is not to use...
Member

bagder commented Aug 12, 2016

Any idea what proxy/version this is that's causing this trouble? As mentioned in the issue, the header is deprecated and the advice is not to use...
@fajan

This comment has been minimized.

Show comment
Hide comment
@fajan

fajan Aug 12, 2016

Well, ntlm isn't today's tech, but still used.
I would suggest to only set 'Proxy-Connection' header if the auth mode is ntlm.
Systems that support this auth won't refuse the header especially if paired with 'Proxy-Authorization: NTLM ...'.

fajan commented Aug 12, 2016

Well, ntlm isn't today's tech, but still used.
I would suggest to only set 'Proxy-Connection' header if the auth mode is ntlm.
Systems that support this auth won't refuse the header especially if paired with 'Proxy-Authorization: NTLM ...'.
@bagder

This comment has been minimized.

Show comment
Hide comment
@bagder

bagder Aug 12, 2016

Member

Well, this isn't strictly because of NTLM as NTLM over proxy works perfectly well without this header. This problem is because of the proxy you're using is relying on this header to mean something more than what it actually does. But sure, we could limit the use of the header to NTLM only to limit the scope of the use.

But again, for documentation and curiosity, do you know which proxy/version this is that causes us this problem?
Member

bagder commented Aug 12, 2016

Well, this isn't strictly because of NTLM as NTLM over proxy works perfectly well without this header. This problem is because of the proxy you're using is relying on this header to mean something more than what it actually does. But sure, we could limit the use of the header to NTLM only to limit the scope of the use.

But again, for documentation and curiosity, do you know which proxy/version this is that causes us this problem?
@fajan

This comment has been minimized.

Show comment
Hide comment
@fajan

fajan Aug 12, 2016

Okay, I missunderstood.
Squid Cache: Version 2.7.5 as the header said.
Meanwhile I found that it also accepts the --proxy-header Connection:keep-alive, which should solve the other issue's as well. "Go's included ReverseProxy removes hop-by-hop HTTP headers, but forgot to remove the ancient and never-standardized "Proxy-Connection" hop-by-hop header."
I don't really know how other proxies handle to this.

 curl --proxy-ntlm --proxy proxy.estest.intra:3128 --proxy-header Connection:keep-alive --proxy-user MyDomain/ntlmtest:test -v https://example.com/file
*   Trying 101.102.103.104...
* Connected to proxy.estest.intra (101.102.103.104) port 3128 (#0)
* Establish HTTP proxy tunnel to example.com:443
* Proxy auth using NTLM with user 'MyDomain/ntlmtest'
> CONNECT example.com:443 HTTP/1.1
> Host: example.com:443
> Proxy-Authorization: NTLM TlRMTVNT...AAAAAA=
> User-Agent: curl/7.50.1
> Connection:keep-alive
>
< HTTP/1.0 407 Proxy Authentication Required
< Server: squid/2.7.STABLE5
< Date: Fri, 12 Aug 2016 09:27:51 GMT
< Content-Type: text/html
< Content-Length: 1330
< X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
< Proxy-Authenticate: NTLM TlRMTVNTU......gAdQAAAAAA
< X-Cache: MISS from intra.my.domain.com
< X-Cache-Lookup: NONE from intra.my.domain.com:3128
< Via: 1.0 intra.my.domain.com:3128 (squid/2.7.STABLE5)
< Connection: keep-alive
< Proxy-Connection: keep-alive
<
* Ignore 1330 bytes of response-body
* TUNNEL_STATE switched to: 0
* Establish HTTP proxy tunnel to example.com:443
* Proxy auth using NTLM with user 'MyDomain/ntlmtest'
> CONNECT example.com:443 HTTP/1.1
> Host: example.com:443
> Proxy-Authorization: NTLM TlRMTVNTUAA......ISS1GRUpMMTM=
> User-Agent: curl/7.50.1
> Connection:keep-alive
>
< HTTP/1.1 200 Connection established
<
* Proxy replied OK to CONNECT request

fajan commented Aug 12, 2016

Okay, I missunderstood.
Squid Cache: Version 2.7.5 as the header said.
Meanwhile I found that it also accepts the --proxy-header Connection:keep-alive, which should solve the other issue's as well. "Go's included ReverseProxy removes hop-by-hop HTTP headers, but forgot to remove the ancient and never-standardized "Proxy-Connection" hop-by-hop header."
I don't really know how other proxies handle to this.

 curl --proxy-ntlm --proxy proxy.estest.intra:3128 --proxy-header Connection:keep-alive --proxy-user MyDomain/ntlmtest:test -v https://example.com/file
*   Trying 101.102.103.104...
* Connected to proxy.estest.intra (101.102.103.104) port 3128 (#0)
* Establish HTTP proxy tunnel to example.com:443
* Proxy auth using NTLM with user 'MyDomain/ntlmtest'
> CONNECT example.com:443 HTTP/1.1
> Host: example.com:443
> Proxy-Authorization: NTLM TlRMTVNT...AAAAAA=
> User-Agent: curl/7.50.1
> Connection:keep-alive
>
< HTTP/1.0 407 Proxy Authentication Required
< Server: squid/2.7.STABLE5
< Date: Fri, 12 Aug 2016 09:27:51 GMT
< Content-Type: text/html
< Content-Length: 1330
< X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0
< Proxy-Authenticate: NTLM TlRMTVNTU......gAdQAAAAAA
< X-Cache: MISS from intra.my.domain.com
< X-Cache-Lookup: NONE from intra.my.domain.com:3128
< Via: 1.0 intra.my.domain.com:3128 (squid/2.7.STABLE5)
< Connection: keep-alive
< Proxy-Connection: keep-alive
<
* Ignore 1330 bytes of response-body
* TUNNEL_STATE switched to: 0
* Establish HTTP proxy tunnel to example.com:443
* Proxy auth using NTLM with user 'MyDomain/ntlmtest'
> CONNECT example.com:443 HTTP/1.1
> Host: example.com:443
> Proxy-Authorization: NTLM TlRMTVNTUAA......ISS1GRUpMMTM=
> User-Agent: curl/7.50.1
> Connection:keep-alive
>
< HTTP/1.1 200 Connection established
<
* Proxy replied OK to CONNECT request

@bagder bagder closed this in fa6b6f1 Aug 16, 2016

@bagder

This comment has been minimized.

Show comment
Hide comment
@bagder

bagder Aug 16, 2016

Member

Thanks for this, should be back and functional in git master now!
Member

bagder commented Aug 16, 2016

Thanks for this, should be back and functional in git master now!

@lock lock bot locked as resolved and limited conversation to collaborators May 7, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.