Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NTLM 401 on HTTPS IIS endpoints #3280

Closed
georgeok opened this issue Nov 15, 2018 · 5 comments

Comments

Projects
None yet
3 participants
@georgeok
Copy link
Contributor

commented Nov 15, 2018

NTLM fails consistently on Windows when targeting https endpoint that are WIA protected. This only happens when "Extended Protection" is set to Accept or Require in the IIS server (Accept is the default). The same endpoint work correct when plain http is in use.

I did this

curl.exe -v -k --ntlm -u user:Password https://my.dc.local/adfs/ls/wia/


*   Trying 10.211.55.65...
* TCP_NODELAY set
* Connected to my.dc.local (10.211.55.65) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / AES128-SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
*  subject: CN=my.dc.local
*  start date: Nov 13 17:36:50 2018 GMT
*  expire date: Nov 13 00:00:00 2019 GMT
*  issuer: CN=my.dc.local
*  SSL certificate verify result: unable to get local issuer certificate (20), c
ontinuing anyway.
* Server auth using NTLM with user 'george'
> GET /adfs/ls/wia/ HTTP/1.1
> Host: my.dc.local
> Authorization: NTLM TlR---------------------------------------------AAAADw==
> User-Agent: curl/7.62.0
> Accept: */*
>
< HTTP/1.1 401 Unauthorized
< Content-Type: text/html; charset=us-ascii
< Server: Microsoft-HTTPAPI/2.0
< WWW-Authenticate: NTLM TlRMTVNTUAACAAAABAAEADgAAAAFgomixeUPaN/XXvgAAAAAAAAAAGI
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
----------------------------------------------------------------------------AQAAAAA=
< Date: Thu, 15 Nov 2018 19:37:18 GMT
< Content-Length: 341
<
* Ignoring the response-body
* Connection #0 to host my.dc.local left intact
* Issue another request to this URL: 'https://my.dc.local/adfs/ls/wia/'
* Found bundle for host my.dc.local: 0x1882600 [can pipeline]
* Could pipeline, but not asked to!
* Re-using existing connection! (#0) with host my.dc.local
* Connected to my.dc.local (10.211.55.65) port 443 (#0)
* Server auth using NTLM with user 'george'
> GET /adfs/ls/wia/ HTTP/1.1
> Host: my.dc.local
> Authorization: NTLM TlRMTVNTUAADAAAAGAAYAHoAAAAKAQoBkgAAAAAAAABYAAAADAAMAFgAAA
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
AAAAAAAAAAAA==
> User-Agent: curl/7.62.0
> Accept: */*
>
< HTTP/1.1 401 Unauthorized
< Content-Type: text/html
< Server: Microsoft-IIS/8.0
* NTLM handshake rejected
* Authentication problem. Ignoring this.
< WWW-Authenticate: NTLM
< X-Powered-By: ASP.NET
< Date: Thu, 15 Nov 2018 19:37:18 GMT
< Content-Length: 1293
<
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/x
html1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"/>
<title>401 - Unauthorized: Access is denied due to invalid credentials.</title>
<style type="text/css">
<!--
body{margin:0;font-size:.7em;font-family:Verdana, Arial, Helvetica, sans-serif;b
ackground:#EEEEEE;}
fieldset{padding:0 15px 10px 15px;}
h1{font-size:2.4em;margin:0;color:#FFF;}
h2{font-size:1.7em;margin:0;color:#CC0000;}
h3{font-size:1.2em;margin:10px 0 0 0;color:#000000;}
#header{width:96%;margin:0 0 0 0;padding:6px 2% 6px 2%;font-family:"trebuchet MS
", Verdana, sans-serif;color:#FFF;
background-color:#555555;}
#content{margin:0 0 0 2%;position:relative;}
.content-container{background:#FFF;width:96%;margin-top:8px;padding:10px;positio
n:relative;}
-->
</style>
</head>
<body>
<div id="header"><h1>Server Error</h1></div>
<div id="content">
 <div class="content-container"><fieldset>
  <h2>401 - Unauthorized: Access is denied due to invalid credentials.</h2>
  <h3>You do not have permission to view this directory or page using the creden
tials that you supplied.</h3>
 </fieldset></div>
</div>
</body>
</html>
* Connection #0 to host my.dc.local left intact

I expected the following

A 200 response from the server. Again everything works fine with plain HTTP

curl/libcurl version

I have tried curl 7.62, 7.55.1 and 7.57.

[curl -V output]

curl 7.55.1 (Windows) libcurl/7.55.1 WinSSL
Release-Date: [unreleased]
Protocols: dict file ftp ftps http https imap imaps pop3 pop3s smtp smtps telnet tftp
Features: AsynchDNS IPv6 Largefile SSPI Kerberos SPNEGO NTLM SSL


curl 7.62.0 (i386-pc-win32) libcurl/7.62.0 OpenSSL/1.1.1 (WinSSL) zlib/1.2.11 brotli/1.0.7 WinIDN libssh2/1.8.0 nghttp2/1.34.0
Release-Date: 2018-10-31
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile SSPI Kerberos SPNEGO NTLM SSL libz brotli TLS-SRP HTTP2 HTTPS-proxy MultiSSL

operating system

Failing behaviour is consistent on Windows 7,8,10.
macOS and Linux work fine.

@danielgustafsson

This comment has been minimized.

Copy link
Member

commented Nov 15, 2018

Please amend this with more information as to what the issue actually is.

@georgeok

This comment has been minimized.

Copy link
Contributor Author

commented Nov 15, 2018

Please amend this with more information as to what the issue actually is.

Sorry for the delay. Just amended the information.

@bagder

This comment has been minimized.

Copy link
Member

commented Nov 15, 2018

These windows builds are built with SSPI enabled, meaning they use native windows function calls for the NTLM magic. That makes this a very windows-specific issue and requires someone to debug this on windows. It would be interesting to learn if someone would try this with a windows-build without SSPI and see if that makes it work.

NTLM debugging is challenging...

@bagder bagder added the help wanted label Nov 15, 2018

@georgeok

This comment has been minimized.

Copy link
Contributor Author

commented Nov 19, 2018

I've just tried with curl 7.59 without the SSPI and everything work as expected. Looks like something is failing only when the SSPI is used

curl 7.59.0 (i686-pc-cygwin) libcurl/7.59.0 OpenSSL/1.0.2p zlib/1.2.11 libidn2/2.0.4 libpsl/0.18.0 (+libidn2/2.0.2) libssh2/1.7.0 nghttp2/1.31.0
Release-Date: 2018-03-14
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: AsynchDNS Debug IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP HTTP2 UnixSockets HTTPS-proxy PSL Metalink```
@georgeok

This comment has been minimized.

Copy link
Contributor Author

commented Nov 23, 2018

After looking deeper to the problem I can see why curl with the SSPI is failing. The problem is the Windows Extended Protection (aka channel binding).
https://blogs.msdn.microsoft.com/openspecification/2013/03/26/ntlm-and-channel-binding-hash-aka-extended-protection-for-authentication/

A solution would be to:

  1. Export the CBT by the Schannel SPI (SSL) with QueryContextAttributes(SECPKG_ATTR_ENDPOINT_BINDINGS). It returns a SecPkgContext_Bindings structure.

  2. Import with the function InitializeSecurityContext in a supplemental buffer of the type SECBUFFER_CHANNEL_BINDINGS when creating the type 3 somewhere message here.

Example implementations:
C:
https://github.com/eurogiciel-oss/freerdp/blob/ff1454a973fa1f5a938dc4af0563e89e95c3489f/libfreerdp/core/gateway/ntlm.c#L221

https://github.com/chromium/chromium/blob/7236fb7258ccb4095b21fd16219236b4835dd849/net/http/http_auth_sspi_win.cc#L376

Python:
https://github.com/brandond/requests-negotiate-sspi/blob/4a115e2cc6ad40929bf0c7722b017a5aee46f5d7/requests_negotiate_sspi/requests_negotiate_sspi.py#L65

I would be to patch and test it with some guidance from someone more experienced with libcurl.

georgeok added a commit to georgeok/curl that referenced this issue Nov 28, 2018

[ntlm] SPPI channel binding suport
Windos extended potection (aka ssl channel binding) is required
to login to ntlm IIS endpoint, otherwise the server return 401
responses.

[Fixes: curl#3280]

@lock lock bot locked as resolved and limited conversation to collaborators Apr 19, 2019

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.