POP3 NTLM authentication got broken in between 7.37 and 7.38

[gyokketto]

I did this

I wanted to connect to our exchange server using POP3. I got a login denied (67) message.

I expected the following

I expected to list/retrieve test emails.

curl/libcurl version

Anything above 7.37 does not work. I tried "official" Win builds from the website. First 7.46 then the current latest 7.47.1. Then I tried it on my Linux box (7.35) and that worked. I had an older build (7.40) (built by myself) that did not work either. I started building all the version between 7.35 and 7.40 and found that 7.38 does not but 7.37 does work. I built the versions in the same way with the same dependencies the only thing that was different from build to build is the curl source code.

If I change the code in 7.38 in pop3.c:
from
/* Calculate the supported authentication mechanism, by decreasing order of
security, as well as the initial response where appropriate */
#if defined(USE_WINDOWS_SSPI)
to
/* Calculate the supported authentication mechanism, by decreasing order of
security, as well as the initial response where appropriate */
#if 0
then it works again.

[curl -V output perhaps?]

BAD:
curl pop3://ex2k7test.ourhost.com -u UserName:Password -v
* Rebuilt URL to: pop3://ex2k7test.ourhost.com/
* Trying 10.1.5.2...
* Connected to ex2k7test.ourhost.com (10.1.5.2) port 110 (#0)
< +OK The Microsoft Exchange POP3 service is ready.
> CAPA
< +OK
< TOP
< UIDL
< SASL NTLM GSSAPI PLAIN
< USER
< STLS
< .
> AUTH GSSAPI
< +
* Closing connection 0
curl: (67) Login denied
GOOD:
>curl pop3://ex2k7test.ourhost.com -u UserName:Password -v
* Rebuilt URL to: pop3://ex2k7test.ourhost.com/
* Hostname was NOT found in DNS cache
* Trying 10.1.5.2...
* Connected to ex2k7test.ourhost.com (10.1.5.2) port 110 (#0)
< +OK The Microsoft Exchange POP3 service is ready.
> CAPA
< +OK
< TOP
< UIDL
< SASL NTLM GSSAPI PLAIN
< USER
< STLS
< .
> AUTH NTLM
< +
> TlRMTVNTUAAB...
< + TlRMTVNTUAAC...
> TlRMTVNTUAAD...
< +OK User successfully logged on.
> LIST
< +OK 2 7804
1 3902
2 3902
* Connection #0 to host ex2k7test.ourhost.com left intact

operating system

Ubuntu 14.04.4 and Windows7

[bagder]

ping @captain-caveman2k can you have a look? It looks like the commit that introduced this change and broke the functionality @leinadn speaks is commit 03f368d which was introduced in the 7.38.0 release.

[captain-caveman2k]

Hi,

My apologies if I am missing something but I don't see what is wrong.

In v7.38 we introduced support for GSSAPI (Kerberos V5) authentication and as the Exchange Server is advertising: NTLM, GSSAPI and PLAIN we are choosing the most secure mechanism as follows:

curl >= 7.38 - GSSAPI
curl < 7.38 - NTLM (as GSSAPI was not supported then)

We can clearly see this in the logs as follows:

BAD:

< SASL NTLM GSSAPI PLAIN 
> AUTH GSSAPI

GOOD:

< SASL NTLM GSSAPI PLAIN 
> AUTH NTLM

I suspect the Kerberos authentication is failing as it is unable to obtain the Security Credentials for the User - Kerberos is a little more picky than NTLM as a Kerberos 5 ticket needs to be obtained before actually sending anything over the wire.

• What format username are you using?
• What URL are you using to contact your Exchange Server?
• What are the registered SPNs for the Exchange Server?

As such I suspect that the call to AcquireCredentialsHandle() in curl_sasl_sspi.c:940 is failing.

If you want to specifically use NTLM you can use the following to request NTLM as opposed to GSSPI:

curl -v pop3://;AUTH=NTLM@mail.example.com --user EXAMPLE\me

Kind Regards

Steve

[gyokketto]

Hi Steve,

Explicitly adding curl_easy_setopt(curl_, CURLOPT_LOGIN_OPTIONS, "AUTH=NTLM"); works as you suggested.

Thank you.

[captain-caveman2k]

Hiya,

Cool - i'm glad NTLM's working for you. You can use GSSAPI with an Exchange Server using curl - in fact it's how I developed the SASL GSSAPI support in curl ;-)

It's just a little more fiddle:

a) You need to include the domain name in the user name (either using down-level format or as an UPN).
b) You can't use a server alias in the URL (like mail.example.com instead of exch-svr01.example.com) unless the SPN is registered for the service (IMAP, POP3, SMTP).

Kind Regards

Steve

[paresy]

This is definitely a backwards compatibility issue!

All users that relied on NTLM authentication (which was working) are now forced to update their configuration, as the newly implemented GSSAPI seems to require a domain name. cURL does not fall back to NTLM if GSSAPI fails. Neither do we have a chance to just disable GSSAPI from the API level. We can just force NTLM, which does not work for all cases.

As cURL strives for backwards compatibility, i hope that you can reconsider reopening the issue.

[bagder]

I'm inclined to agree with you. As GSSAPI was previously ignored curl picked NTLM just fine so by adding support for that we broke how curl used to work against the same server. I'm not sure exactly what the spec says, but I would guess that we could try another if the first method fails. Alternatively, we could alter the priority order of the auth methods or something.

This said, just opening this bug will not magically result in this problem getting fixed. Someone also needs to actually step up and work on the code. I'm opening this now, but I will close it again if it goes stale and then instead add the bug to the KNOWN_BUGS document where old bugs that nobody works on go to grow old...

[paresy]

Thanks for reopening. You can also add SMTP and IMAP labels, as those are affected as well.

Hints to the ideas on how to fix the problem:

1. Altering the priority would fix the problem, but would break the rules of using the "safest" authentication method first. Also breaks for all users that successfully use GSSAPI since the release. I don't know what cURL rules are for this case.

2. Using NTLM after GSSAPI would fix the bug very nicely, but how about a failing NTLM authentication? Will it revert to PLAIN? This sound like a potential downgrade attack.

3. As a workaround: Is it possible to disable GSSAPI at the API level?
   e.x blacklist: curl_easy_setopt(curl_, CURLOPT_LOGIN_OPTIONS, "AUTH=-GSSAPI");
   e.x. whitelist: curl_easy_setopt(curl_, CURLOPT_LOGIN_OPTIONS, "AUTH=NTLM PLAIN");

Regards,
Michael

[iboukris]

In my opinion GSSAPI differs from other auth method, while with other methods if auth failed via NTLM (on supporting server) it'll probably fail with Basic with the same password, but GSSAPI fails if it doesn't have a ticket, without actually trying.

So we could just remove gssapi from supported auth method of this request if it fails to generate a token (I don't think it is a security concern as the user denotes what auth methods to support).

I'm not familiar with POP code but I can picture how it would look in HTTP Negotiate code where we have the same problem basically, see issue #876 for instance.
Note that in the HTTP case, ignoring gssapi would only occur after trying with an additional empty request as only then we try to generate the token but it should work.

[captain-caveman2k]

This fixes the issue for the most part except possibly when a GSS-API based library is used and the user in the credientials cache still fails to generate a token.

As such we could modify the new Curl_auth_user_contains_domain() function futher to attempt to generate a token when using GSS-API.

But at least it is a step in the right direction ;-)