LDAPS not possible with MAC and Windows with Certificate-Based Authentication

[kheinrich188]

I did this

.\curl.exe -v -k --insecure --cert "C:\xxx\crt.pem" --key "C:\xxx\key.pem" --url "ldaps://xxx.xx.xxx.xxx:636/dc=data,dc=vzd?cn,mail,organization,displayName,specialization,professionOID,l,postalCode,street,personalEntry,entryType?sub?(&(%7C(sn=TEST*)(givenName=TEST*)(mail=TEST*)(domainId=TEST*)))"
*   Trying xxx.xx.xxx.xxx:636...
* Connected to xxx.xx.xxx.xxx (xxx.xx.xxx.xxx) port 636 (#0)
* LDAP local: LDAP Vendor = Microsoft Corporation. ; LDAP Version = 510
* LDAP local: ldaps://xxx.xx.xxx.xxx:636/dc=data,dc=vzd?cn,mail,organization,displayName,specialization,professionOID,l,postalCode,street,personalEntry,entryType?sub?(&(%7C(sn=TEST*)(givenName=TEST*)(mail=TEST*)(domainId=TEST*)))
* LDAP local: trying to establish encrypted connection
* LDAP local: bind via ldap_win_bind Server heruntergefahren
* Closing connection 0
curl: (38) LDAP local: bind via ldap_win_bind Server heruntergefahren

I expected the following for Windows

Output of the same URL Request with Mac (version listed below):

curl -v -k --insecure --cert "/Applications/xxx/crt.pem" --key "/Applications/xxxx/key.pem" --url "ldaps://xxx.xx.xxx.xx:636/dc=data,dc=vzd?cn,mail,organization,displayName,specialization,professionOID,l,postalCode,street,personalEntry,entryType?sub?(&(|(sn=TEST*)(givenName=TEST*)(mail=TEST*)(domainId=TEST*)))"
*  Trying xxx.xx.xxx.xx:636...
* Connected to xxx.xx.xxx.xx (xxx.xx.xxx.xx) port 636 (#0)
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Request CERT (13):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS handshake, CERT verify (15):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* Server certificate:
* subject: XXX
* start date: Dec 2 10:07:11 2020 GMT
* expire date: Jul 29 15:20:51 2025 GMT
* issuer: XXX
* SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
* LDAP local: ldaps://xxx.xx.xxx.xx:636/dc=data,dc=vzd?cn,mail,organization,displayName,specialization,professionOID,l,postalCode,street,personalEntry,entryType?sub?(&(|(sn=TEST*)(givenName=TEST*)(mail=TEST*)(domainId=TEST*)))
DN: uid=aa894e23-6899-4c25-806a-e2af50a5e750,dc=data,dc=vzd... Data loaded successfully 

curl/libcurl version (Windows)

curl 7.84.0 (i686-w64-mingw32) libcurl/7.84.0 OpenSSL/3.0.5 (Schannel) zlib/1.2.12 brotli/1.0.9 zstd/1.5.2 libidn2/2.3.3 libssh2/1.10.0 nghttp2/1.48.0 ngtcp2/0.7.0 nghttp3/0.6.0 libgsasl/1.10.0
Release-Date: 2022-06-27
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli gsasl HSTS HTTP2 HTTP3 HTTPS-proxy IDN IPv6 Kerberos Largefile libz MultiSSL NTLM SPNEGO SSL SSPI threadsafe TLS-SRP UnixSockets zstd

curl/libcurl version (Mac) (working version)

curl 7.84.0 (aarch64-apple-darwin21.5.0) libcurl/7.84.0 (SecureTransport) OpenSSL/1.1.1q zlib/1.2.11 brotli/1.0.9 zstd/1.5.2 libidn2/2.3.3 libssh2/1.10.0 nghttp2/1.48.0 librtmp/2.3 OpenLDAP/2.6.3
Release-Date: 2022-06-27
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp 
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz MultiSSL NTLM NTLM_WB SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd

operating system

Microsoft Windows 10 Enterprise N
10.0.19044 Build 19044

[kheinrich188]

Update: I installed latest version on my Mac.

curl/libcurl version

curl 7.85.0 (aarch64-apple-darwin21.6.0) libcurl/7.85.0 (SecureTransport) OpenSSL/1.1.1q zlib/1.2.11 brotli/1.0.9 zstd/1.5.2 libidn2/2.3.3 libssh2/1.10.0 nghttp2/1.50.0 librtmp/2.3
Release-Date: 2022-08-31
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp 
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz MultiSSL NTLM NTLM_WB SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd

what I got

curl -v -k --insecure --cert "/Applications/xxx/crt.pem" --key "/Applications/xxxx/key.pem" --url "ldaps://xxx.xx.xxx.xx:636/dc=data,dc=vzd?cn,mail,organization,displayName,specialization,professionOID,l,postalCode,street,personalEntry,entryType?sub?(&(|(sn=TEST*)(givenName=TEST*)(mail=TEST*)(domainId=TEST*)))"
*   Trying xxx.xx.xxx.xx:636...
* Connected to xxx.xx.xxx.xx (xxx.xx.xxx.xx) port 636 (#0)
* LDAP local: LDAP Vendor = OpenLDAP ; LDAP Version = 20603
* LDAP local: ldaps://xxx.xx.xxx.xx:636/dc=data,dc=vzd?cn,mail,organization,displayName,specialization,professionOID,l,postalCode,street,personalEntry,entryType?sub?(&(|(sn=TEST*)(givenName=TEST*)(mail=TEST*)(domainId=TEST*)))
* LDAP local: trying to establish encrypted connection
* LDAP local: bind via ldap_simple_bind_s Can't contact LDAP server
* Closing connection 0
curl: (38) LDAP local: bind via ldap_simple_bind_s Can't contact LDAP server

operating system

MacOS 12.6 (21G115)
Apple M1 Max

[vszakats]

The Windows build is using the Windows system LDAP library wldap32, while the macOS one links to OpenLDAP. libcurl has separate interface implementations for these LDAP-backends. Also, wldap32 implements it's own TLS support and isn't using OpenSSL.

It means OpenSSL (or it's specific version) is most likely not a factor here.

[kheinrich188]

@vszakats thank you for your quick response.
What would you suggest as a short-term solution?
Unfortunately, we currently cannot use any alternative other than CURL to query data from LDAP cause there is no cli that we know that runs under MAC and Windows that we can use to do this.

Do you have any idea why we got a successful call on MAC with version 7.84 but not with the current version 7.85?

[vszakats]

@kheinrich188 I had only been using LDAPS without cert-based-auth, so can't offer much hints. But, can you try if the curl 7.84.0 Windows build also reproduces the issue? You can find it here: https://curl.se/windows/dl-7.84.0_9/

Mac uses OpenLDAP with a different TLS stack, so the codepath is completely different from the Windows case.

[kheinrich188]

@vszakats Thank you again for your quick reply!
I tried it with nearly every version.
Unfortunately is the result still the same also with the above linked one.

...
* LDAP local: trying to establish encrypted connection
* LDAP local: bind via ldap_win_bind Server heruntergefahren
* Closing connection 0
curl: (38) LDAP local: bind via ldap_win_bind Server heruntergefahren

For the Mac Topic:
Ok hm yeah would be interesting what changed that now this also not working anymore cause we currently can also not rollback to the 7.84 via homebrew what would helped us a lot

[vszakats]

@kheinrich188 No worries at all. If this doesn't work with any recent curl + WinLDAP combos, we can rule out a curl regression.

Skimming curl sources, I can't find a place where the cert / key is passed to WinLDAP. Nor much evidence that WinLDAP can accept one. Maybe it's done outside an API with the system certificate store?

[ With some luck, it's possible to revert the Homebrew formula (curl.rb) to a previous version and build it from source with brew install --build-from-source ./curl.rb. ]

[kheinrich188]

@vszakats
Windows:
We found this: https://learn.microsoft.com/en-us/windows/win32/api/winldap/nc-winldap-queryclientcert
Are you sure that cert + key are not passed currently?

Mac:
After your suggestion we managed it to work again with 7.84.

[vszakats]

@kheinrich188 👍 for the Mac. Good find with the Windows API! The page says LDAP_OPT_CLIENT_CERTIFICATE needs to be set as an option, and that is not being set by curl. So this feature might be possible, but needs to be implemented.

[ Yesterday I did git grep clientcert, which has no hits in lib/ldap.c (the curl WinLDAP interface). ]

[kheinrich188]

Thank you very much for your feedback.
We would of course be happy to make ourselves available to test a pre-release as soon as development has started with this feature. Provided the feature is adapted by you 😇

[wjeral]

Hello, just wanted to comment that this issue exists for me on RHEL8 as well, with both 7.84, 7.85, and 7.86. The previous version I had installed was 7.78 and it works. 7.78 calls ldap_result in oldap_recv with LDAP_MSG_RECEIVED and subsequently loops through all messages, while later versions (not sure when it was changed) call ldap_result with LDAP_MSG_ONE, and set the result from oldap_recv to CURLE_AGAIN, indicating more stuff (the result msg) is still expected. But nothing interprets that result so curl just sits forever unless max-time is set on the call. There may be more than one way to fix it, but if CURLE_AGAIN is checked at the end of the do loop in readwrite_data, or if conn->cselect_bits is set to CURL_CSELECT_IN in oldap_recv after processing the ldap entry message, oldap_recv will get called again and pull in the result message. I couldn't see how to get it working otherwise.

For 7.85 and 86, it's the same, but you now have to define export USE_OPENLDAP=1 in either CFLAGS or CPPFLAGS before running configure in order for the oldap modules in openldap.c to be executed. It used to be the default but I think it was changed for 7.85. The default now is for the path to go through ldap.c, and, for me, at the moment, that same curl call results in the "bind via ldap_simple_bind_s Can't contact LDAP server" message. I haven't done much to trace it out, but I think it may have to do with the CA's - not sure yet.

My plan is to deploy 7.86 with USE_OPENLDAP=1 until I can get the ldap.c path working.