curl returns correct LDAP result but after that hangs

[alliquot]

Hello,

I faced a strange issue while querying LDAP Active Directory via curl: directly and via R-language wrapper - RCurl.

When I run RCurl under R session on Windows, my query is evaluated and result is returned immediately. Everything is fine.

When I do the same under R on Linux Debian, it hangs. So I've run the query via curl from the Linux command line. It turned out that the result is returned correctly (all information is correct), but curl is waiting for something. Only Ctrl+C can stop that.

My query:

 LDAP://x.x.x.x/dc=company,dc=com?userPrincipalName,msExchShadowGivenName,msExchShadowSn,sAMAccountName?sub?(&(objectCategory=person)(msExchShadowProxyAddresses=*some@email.com*))

My call:

curl -u user@domain:pwd "ldap://ip/......"

Why the query on Windows is evaluated successfully and curl returns results immediately, while the same query on Linux makes curl hanged? Should I set some additional options?

This may be important: Debian is running under Hyper-V virtual machine.
The version of curl I'm using is: 7.38.0

Thanks in advance.

EDIT:
adding --max-time 1 partially solves this issue, but this completely unprofessional solution. If the response will come in 2 seconds, it will be lost. And setting the max-time to 2 seconds will make the process waiting always 2 seconds.

[monnerat]

I think you've hit the same problem discussed 10 days ago on the mailing list. Please look at thread http://curl.haxx.se/mail/lib-2016-01/0101.html.
This occurs when:

• The ldap backend used by your curl installation is openldap: probably the case on Debian.
• Queries are made to a server that returns referrals: M$AD is one of them :-(
• External ldap configuration allows automatic referrals chasing.

There is no easy way to support automatic referrals chasing in curl yet.
As a workaround, I can suggest you disable this feature by ldap configuration ("REFERRALS off" in ldap.conf): this will release the hang. Of course, this will also miss the results from referrals, but in the current curl's state, they won't be returned anyway.
I started to investigate to fix this problem, but this is a huge work, so don't expect to have a real fix soon...

[alliquot]

Yes, you are absolutely right. I'm querying the Active Directory service. As for now I switched to the "ldapsearch" (from the openldap-tools package) tool and yes, it certainly displays "#refldap://". So yes, this may cause the problem. But, from the other side, why does it work on Windows? Exactly the same query is working smoothly under RCurl package on Windows.
RCurl is an R-language wrapper for Curl. The same library (as well as the "pure curl") hangs on Debian.

Perhaps the source code of "lpdasearch" tool could help you to address this issue and handle the "referrals"?

Nevertheless, thank you very much for the quick and informative response!

[monnerat]

But, from the other side, why does it work on Windows?

I suppose the W$ curl version uses the ldap backend, not the openldap. I really do not know how referrals are processed in it, if at all.

On linux, if you use the workaround from my 1st reply, it should work.

The openldap backend chases referrals on secondary socket descriptors. Since curl is only able to monitor the primary socket descriptor for input availability, data from the secondary descriptors never trigger an "ldap_result()" call and thus, curl waits for the end-of-data record forever. This is the real problem.

Perhaps the source code of "lpdasearch" tool could help you to address this issue and handle the "referrals"?

I don't think so: ldapsearch has a single thing to do: satisfy your request, so it can handle requests synchronously. On the other hand, libcurl should be able to manage several easy handles at the same time (even if you have only one), so it uses asynchronous requests and select/poll calls to avoid suspension of program on socket reads.

Featuring the monitoring of several sockets for a single easy handle requires huge changes in the libcurl generic core, thus the correct fix for this issue will not be ready tomorrow :-/

[bagder]

This is now "12.1 OpenLDAP hangs after returning results" in the KNOWN_BUGS document.

[larsxschneider]

Thank you! I experienced the same issue and the REFERRALS off configuration fixed it.
Unfortunately, I cannot change the ldap.conf on my production machine. Is there a way to pass the LDAP config REFERRALS off to the curl call?

[larsxschneider]

I found it: LDAPREFERRALS=off curl .... Sorry for the noise!