libcurl: 6765e6d breaks dns resolution in my setup with c-ares

[ernstp]

I did this

Curl compiled for arm with yocto like this:
Host setup: arm-fslc-linux-gnueabi
Install prefix: /usr
Compiler: arm-fslc-linux-gnueabi-gcc -mthumb -mfpu=neon -mfloat-abi=hard -mcpu=cortex-a7 -fstack-protector-strong -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security --sysroot=/work/build/tmp/work/cortexa7t2hf-neon-fslc-linux-gnueabi/curl/7.64.1-r0/recipe-sysroot
CFLAGS: -O2 -pipe -feliminate-unused-debug-types -fmacro-prefix-map=/work/build/tmp/work/cortexa7t2hf-neon-fslc-linux-gnueabi/curl/7.64.1-r0=/usr/src/debug/curl/7.64.1-r0 -fdebug-prefix-map=/work/build/tmp/work/cortexa7t2hf-neon-fslc-linux-gnueabi/curl/7.64.1-r0=/usr/src/debug/curl/7.64.1-r0 -fdebug-prefix-map=/work/build/tmp/work/cortexa7t2hf-neon-fslc-linux-gnueabi/curl/7.64.1-r0/recipe-sysroot= -fdebug-prefix-map=/work/build/tmp/work/cortexa7t2hf-neon-fslc-linux-gnueabi/curl/7.64.1-r0/recipe-sysroot-native= -Werror-implicit-function-declaration -Wno-system-headers
CPPFLAGS:
LDFLAGS: -Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed -fstack-protector-strong -Wl,-z,relro,-z,now
LIBS: -lcares

curl version: 7.64.1
SSL: no (--with-{ssl,gnutls,nss,polarssl,mbedtls,cyassl,schannel,secure-transport,mesalink,amissl} )
SSH: no (--with-libssh2)
zlib: no (--with-zlib)
brotli: no (--with-brotli)
GSS-API: no (--with-gssapi)
TLS-SRP: no (--enable-tls-srp)
resolver: c-ares
IPv6: no (--enable-ipv6)
Unix sockets: enabled
IDN: no (--with-{libidn2,winidn})
Build libcurl: Shared=yes, Static=no
Built-in manual: no (--enable-manual)
--libcurl option: no
Verbose errors: enabled (--disable-verbose)
Code coverage: disabled
SSPI: no (--enable-sspi)
ca cert bundle: no
ca cert path:
ca fallback:
LDAP: no (--enable-ldap / --with-ldap-lib / --with-lber-lib)
LDAPS: no (--enable-ldaps)
RTSP: no (--enable-rtsp)
RTMP: no (--with-librtmp)
Metalink: no (--with-libmetalink)
PSL: no (--with-libpsl)
Alt-svc: no (--enable-alt-svc)
HTTP2: disabled (--with-nghttp2)
Protocols: FILE FTP HTTP
Features: UnixSockets AsynchDNS

I expected the following

I do something like this:
curl_easy_setopt(curl_master, CURLOPT_DNS_SERVERS, "1.2.3.4,1.2.3.5");
new_handle = curl_easy_duphandle(curl_master);
setup request on new_handle
curl_easy_perform(new_handle);

• Could not resolve: foo.bar.stuff (Domain name not found)
• Closing connection 0
  curl_easy_perform failed: Couldn't resolve host name (6)

curl/libcurl version

I notice this when upgrading from libcurl 7.62 to 7.64. I took a guess and reverted 6765e6d and that made it work again.
My setup also doesn't work with 7.68.

operating system

Linux Yocto 2.7 kernel 4.14 armv7l

[bagder]

Can you be more specific? Can you deduct what causes the issue and produce a full demo source code that reproduces the issue for you when you run it? That way we would a good way to debug this. Simply reverting that commit is too crude and will only bring back the bug it fixed.

[ernstp]

Sure, I'll try to make a small demo. Sharing what I've done so far and looking for possible tips or clues.

[ernstp]

So
curl_easy_setopt(curl_master, CURLOPT_DNS_SERVERS, "1.2.3.4,1.2.3.5");
new_handle = curl_easy_duphandle(curl_master);
curl_easy_perform(new_handle);

...doesn't work anymore, but:

new_handle = curl_easy_duphandle(curl_master);
curl_easy_setopt(curl_master, CURLOPT_DNS_SERVERS, "1.2.3.4,1.2.3.5");
curl_easy_perform(new_handle);

...still works.

[ernstp]

Ok this was actually pretty easy to write a testcase for.

This is the output I expect:

LD_LIBRARY_PATH=/home/ernsjo/curltest/install-762/lib/ ./test 
hello libcurl/7.62.0-DEV OpenSSL/1.0.2n zlib/1.2.11 c-ares/1.14.0 libidn2/2.0.4
* Could not resolve: really.not.a.server (Could not contact DNS servers)
* Closing connection 0

However with 7.63+ I get:

LD_LIBRARY_PATH=/home/ernsjo/curltest/install/lib/ ./test 
hello libcurl/7.69.0-DEV OpenSSL/1.0.2n zlib/1.2.11 c-ares/1.14.0 libidn2/2.0.4
* Could not resolve: really.not.a.server (Domain name not found)
* Closing connection 0

Here it's forgotten all about my special DNS server.

test.c.txt

[ernstp]

Oh I didn't realize there actually is a DNS server on 9.9.9.9, but the port is wrong at least. :-)
Bugs in test.c aside, we want the "Could not contact DNS servers" print.

[bagder]

Clearly, options in the CURLOPT_DNS_* family don't survive a call to curl_easy_duphandle() so when you use the duplicate you get the c-ares default DNS settings. Are you saying this actually did the right thing back in 7.63.0 ?

[ernstp]

In 7.62.0 it did the right thing yes. In 7.63.0 and higher it does not work.

[bagder]

Are you sure this is actually the same problem then? Because libcurl never duplicated the CURLOPT_DNS_* options when you called curl_easy_duphandle() which is quite easy to verify by reading the code.

[ernstp]

Well you can see that test.c prints "Could not contact DNS servers" with 7.62.

Ok so you're saying that I'm using the api in a wrong way then? I haven't been able to clearly find this in the documentation (which is otherwise very thorough).

[bagder]

This code:

  curl_master = curl_easy_init();
  curl_easy_setopt(curl_master, CURLOPT_URL, "https://curl.haxx.se/");
  curl_easy_setopt(curl_master, CURLOPT_DNS_SERVERS, "1.2.3.4,1.2.3.5");
  new_handle = curl_easy_duphandle(curl_master);
  ret = curl_easy_perform(new_handle);

Will simply use the default name server c-ares figures out, not the one set in the master handle.

Thus a simpler code that does the same thing is:

  curl_master = curl_easy_init();
  curl_easy_setopt(curl_master, CURLOPT_URL, "https://curl.haxx.se/");
  ret = curl_easy_perform(curl_master);

[ernstp]

Will simply use the default name server c-ares figures out, not the one set in the master handle.

Not with 7.62, I clearly see that it uses the one from the master handle.

[bagder]

Ok, I read you. Will dig deeper and see what I'm missing...