DNS server update causes problem

[Jyoti-bdk]

I did this

I have an application running on Free BSD on a i386 machine.The application wakes up every 10 seconds and tries to do a curl_easy_perform() .

1-At the beginning , I don't have the DNS server config on my system , hence curl_easy_perform fails (ON DNS resolution) .
2- Now I added the DNS server config and after that ping/host command etc works on that domain name

ping google.com
PING6(56=40+8+8 bytes) :: --> 2404:6800:4003:804::200e

host google.com
google.com has address 172.217.160.14

However curl_easy_perform() keeps failing for the DNS resolution.
The problem resolves only when I restart my application after adding the DNS server configuration on the system .
Please note I have not done curl_global_cleanup() on failure , hence curl_global_init() won't be done on every attempt .I am wondering if that causes some missing initializations with respect to DNS as the DNS server config was missing when curl_easy_init() (and hence implicit curl_global_init() ) was done for the first time .

cat /etc/nsswitch.conf

nsswitch.conf - name service switch configuration file

$Id:$

passwd: files sdk

Code

static size_t
write_data_in_mem (void *buffer, size_t size, size_t nmemb, void *userp)
{
    size_t           realsize = size * nmemb;
    struct memStruct *in_mem = (struct memStruct *) userp;

    in_mem->buff = realloc(in_mem->buff, in_mem->buff_size + realsize + 1);
    if (in_mem->buff == NULL) {
	syslog(LOG_ERR, "failed to allocate %d bytes for buffer",
	       realsize+1);
	return 0;
    }

    bzero(&(in_mem->buff[in_mem->buff_size - 1]), realsize+1);
    memcpy(&(in_mem->buff[in_mem->buff_size - 1]), buffer, realsize);
    in_mem->buff_size += realsize;
    in_mem->buff[in_mem->buff_size] = 0;

    return realsize;
}

static CURL*
setup_curl_options (char *login_passwd, void *write_func,
			 void *hdrdata, void *write_data,
			 char *server_url)
{
    CURL   *curl_handle = NULL;
    object_t     *obj = get_object();
    long port = obj->curr_port;
    unsigned char *cert = obj->curr_cert;
    boolean is_https = FALSE;
    char *real_url = NULL;

    curl_handle = curl_easy_init();
    if (!curl_handle) {
	PRINT("failed to initialize curl obj");
	return NULL;
    }

    
    curl_easy_setopt(curl_handle, CURLOPT_FOLLOWLOCATION, 1L);
   
    curl_easy_setopt(curl_handle, CURLOPT_CONNECTTIMEOUT, 10L);

  
    if (strstr(server_url, "https://") != NULL) {
	is_https = TRUE;
	curl_easy_setopt(curl_handle, CURLOPT_CAINFO, obj->ca_cert_file);
    }

    if (is_https ==TRUE) {
	/* SSL options */
	curl_easy_setopt(curl_handle, CURLOPT_SSL_VERIFYPEER , 1);
	curl_easy_setopt(curl_handle, CURLOPT_SSL_VERIFYHOST , 2);
    }

    /* providing login/passwd */
    curl_easy_setopt(curl_handle, CURLOPT_USERPWD, login_passwd);
    /* send the output to particular function */
    curl_easy_setopt(curl_handle, CURLOPT_WRITEFUNCTION, write_func);
   /* pass structure to write function */
    curl_easy_setopt(curl_handle, CURLOPT_HEADERDATA, (void *)hdrdata);
    curl_easy_setopt(curl_handle, CURLOPT_WRITEDATA, (void *)write_data);

  
    if (NULL != (real_url = get_complete_url_str(server_url, is_https))) {
	curl_easy_setopt(curl_handle, CURLOPT_URL, real_url);
	free(real_url);
    } else {
	curl_easy_setopt(curl_handle, CURLOPT_URL, server_url);
    }

    PRINT("Current Port:%ld", port);
    if (port) {
	curl_easy_setopt(curl_handle, CURLOPT_PORT, port);
    }

    return curl_handle;
}

int contact_server (const char* name, char *login_passwd, char * server_url )
 {
   
    CURL              *curl_handle = NULL;
    CURLcode          res = CURLE_COULDNT_CONNECT;
    struct memStruct  header = {NULL, 0};
    struct memStruct  body = {NULL, 0};
    long              resp_code = 0;
   int                 retval = -1;
   

   
    /* init buffer to save server response */
    header.buff = calloc(1, 1);
    header.buff_size = 1;
    body.buff = calloc(1, 1);
    body.buff_size = 1;

    
    curl_handle = setup_curl_options(login_passwd, write_data_in_mem,
					  (void*)&header, (void*)&body,
					  server_url);
    if (!curl_handle) {
	
	PRINT("failed to initialize curl obj");
        free(header.buff);
        free(body.buff);
	return retval;
    }

    res = curl_easy_perform(curl_handle);

    /* Check for errors */
    if (res != CURLE_OK) {
	PRINT("curl_easy_perform() failed: %s\n",
                         curl_easy_strerror(res));
        retval = -1;
        
    }
    } else {
       curl_easy_getinfo(curl_handle, CURLINFO_RESPONSE_CODE, &resp_code);
       retval = 0 ;
    }  
      free(header.buff);
      free(body.buff);
      if (curl_handle) {	/* cleanup */
	curl_easy_cleanup(curl_handle);
      }
      return retval;
}



thread_main(void) 
{ 
  int retval = -1;

  while(1) 
  { 
    retval = contact_server("server1", "pwd", "https://google.com")
    if(retval == -1)
    {
        sleep(10);
    }
    else {
         break;
    }
  }
}

I expected the following

curl/libcurl version

7.43.0
[curl -V output]

operating system

FreeBSD 6

[bagder]

I presume this is built to use the stock resolver synchronously or threaded? (you didn't answer the question for curl -V output). While you're using a fairly old libcurl version, I don't think this issue has been addressed in the many releases after yours.

As a work-around, can you try calling res_init() when it fails to resolve the host and see if that corrects the situation for the next attempt?

[Jyoti-bdk]

Thanks for the quick response !!
Here's the o/p of curl -V

libcurl/7.43.0 OpenSSL/1.0.2k
Protocols: dict file ftp gopher http https imap imaps pop3 pop3s rtsp smtp smtps telnet tftp
Features: IPv6 Largefile UnixSockets

I tried the suggested workaround and it seems to be working fine after that .
Can I go ahead with this as the final fix ?
I am planning to call res_init() everytime curl_easy_perform() returns CURLE_COULDNT_RESOLVE_HOST .

Kindly advise .

[bagder]

Yes, I think that's an excellent work-around. Ideally curl should do that res_init() call and retry the name resolve itself before returning the error. I've mentioned that suggestion to the TODO file now so that it won't be forgotten, as I don't think anyone is likely to work on that right now.

[Jyoti-bdk]

Thank you so much !!! Really appreciate the help ,I will go ahead and add this in my application .

[bagder]

This issue is now mentioned in the TODO to make it possible for others to find and help out with. It is not going to be fixed in the short term so this issue is hereby closed.

Thanks for the report!

[timestee]

Is it also suitable for mobile devices? Using libcurl

[jay]

Is it also suitable for mobile devices? Using libcurl

I'm not sure what you're asking or how it relates to this issue. If you have a question please ask it on the mailing list and be sure to give us as much detail as possible.

[Jyoti-bdk]

Hi ,
I tried the suggested fix of calling res_init() ,every time curl_easy_perform() returns CURLE_COULDNT_RESOLVE_HOST , it resolved the problem as I mentioned earlier but the problem is ,
it creates a socket and doesn't close it with every invocation. This over a sort period of time eats up all the file descriptors in the system .

Also if I remove it as such each curl_easy_perform() invocation opens up a socket and don't close it eating up the file descriptors causing the same problem .The below stack trace suggests the path in which the socket is created

Breakpoint 1, 0x28ffa98c in socket () from /usr/lib/libc.so.6
(gdb) bt
#0 0x28ffa98c in socket () from /usr/lib/libc.so.6
#1 0x28fcc32c in res_send () from /usr/lib/libc.so.6
#2 0x28fd7938 in freeaddrinfo () from /usr/lib/libc.so.6
#3 0x3fdfc228 in ?? ()

Here's the fstat o/p showing the fd consumption with just the easy_perform without the res_init

---

fstat -p 43722
root connect-app 43722 3* local dgram c8c74120 <-> c808b3f0
root connect-app 43722 4 /mfs 43915 -rw-r--r-- 6 rw
root connect-app 43722 5* local stream c795d900
root connect-app 43722 7* local stream c8b51510
root connect-app 43722 8* kqueue pending_events:0
root connect-app 43722 9* kqueue pending_events:0
root connect-app 43722 10 /cf/var 400492 -rw-r----- 616412 w
root connect-app 43722 11* kqueue pending_events:0
root connect-app 43722 12* kqueue pending_events:0
root connect-app 43722 13* kqueue pending_events:0
root connect-app 43722 14* kqueue pending_events:0
root connect-app 43722 15* kqueue pending_events:0
root connect-app 43722 16* kqueue pending_events:0
root connect-app 43722 17* kqueue pending_events:0
root connect-app 43722 18* internet dgram udp c7a72990
root connect-app 43722 19* kqueue pending_events:0

fstat -p 43357 >>> this is with res_init

---

root connect-app 43357 11* kqueue pending_events:0
root connect-app 43357 12* internet dgram udp c7989440
root connect-app 43357 13* kqueue pending_events:0
root connect-app 43357 14* internet dgram udp c7989990>>>>>>These ones are seemingly from res_init call
root connect-app 43357 15* kqueue pending_events:0
root connect-app 43357 16* internet dgram udp c797f220
root connect-app 43357 17* kqueue pending_events:0
root connect-app 43357 18* internet dgram udp c7a72bb0
root connect-app 43357 19* internet dgram udp c7abd220

My application is calling curl_easy_perform in a loop every 10 seconds till it succeeds .So this builds up really fast .I noticed if no DNS servers are configured in the system , this does not happen .But if the DNS server(s) are configured but still the address resolution keeps failing , I see the above issue .Please help !

Thanks & Regards
-Jyoti