Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

“Could not resolve host” error with curl_multi on cURL version 7.64 #3629

Closed
Schnitzel opened this issue Feb 28, 2019 · 4 comments

Comments

Projects
None yet
2 participants
@Schnitzel
Copy link

commented Feb 28, 2019

Basically copying this over from https://stackoverflow.com/questions/54771917/could-not-resolve-host-error-with-curl-multi-on-curl-version-7-64
I'm not the author of the StackOverflow question, but I have the exact same issue.

I did this

php script:

<?php
$url = "https://www.example.com";

$options = array(
    CURLOPT_RETURNTRANSFER => true,     // don't echo page
    CURLOPT_VERBOSE =>true,
);

$cm = curl_multi_init();


// step 1 of 3 (adding requests)

//request 1
$ch1 = curl_init( $url );
$verboseH1 = fopen('php://temp', 'w+');
$options[CURLOPT_STDERR] = $verboseH1;
curl_setopt_array( $ch1, $options);
curl_multi_add_handle($cm, $ch1);

//request 2
$ch2 = curl_init( $url );
$verboseH2 = fopen('php://temp', 'w+');
$options[CURLOPT_STDERR] = $verboseH2;
curl_setopt_array( $ch2, $options);
curl_multi_add_handle($cm, $ch2);

//request 3
$ch3 = curl_init( $url );
$verboseH3 = fopen('php://temp', 'w+');
$options[CURLOPT_STDERR] = $verboseH3;
curl_setopt_array( $ch3, $options);
curl_multi_add_handle($cm, $ch3);

// step 2 of 3 (executing requests in parallel)
// for more information about these strange loops , check http://php.net/manual/en/function.curl-multi-init.php#115055
$active = null;    
do {
    $mrc = curl_multi_exec($cm, $active);
} while ($mrc == CURLM_CALL_MULTI_PERFORM);

while ($active && $mrc == CURLM_OK) {
    if (curl_multi_select($cm) == -1) {
        usleep(100);
    }
    do {
        $mrc = curl_multi_exec($cm, $active);
    } while ($mrc == CURLM_CALL_MULTI_PERFORM);
}

// step 3 of 3 (getting results)
echo "<h1>Requet 1</h1>\n";
echo "errmsg1: '" .  curl_error($ch1) . "'\n";
rewind($verboseH1);
echo "verbose1:\n" .  stream_get_contents( $verboseH1) . "\n";

echo "===============================================================\n";
echo "<h1>Requet 2</h1>\n";
echo "errmsg2: '" .  curl_error($ch2) . "'\n";
rewind($verboseH2);
echo "verbose2:\n" .  stream_get_contents( $verboseH2) . "\n";

echo "===============================================================\n";
echo "<h1>Requet 3</h1>\n";
echo "errmsg3: '" .  curl_error($ch3) . "'\n";
rewind($verboseH3);
echo "verbose3:\n" .  stream_get_contents( $verboseH3) . "\n";

exit;

output:

<h1>Requet 1</h1>
errmsg1: ''
verbose1:
* Expire in 0 ms for 6 (transfer 0x2b750f0)
* Expire in 1 ms for 1 (transfer 0x2b750f0)
* Expire in 0 ms for 1 (transfer 0x2b750f0)
* Expire in 0 ms for 1 (transfer 0x2b750f0)
*   Trying 93.184.216.34...
* TCP_NODELAY set
* Expire in 149999 ms for 3 (transfer 0x2b750f0)
* Expire in 200 ms for 4 (transfer 0x2b750f0)
* Connected to www.example.com (93.184.216.34) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=Los Angeles; O=Internet Corporation for Assigned Names and Numbers; OU=Technology; CN=www.example.org
*  start date: Nov 28 00:00:00 2018 GMT
*  expire date: Dec  2 12:00:00 2020 GMT
*  subjectAltName: host "www.example.com" matched cert's "www.example.com"
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert SHA2 Secure Server CA
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x2b750f0)
> GET / HTTP/2
Host: www.example.com
Accept: */*

* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 200 
< cache-control: max-age=604800
< content-type: text/html; charset=UTF-8
< date: Tue, 19 Feb 2019 17:01:16 GMT
< etag: "1541025663+ident"
< expires: Tue, 26 Feb 2019 17:01:16 GMT
< last-modified: Fri, 09 Aug 2013 23:54:35 GMT
< server: ECS (dfw/27B2)
< vary: Accept-Encoding
< x-cache: HIT
< content-length: 1270
< 
* Connection #0 to host www.example.com left intact

===============================================================
<h1>Requet 2</h1>
errmsg2: 'Could not resolve host: www.example.com'
verbose2:
* Expire in 0 ms for 6 (transfer 0x2b7a9a0)
* Found bundle for host www.example.com: 0x2b86800 [serially]
* Server doesn't support multi-use (yet)
* Expire in 1 ms for 1 (transfer 0x2b7a9a0)
* Expire in 0 ms for 1 (transfer 0x2b7a9a0)
* Hostname 'www.example.com' was found in DNS cache
* Expire in 0 ms for 1 (transfer 0x2b7a9a0)
*   Trying 93.184.216.34...
* TCP_NODELAY set
* Expire in 149999 ms for 3 (transfer 0x2b7a9a0)
* Expire in 200 ms for 4 (transfer 0x2b7a9a0)
* Connected to www.example.com (93.184.216.34) port 443 (#1)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=Los Angeles; O=Internet Corporation for Assigned Names and Numbers; OU=Technology; CN=www.example.org
*  start date: Nov 28 00:00:00 2018 GMT
*  expire date: Dec  2 12:00:00 2020 GMT
*  subjectAltName: host "www.example.com" matched cert's "www.example.com"
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert SHA2 Secure Server CA
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x2b7a9a0)
> GET / HTTP/2
Host: www.example.com
Accept: */*

* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 200 
< cache-control: max-age=604800
< content-type: text/html; charset=UTF-8
< date: Tue, 19 Feb 2019 17:01:16 GMT
< etag: "1541025663+ident"
< expires: Tue, 26 Feb 2019 17:01:16 GMT
< last-modified: Fri, 09 Aug 2013 23:54:35 GMT
< server: ECS (dfw/562D)
< vary: Accept-Encoding
< x-cache: HIT
< content-length: 1270
< 
* Could not resolve host: www.example.com
* Closing connection 1

===============================================================
<h1>Requet 3</h1>
errmsg3: 'Could not resolve host: www.example.com'
verbose3:
* Expire in 0 ms for 6 (transfer 0x2b80250)
* Found bundle for host www.example.com: 0x2b86800 [serially]
* Server doesn't support multi-use (yet)
* Expire in 1 ms for 1 (transfer 0x2b80250)
* Expire in 0 ms for 1 (transfer 0x2b80250)
* Hostname 'www.example.com' was found in DNS cache
* Expire in 0 ms for 1 (transfer 0x2b80250)
*   Trying 93.184.216.34...
* TCP_NODELAY set
* Expire in 150000 ms for 3 (transfer 0x2b80250)
* Expire in 200 ms for 4 (transfer 0x2b80250)
* Connected to www.example.com (93.184.216.34) port 443 (#2)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=Los Angeles; O=Internet Corporation for Assigned Names and Numbers; OU=Technology; CN=www.example.org
*  start date: Nov 28 00:00:00 2018 GMT
*  expire date: Dec  2 12:00:00 2020 GMT
*  subjectAltName: host "www.example.com" matched cert's "www.example.com"
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert SHA2 Secure Server CA
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x2b80250)
> GET / HTTP/2
Host: www.example.com
Accept: */*

* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 200 
< accept-ranges: bytes
< cache-control: max-age=604800
< content-type: text/html; charset=UTF-8
< date: Tue, 19 Feb 2019 17:01:16 GMT
< etag: "1541025663"
< expires: Tue, 26 Feb 2019 17:01:16 GMT
< last-modified: Fri, 09 Aug 2013 23:54:35 GMT
< server: ECS (dfw/F4AA)
< vary: Accept-Encoding
< x-cache: HIT
< content-length: 1270
< 
* Could not resolve host: www.example.com
* Closing connection 2

Reproduce:

This will upgrade curl to 7.64.0 which has the issue:

docker pull php:7.2-alpine
docker run --rm -it php:7.2-alpine sh
apk upgrade curl
curl https://gist.githubusercontent.com/Schnitzel/ed65648571f25647eb0bc7b2e5a1b998/raw/6876ce533e6085656fac039492027c98372f27bf/test.php > test.php
php test.php

I expected the following

This will keep curl on 7.63.0 which does not have the issue

docker pull php:7.2-alpine
docker run --rm -it php:7.2-alpine sh
curl https://gist.githubusercontent.com/Schnitzel/ed65648571f25647eb0bc7b2e5a1b998/raw/6876ce533e6085656fac039492027c98372f27bf/test.php > test.php
php test.php

curl/libcurl version

$ curl --version
curl 7.64.0 (x86_64-alpine-linux-musl) libcurl/7.64.0 OpenSSL/1.1.1a zlib/1.2.11 libssh2/1.8.0 nghttp2/1.35.1
Release-Date: 2019-02-06
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP HTTP2 UnixSockets HTTPS-proxy

operating system

cat /etc/alpine-release
3.9.0

Could this maybe have to do with #3446 ?

@bagder

This comment has been minimized.

Copy link
Member

commented Feb 28, 2019

It's a little unclear what this actually says the problem is. Reading the PHP code, it seems to suggest that even if the transfers run OK, curl_error can return an error string saying it failed to resolve the host name. Is that correct?

curl_error is a PHP binding function that doesn't exist in libcurl itself. The curl_error implementation in PHP shows that it basically just returns the error buffer for the handle that was used for the transfer.

This code seems to ignore this detail that is specifically noted in the man page:

Do not rely on the contents of the buffer unless an error code was returned.

But okay, I created a version of this code written in plain C that to the best of my understanding uses the same logic as the PHP code.

Running this with the latest libcurl I cannot reproduce this problem.

What makes you even suspect this is related to #3446 ?

@Schnitzel

This comment has been minimized.

Copy link
Author

commented Feb 28, 2019

wow, thanks very much for your immediate help :)

Do not rely on the contents of the buffer unless an error code was returned.

Oh wow I think you are correct. It looks like that the return code is correctly HTTP/2 200.

Is there anything that changed though in 7.63 and 7.64 which could make the buffer be filled with a message like Could not resolve host: www.example.com even though the response from the server was returned correctly?

What makes you even suspect this is related to #3446 ?

Sorry for confusing here, I was just searching through the git issues and found a very suspect issue regarding multi connections in curl. Realizing now that the response actually worked, I agree that it has nothing to do with it.

@bagder

This comment has been minimized.

Copy link
Member

commented Feb 28, 2019

I've managed to reproduce. The work-around is still to not check the error buffer if there's no error. Will file a PR with a fix as soon as I have one.

bagder added a commit that referenced this issue Feb 28, 2019

threaded-resolver: shutdown the resolver thread without error message
When a transfer is done, the resolver thread will be brought down. That
could accidentally generate an error message in the error buffer even
though this is not an error situationand the transfer would still return
OK.  An application that still reads the error buffer could find a
"Could not resolve host: [host name]" message there and get confused.

Reported-by: Michael Schmid
Fixes #3629
@bagder

This comment has been minimized.

Copy link
Member

commented Feb 28, 2019

For the record: I just needed to make sure I ran my sample code using a libcurl built to use the threaded resolver and it occurred at once.

jay added a commit to jay/php-src that referenced this issue Mar 1, 2019

curl_error: return an empty string if no error occurred
CURLOPT_ERRORBUFFER doc says "Do not rely on the contents of the
buffer unless an error code was returned." [1]

Prior to this change the error buffer was returned even if no error had
occurred, and that buffer may contain incorrect information in such a
case. [2]

[1]: https://curl.haxx.se/libcurl/c/CURLOPT_ERRORBUFFER.html
[2]: curl/curl#3629

Closes #xxxx

@bagder bagder closed this in 754ae10 Mar 1, 2019

php-pulls pushed a commit to php/php-src that referenced this issue Mar 1, 2019

curl_error: return an empty string if no error occurred
CURLOPT_ERRORBUFFER doc says "Do not rely on the contents of the
buffer unless an error code was returned." [1]

Prior to this change the error buffer was returned even if no error had
occurred, and that buffer may contain incorrect information in such a
case. [2]

[1]: https://curl.haxx.se/libcurl/c/CURLOPT_ERRORBUFFER.html
[2]: curl/curl#3629
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.