[HttpClient] cURL: Unable to limit connection timeout

[deeky666]

Symfony version(s) affected

4.4.x and higher

Description

In #33022 the CURL_CONNECTTIMEOUT_MS was removed but it was not added to the allow list of curl.extra again. As such its not possible to set this setting, causing the timeout / max_duration option to be used when connecting to a host that is not available (port not open).

See #47246 (comment)

When not being able to distinguish between connection timeout and max duration, it is not possible to set a reasonable timeout in order to let requests fail fast on connection issues and still allow requests to take quite some time to finish overall. Especially in combination with a retry strategy this is a big problem if you do not want to let users wait for a long time when a host is unavailable.

How to reproduce

<?php

use Symfony\Component\HttpClient\HttpClient;
use Symfony\Component\HttpClient\Exception\TimeoutException;

require_once __DIR__ . '/vendor/autoload.php';

$client = HttpClient::create();
var_dump(get_class($client));
$ts = hrtime(true);
try {
    $response = $client->request(
        'GET',
        'http://172.67.72.38:1080',
        options: ['timeout' => 3, 'max_duration' => 10]
    );

    $messages = $response->toArray();
} catch (TimeoutException $e) {
    var_dump($e->getMessage());
}
echo "Duration: " . ((hrtime(true) - $ts) / 1e+6) . "\n";

/*
string(43) "Symfony\Component\HttpClient\CurlHttpClient"
string(53) "Idle timeout reached for "http://172.67.72.38:1080/"."
Duration: 3003.314591
*/

$h = curl_init('http://172.67.72.38:1080');
curl_setopt_array($h, [
    CURLOPT_CONNECTTIMEOUT => 2,
    CURLOPT_TIMEOUT => 10,
    CURLOPT_VERBOSE => true,
]);

curl_exec($h);

/*
*   Trying 172.67.72.38:1080...
* After 2000ms connect time, move on!
* connect to 172.67.72.38 port 1080 failed: Connection timed out
* Connection timeout after 2001 ms
* Closing connection 0
 * 
 */

Possible Solution

Allow setting CURLOPT_CONNECTTIMEOUT and CURLOPT_CONNECTTIMEOUT_MS via curl.extra.

Additional Context

#33022
#32807
#47246

[nicolas-grekas]

I'm sorry I don't understand what your expectations are. You set a timeout of 3s in the first example, and it stops after that time. This fulfills your need apparently.

[deeky666]

@nicolas-grekas I will try to clarify my point. The problem here is that Symfony does not have an option to distinguish between "how long do I want to wait before the whole request finishes and I get the complete response" and "how long do I want to wait until the connection to the host is established". Why do we need to distinguish here? Because in case of the host in question not being available, we want to be able to fail fast in order to retry or give immediate feedback to the user. I'll give an example:

We know we have to wait up to 10 seconds for a specific request to finish because the requested endpoint needs to do heavy work before being able to respond with a result. So with that expectation we set Symfony's timeout / max_duration option to 10.
Now the target server is not available because of some problem. What Symfony does is trying to connect to the server and waits up to 10 seconds before giving up. BUT the intention of a 10 seconds timeout was not to wait 10 seconds to establish a connection but wait up to 10 seconds for the whole request to finish. To detect connection problems and being able to react to that requires to have a much lower timeout for the connection phase. Otherwise we cannot make any assumptions about the semantics of an endpoint. Therefore cURL has both CURLOPT_CONNECTTIMEOUT (connection phase) and CURLOPT_TIMEOUT (complete request).

[nicolas-grekas]

I'm sorry but your description doesn't match the behavior nor the definition of timeout & max_duration options.
Without going into much explanation : timeout === CURLOPT_CONNECTTIMOUT and max_duration === CURLOPT_TIMEOUT, which is what the code in your initial example proves BTW. The client doesn't use option CURLOPT_CONNECTTIMEOUT because it implements the feature differently, but what matters is the behavior, and the behavior is the one you want.

[OskarStark]

Let's close here then.

Feel free to reopen.