8.18: re-adding of handle into multi while paused due to asynchronous certificate verification does not unpause it. #20641
Description
I did this
returned "SSL_set_retry_verify(ssl)" from openssl cert verification callback
called curl_multi_remove_handle() on the handle
called curl_multi_add_handle() on the handle
I expected the following
In 8.16 I see that removing handle causes connection to be closed, and adding it initiates new connect.
In 8.18 the latter does not happen.
The text below is AI analysis of changes and may be wrong:
Root Cause
The regression was introduced by commit 24b36fdd15 ("ratelimit: redesign"), which landed between 8.16 and 8.18.
What changed
The commit moved the "transfer is paused" state from data->req.keepon flags to the new data->progress.dl.rlimit.blocked / data->progress.ul.rlimit.blocked fields:
8.16 — pause tracked in req.keepon:
bool Curl_xfer_recv_is_paused(struct Curl_easy *data)
{
return (data->req.keepon & KEEP_RECV_PAUSE);
}
8.18 — pause tracked in rlimit:
bool Curl_xfer_recv_is_paused(struct Curl_easy *data)
{
return Curl_rlimit_is_blocked(&data->progress.dl.rlimit);
}
Why it worked in 8.16
When the handle is re-added and the state machine reaches MSTATE_CONNECT, it calls Curl_connect() → Curl_req_hard_reset(), which does:
lib/request.c lines 135-135
req->keepon = 0;
This cleared all keepon flags including KEEP_RECV_PAUSE. So when MSTATE_CONNECTING checked Curl_xfer_recv_is_paused(), it returned FALSE, and Curl_conn_connect() proceeded normally.
Why it breaks in 8.18
Curl_req_hard_reset() still sets req->keepon = 0, but that field no longer controls the pause state. The rlimit's blocked flag at data->progress.dl.rlimit.blocked is never cleared during the remove/add cycle:
• curl_multi_remove_handle doesn't touch it
• curl_multi_add_handle doesn't touch it
• Curl_pretransfer() doesn't touch it
• Curl_req_hard_reset() doesn't touch it
• Curl_pgrsReset() / Curl_pgrsStartNow() don't touch it
• Curl_rlimit_init() does set blocked = FALSE, but it's only called from setopt.c when CURLOPT_MAX_RECV_SPEED_LARGE is set
So the handle remains "paused" from the old connection's async cert verification. The state machine in MSTATE_CONNECTING then:
lib/multi.c lines 2459-2459
if(!Curl_xfer_recv_is_paused(data)) {
...skips Curl_conn_connect() entirely. And the pollset function also bails out early:
lib/multi.c lines 996-997
if(Curl_xfer_recv_is_paused(data))
return CURLE_OK;
...returning an empty pollset. The result: the handle is stuck in MSTATE_CONNECTING with no sockets registered and no way to make progress. The new connection never starts.
Fix
The blocked flag on the rlimits needs to be cleared when resetting the transfer for a new request. The most appropriate place would be in Curl_req_hard_reset(), since that's where keepon (the old home of the pause state) was already being cleared. Something like:
Curl_rlimit_block(&data->progress.dl.rlimit, FALSE, &t0);
Curl_rlimit_block(&data->progress.ul.rlimit, FALSE, &t0);
Or alternatively, Curl_rlimit_start() could be called to re-initialize the rlimit without losing the rate configuration.
curl/libcurl version
libcurl 8.18
operating system
MacOS Sonoma