curl_easy_perform() waits full response with CURLOPT_CONNECT_ONLY enabled

[KirillObukhov]

I did this

I'm trying to receive Server-Sent Events using libcurl. I've found CURLOPT_CONNECT_ONLY option which

can be set to '2' and if HTTP or WebSocket are used, libcurl performs the request and reads all response headers before handing over control to the application

but curl_easy_perform() call blocks while connection alive.

For demonstration, I have attached a simple server and client applications.
demo.zip

The server (Kotlin) sends 5 messages every 3 seconds and terminates the connection.

           sse {
                repeat(5) {
                    delay(3.seconds)
                    send(LocalTime.now().toString())
                }
            }

The client (C) connects to the server, exchange HTTP headers, report the execution time and should exit without waiting events.

  CURL *curl = curl_easy_init();
  if (!curl)
    goto cleanup;
  if (curl_easy_setopt(curl, CURLOPT_URL, "http://localhost:8080"))
    goto cleanup;
  if (curl_easy_setopt(curl, CURLOPT_CONNECT_ONLY, 2L))
    goto cleanup;

  time_t start_time = time(NULL);
  if (curl_easy_perform(curl))
    goto cleanup;
  time_t end_time = time(NULL);

  double duration = difftime(end_time, start_time);
  printf("perform takes %.2f seconds\n", duration);

but outputs

perform takes 15.00 seconds

How-to build & run server:

cd server
gradle build
cd build/distributions
tar -xf server.tar
./server/bin/server

curl -v http://localhost:8080

* Host localhost:8080 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:8080...
* Connected to localhost (::1) port 8080
* using HTTP/1.x
> GET / HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/8.12.1
> Accept: */*
> 
* Request completely sent off
< HTTP/1.1 200 OK
< Content-Type: text/event-stream
< Cache-Control: no-store
< Connection: keep-alive
< X-Accel-Buffering: no
< Transfer-Encoding: chunked
< 
data: 22:22:18.489580323

data: 22:22:21.490710433

data: 22:22:24.491684059

data: 22:22:27.492930039

data: 22:22:30.494157034

* Connection #0 to host localhost left intact

I expected the following

No response

curl/libcurl version

curl 8.12.1 (x86_64-pc-linux-gnu) libcurl/8.12.1 OpenSSL/3.4.1 zlib/1.3.1 brotli/1.1.0 zstd/1.5.6 libidn2/2.3.8 libpsl/0.21.2 libssh2/1.11.1 nghttp2/1.64.0 librtmp/2.3 OpenLDAP/2.6.9
Release-Date: 2025-02-13, security patched: 8.12.1-3ubuntu1
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM PSL SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd

operating system

Linux lpc 6.14.0-15-generic #15-Ubuntu SMP PREEMPT_DYNAMIC Sun Apr 6 15:05:05 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

[dfandrich]

When using CURLOPT_CONNECT_ONLY, curl hands over control to the application. curl_easy_perform() will have nothing good to do in that case; the application must take over. In the WebSocket case, the application should be calling curl_ws_send() and curl_ws_recv() etc. instead.

[KirillObukhov]

When the option parameter is 1, then curl performs connection setup and returns control to the application.
When the option parameter is 2, then curl performs connection setup, makes the request, reads all response headers and returns control to the application.
In my case the option parameter is 2, but curl continues reading the response after the headers have been received.

[dfandrich]

Does the URL support WebSockets? Does the example program docs/examples/websocket.c work against that site? Does it work with curl 8.14.1? What do the logs show in the WebSockets case? Are you using a wss: scheme or http: scheme (it should be the former)?

[KirillObukhov]

It's Server-Sent Events, not WebSockets. It is unidirectional messages stream over HTTP connection.

CURL woks nice with it in blocking mode:

#include <curl/curl.h>
#include <stdio.h>

size_t write_data(void *buffer, size_t size, size_t data_size, void *data) {
  const size_t total_size = size * data_size;

  char *message = (char *)buffer;
  size_t length = total_size / sizeof(char);
  while (message[length] == '\n' || message[length] == '\r')
    --length;
  printf("%.*s\n", length, message);

  return total_size;
}

int main() {
  curl_global_init(CURL_GLOBAL_DEFAULT);
  CURL *curl = curl_easy_init();
  if (!curl)
    goto cleanup;
  if (curl_easy_setopt(curl, CURLOPT_URL, "http://localhost:8080"))
    goto cleanup;
  //  if (curl_easy_setopt(curl, CURLOPT_CONNECT_ONLY, 2L))
  //    goto cleanup;
  curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);

  time_t start_time = time(NULL);
  if (curl_easy_perform(curl))
    goto cleanup;
  time_t end_time = time(NULL);

  double duration = difftime(end_time, start_time);
  printf("perform takes %.2f seconds\n", duration);

cleanup:
  curl_easy_cleanup(curl);
  curl_global_cleanup();
}

outputs:

data: 08:44:38.74726645
data: 08:44:41.74824323
data: 08:44:44.74934386
data: 08:44:47.75059053
data: 08:44:50.75151093
perform takes 15.00 seconds

But I need the curl_easy_perform() call to return control to the application after receiving HTTP headers.

[dfandrich]

For some reason I thought you were trying to do WebSockets. Can you show the trace output for the connection?

[KirillObukhov]

curl_global_trace("all") + curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L)

* [READ] client_reset, clear readers
* Host localhost:8080 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
* [SETUP] added
* [HAPPY-EYEBALLS] created ipv6 (timeout 300000ms)
* [HAPPY-EYEBALLS] created ipv4 (timeout 300000ms)
* [HAPPY-EYEBALLS] ipv6 starting (timeout=300000ms)
*   Trying [::1]:8080...
* [TCP] cf_socket_open() -> 0, fd=4
* [TCP] local address ::1 port 45174...
* [HAPPY-EYEBALLS] ipv6 connect -> 0, connected=0
* [TCP] adjust_pollset, !connected, POLLOUT fd=4
* [HAPPY-EYEBALLS] adjust_pollset -> 1 socks
* [TCP] connected
* [HAPPY-EYEBALLS] ipv6 connect -> 0, connected=1
* [HAPPY-EYEBALLS] Connected to localhost (::1) port 8080
* Connected to localhost (::1) port 8080
* using HTTP/1.x
* [TCP] send(len=53) -> 53, err=0
> GET / HTTP/1.1

Host: localhost:8080

Accept: */*



* [TCP] recv(len=16384) -> -1, err=81
* Request completely sent off
* [TCP] recv(len=16384) -> 152, err=0
< HTTP/1.1 200 OK

* [WRITE] download_write header(type=c, blen=17) -> 0
* [WRITE] client_write(type=c, len=17) -> 0
< Content-Type: text/event-stream

* [WRITE] header_collect pushed(type=1, len=33) -> 0
* [WRITE] download_write header(type=4, blen=33) -> 0
* [WRITE] client_write(type=4, len=33) -> 0
< Cache-Control: no-store

* [WRITE] header_collect pushed(type=1, len=25) -> 0
* [WRITE] download_write header(type=4, blen=25) -> 0
* [WRITE] client_write(type=4, len=25) -> 0
< Connection: keep-alive

* [WRITE] header_collect pushed(type=1, len=24) -> 0
* [WRITE] download_write header(type=4, blen=24) -> 0
* [WRITE] client_write(type=4, len=24) -> 0
< X-Accel-Buffering: no

* [WRITE] header_collect pushed(type=1, len=23) -> 0
* [WRITE] download_write header(type=4, blen=23) -> 0
* [WRITE] client_write(type=4, len=23) -> 0
* [WRITE] looking for transfer decoder: chunked
* [WRITE] added transfer decoder chunked -> 0
< Transfer-Encoding: chunked

* [WRITE] header_collect pushed(type=1, len=28) -> 0
* [WRITE] download_write header(type=4, blen=28) -> 0
* [WRITE] client_write(type=4, len=28) -> 0
< 

* [WRITE] header_collect pushed(type=1, len=2) -> 0
* [WRITE] download_write header(type=4, blen=2) -> 0
* [WRITE] client_write(type=4, len=2) -> 0
* [WRITE] xfer_write_resp(len=152, eos=0) -> 0
* [TCP] recv(len=16384) -> -1, err=81
* [TCP] recv(len=16384) -> -1, err=81
* [TCP] recv(len=16384) -> -1, err=81
* [TCP] recv(len=16384) -> -1, err=81
* [TCP] recv(len=16384) -> 34, err=0
* [WRITE] http_chunked, chunk start of 28 bytes
* [WRITE] cw_out, wrote 28 body bytes -> 28
* [WRITE] download_write body(type=1, blen=28) -> 0
* [WRITE] http_chunked, write 28 body bytes, 0 bytes in chunk remain
* [WRITE] client_write(type=1, len=34) -> 0
* [WRITE] xfer_write_resp(len=34, eos=0) -> 0
* [TCP] recv(len=16384) -> -1, err=81
data: 10:47:07.884730019



* [TCP] recv(len=16384) -> -1, err=81
* [TCP] recv(len=16384) -> -1, err=81
data: 10:47:10.885592587



* [TCP] recv(len=16384) -> 34, err=0
* [WRITE] http_chunked, chunk start of 28 bytes
* [WRITE] cw_out, wrote 28 body bytes -> 28
* [WRITE] download_write body(type=1, blen=28) -> 0
* [WRITE] http_chunked, write 28 body bytes, 0 bytes in chunk remain
* [WRITE] client_write(type=1, len=34) -> 0
* [WRITE] xfer_write_resp(len=34, eos=0) -> 0
* [TCP] recv(len=16384) -> -1, err=81
* [TCP] recv(len=16384) -> -1, err=81
* [TCP] recv(len=16384) -> -1, err=81
data: 10:47:13.886572442



* [TCP] recv(len=16384) -> 34, err=0
* [WRITE] http_chunked, chunk start of 28 bytes
* [WRITE] cw_out, wrote 28 body bytes -> 28
* [WRITE] download_write body(type=1, blen=28) -> 0
* [WRITE] http_chunked, write 28 body bytes, 0 bytes in chunk remain
* [WRITE] client_write(type=1, len=34) -> 0
* [WRITE] xfer_write_resp(len=34, eos=0) -> 0
* [TCP] recv(len=16384) -> -1, err=81
* [TCP] recv(len=16384) -> -1, err=81
* [TCP] recv(len=16384) -> -1, err=81
* [TCP] recv(len=16384) -> 34, err=0
* [WRITE] http_chunked, chunk start of 28 bytes
* [WRITE] cw_out, wrote 28 body bytes -> 28
* [WRITE] download_write body(type=1, blen=28) -> 0
* [WRITE] http_chunked, write 28 body bytes, 0 bytes in chunk remain
* [WRITE] client_write(type=1, len=34) -> 0
* [WRITE] xfer_write_resp(len=34, eos=0) -> 0
* [TCP] recv(len=16384) -> -1, err=81
data: 10:47:16.887418831



* [TCP] recv(len=16384) -> -1, err=81
* [TCP] recv(len=16384) -> -1, err=81
data: 10:47:19.888419360



* [TCP] recv(len=16384) -> 34, err=0
* [WRITE] http_chunked, chunk start of 28 bytes
* [WRITE] cw_out, wrote 28 body bytes -> 28
* [WRITE] download_write body(type=1, blen=28) -> 0
* [WRITE] http_chunked, write 28 body bytes, 0 bytes in chunk remain
* [WRITE] client_write(type=1, len=34) -> 0
* [WRITE] xfer_write_resp(len=34, eos=0) -> 0
* [TCP] recv(len=16384) -> -1, err=81
* [TCP] recv(len=16384) -> 5, err=0
* [WRITE] http_chunk, response complete
* [WRITE] client_write(type=1, len=5) -> 0
* [WRITE] xfer_write_resp(len=5, eos=0) -> 0
* [WRITE] cw-out is notpaused
* [WRITE] cw-out done
* [READ] client_reset, clear readers
* Connection #0 to host localhost left intact

[bagder]

Setting it to 2 was only ever thought of and designed for WebSocket. As long as you speak HTTP, we don't recommend it. If you don't want blocking, I propose you use the multi interface instead.

[KirillObukhov]

So, confusion is in the documentation?

this option can be set to '2' and if HTTP or WebSocket are used,

It seems that in my case it is easier to use the value 1 and write custom HTTP headers processing.
Thanks for the help!

[icing]

Propose to update the documentation in #17635