QUIC: recvmsg() unexpectedly returned -1 (errno=90; Message too long)

[bagder]

Discussed in #16844

^{Originally posted by khteh March 27, 2025}
I have to carry out this test in a local microk8s cluster where I only enable UDP protocol to port 443. Curl hits the following error:
Nodeport:

svc-pythonrestapi-nodeport   NodePort    10.152.183.195   <none>        443:31002/UDP                                  5m10s

$ curl --http3 --insecure -v https://10.152.183.195
*   Trying 10.152.183.195:443...
* GnuTLS priority: NORMAL:-VERS-ALL:+VERS-TLS1.3:-CIPHER-ALL:+AES-128-GCM:+AES-256-GCM:+CHACHA20-POLY1305:+AES-128-CCM:-GROUP-ALL:+GROUP-SECP256R1:+GROUP-X25519:+GROUP-SECP384R1:+GROUP-SECP521R1:%DISABLE_TLS13_COMPAT_MODE
* Connected to 10.152.183.195 (10.152.183.195) port 443
* using HTTP/3
* [HTTP/3] [0] OPENED stream for https://10.152.183.195/
* [HTTP/3] [0] [:method: GET]
* [HTTP/3] [0] [:scheme: https]
* [HTTP/3] [0] [:authority: 10.152.183.195]
* [HTTP/3] [0] [:path: /]
* [HTTP/3] [0] [user-agent: curl/8.13.0-DEV]
* [HTTP/3] [0] [accept: */*]
> GET / HTTP/3
> Host: 10.152.183.195
> User-Agent: curl/8.13.0-DEV
> Accept: */*
> 
* Request completely sent off
* QUIC: recvmsg() unexpectedly returned -1 (errno=90; Message too long)
* Connection #0 to host 10.152.183.195 left intact
curl: (56) QUIC: recvmsg() unexpectedly returned -1 (errno=90; Message too long)
```</div>

[icing]

@khteh What is the value of IOV_MAX on your system? curl uses a fixed strut iovec array of length 16 and EMSGSIZE may be returned when that exceeds IOV_MAX.

Update: made #16849 so that we respect IOV_MAX limits for small systems.

[khteh]

What is the value of IOV_MAX on your system

./x86_64-linux-gnu/bits/uio_lim.h:#define __IOV_MAX	1024

I have pulled pulled the latest code and rebuilt still hitting the same error. Do I need to increase it? How and how much to increase it to?

[icing]

No need. The changed code was just adding a sanity check. Your system with IOV_MAX at 1024 is fine.

Which leaves the question on why recvmmsg() fails with errno EMSGSIZE on your system.

[khteh]

Then why is this closed?

[icing]

I reopened it. Looking at the code and documentation again, curl provides 64 KB buffers to recvmmsg() and EMSGSIZE is supposed to indicate that this is too small for the packet to fit into that.

Since I am unable to reproduce, I make a PR with increased values, if you can test that in your setup?

[khteh]

Yes I can.

[icing]

Please try the version in #16856. Thanks!

[khteh]

$ c3 -v https://10.152.183.214
*   Trying 10.152.183.214:443...
* GnuTLS priority: NORMAL:-VERS-ALL:+VERS-TLS1.3:-CIPHER-ALL:+AES-128-GCM:+AES-256-GCM:+CHACHA20-POLY1305:+AES-128-CCM:-GROUP-ALL:+GROUP-SECP256R1:+GROUP-X25519:+GROUP-SECP384R1:+GROUP-SECP521R1:%DISABLE_TLS13_COMPAT_MODE
*   Trying 10.152.183.214:443...
* ngtcp2_conn_handle_expiry returned error: ERR_HANDSHAKE_TIMEOUT
* Failed to connect to 10.152.183.214 port 443 after 10002 ms: Failed sending data to the peer
^C

$ curl --version
curl 8.13.0-DEV (x86_64-pc-linux-gnu) libcurl/8.13.0-DEV GnuTLS/3.8.9 zlib/1.3.1 brotli/1.1.0 zstd/1.5.6 libpsl/0.21.2 ngtcp2/1.11.0 nghttp3/1.8.0
Release-Date: [unreleased]
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS brotli HSTS HTTP3 HTTPS-proxy IPv6 Largefile libz NTLM PSL SSL threadsafe TLS-SRP UnixSockets zstd

[khteh]

Sometimes it works.

[icing]

if you run curl -v --http3-only https://curl.se, does it always work?

[khteh]

Yes, it always works.

[icing]

Then I would suspect that the server at 10.152.183.195 is either buggy or that there is some middle box in play that does weird things.

[khteh]

Close this until pgjones/hypercorn#295 is looked into.

[icing]

@khteh I will close. We can always reopen when you find that the mistake is really at curl and curl only.