http3+ngtcp2: macOS build can't send packets

[jblazquez]

I did this

Trying to use curl built against ngtcp2 to make any kind of request on macOS fails due to a socket send error:

$ curl --http3 https://cloudflare-quic.com/
I00000000 0x7011bb1fb1b237e676ce836ab460d9bd339a7464 pkt tx pkn=0 dcid=0x3e3b925cb705ac7091de783f373db92a15be4cec scid=0x7011bb1fb1b237e676ce836ab460d9bd339a7464 type=Initial(0x00) len=0 k=0
I00000000 0x7011bb1fb1b237e676ce836ab460d9bd339a7464 frm tx 0 Initial(0x00) CRYPTO(0x06) offset=0 len=304
I00000000 0x7011bb1fb1b237e676ce836ab460d9bd339a7464 frm tx 0 Initial(0x00) PADDING(0x00) len=877
I00000000 0x7011bb1fb1b237e676ce836ab460d9bd339a7464 rcv loss_detection_timer=191779896458000 last_tx_pkt_ts=191778896458000 timeout=1000
I00000000 0x118858ee27668c98ed0558a9a115fe205b3ec5a4 pkt tx pkn=0 dcid=0x8fcf67325b645523295dfaed70c9f1e9f3ec3ee8 scid=0x118858ee27668c98ed0558a9a115fe205b3ec5a4 type=Initial(0x00) len=0 k=0
I00000000 0x118858ee27668c98ed0558a9a115fe205b3ec5a4 frm tx 0 Initial(0x00) CRYPTO(0x06) offset=0 len=304
I00000000 0x118858ee27668c98ed0558a9a115fe205b3ec5a4 frm tx 0 Initial(0x00) PADDING(0x00) len=877
I00000000 0x118858ee27668c98ed0558a9a115fe205b3ec5a4 rcv loss_detection_timer=191779897566000 last_tx_pkt_ts=191778897566000 timeout=1000
curl: (7) sendto() returned -1 (errno 56)

I expected the following

I expected the request to succeed.

When building curl against quiche the requests succeed.

curl/libcurl version

curl 7.67.0-DEV (x86_64-apple-darwin18.7.0) libcurl/7.67.0-DEV OpenSSL/3.0.0 zlib/1.2.11 ngtcp2/0.1.0-DEV nghttp3/0.1.0-DEV
Release-Date: [unreleased]
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS HTTP3 HTTPS-proxy IPv6 Largefile libz NTLM NTLM_WB SSL TLS-SRP UnixSockets

This was built as follows:

$ git clone --depth 1 -b openssl-quic-draft-23 https://github.com/tatsuhiro-t/openssl
$ cd openssl/
$ ./config enable-tls1_3 --prefix=$PWD/out
$ make -j$(sysctl -n hw.ncpu)
$ make install_sw
$ cd ..
$ git clone https://github.com/ngtcp2/nghttp3
$ cd nghttp3/
$ autoreconf -i
$ ./configure --prefix=$PWD/out --enable-lib-only --enable-debug
$ make -j$(sysctl -n hw.ncpu) install
$ cd ..
$ git clone https://github.com/ngtcp2/ngtcp2
$ cd ngtcp2/
$ autoreconf -i
$ ./configure PKG_CONFIG_PATH=$PWD/../openssl/out/lib/pkgconfig:$PWD/../nghttp3/out/lib/pkgconfig LDFLAGS="-Wl,-rpath,$PWD/../openssl/out/lib" --prefix=$PWD/out --enable-debug
$ make -j$(sysctl -n hw.ncpu) install
$ cd ..
$ git clone https://github.com/curl/curl
$ cd curl
$ ./buildconf
$ CPPFLAGS="-DDEBUG_NGTCP2" LDFLAGS="-Wl,-rpath,$PWD/../openssl/out/lib" ./configure --with-ssl=$PWD/../openssl/out --with-nghttp3=$PWD/../nghttp3/out --with-ngtcp2=$PWD/../ngtcp2/out --prefix=$PWD/out
$ make -j$(sysctl -n hw.ncpu) install

operating system

$ uname -a
Darwin 18.7.0 Darwin Kernel Version 18.7.0: Tue Aug 20 16:57:14 PDT 2019; root:xnu-4903.271.2~2/RELEASE_X86_64 x86_64

[bagder]

curl: (7) sendto() returned -1 (errno 56)

Errno 56 according to the errno man page on macOS:

56 EISCONN Socket is already connected.  A connect or connectx request
       was made on an already connected socket; or, a sendto or sendmsg
       request on a connected socket specified a destination when
       already connected.

[jblazquez]

OK, looks like changing this sendto call to a plain send will do the trick. The issue is that singleipconnect connects the UDP socket and on macOS that means sendto can't be used.

I'm going to test against other platforms to make sure moving to plain send doesn't cause unintended consequences.