[WIP] QUIC and HTTP/3 #3500

bagder · 2019-01-25T10:30:48Z

This is work in progress (and incomplete) work on QUIC and HTTP/3 support.

The first steps of the plan;

In this branch, see docs/HTTP3.md for basic and initial docs.

There's no general HTTP/3 support for ngtcp2 nor quiche available yet.

With the quiche backend a first trial HTTP/1.1 request over QUIC can be done with:

$ ./src/curl --http3-direct https://cloudflare-quic.com/

Wiki page for QUIC implementation thoughts

ghedo · 2019-01-25T12:45:58Z

🎉

FTR, we started working on HTTP/3 support for quiche (starting with client-side), though it's still quite far from being ready for use.

ghedo · 2019-01-25T17:05:51Z

Heads up, we just slightly changed the quiche stream recv API to be a bit nicer, so if you use latest quiche master you now need these changes:

diff --git a/lib/vquic/quiche.c b/lib/vquic/quiche.c
index 4cff3c6d9..41e837f1d 100644
--- a/lib/vquic/quiche.c
+++ b/lib/vquic/quiche.c
@@ -167,8 +167,8 @@ static ssize_t quic_stream_recv(struct connectdata *conn,
                                 size_t buffersize,
                                 CURLcode *curlcode)
 {
+  bool fin;
   ssize_t recvd;
-  quiche_rangebuf *data;
   struct quicsocket *qs = &conn->quic;
   curl_socket_t sockfd = conn->sock[sockindex];
 
@@ -177,18 +177,16 @@ static ssize_t quic_stream_recv(struct connectdata *conn,
     return -1;
   }
 
-  data = quiche_conn_stream_recv(qs->conn, 0, buffersize);
-  if(data == NULL) {
+  recvd = quiche_conn_stream_recv(qs->conn, 0, (uint8_t *) buf, buffersize, &fin);
+  if(recvd == QUICHE_ERR_DONE) {
     *curlcode = CURLE_AGAIN;
     return -1;
   }
 
-  recvd = quiche_rangebuf_len(data);
-
-  if(recvd > 0)
-    memcpy(buf, quiche_rangebuf_data(data), recvd);
-
-  quiche_rangebuf_free(data);
+  if(recvd < 0) {
+    *curlcode = CURLE_RECV_ERROR;
+    return -1;
+  }
 
   *curlcode = CURLE_OK;
   return recvd;

The nice thing about this is that there's no need for the additional memcpy() anymore.

bagder · 2019-01-25T22:07:47Z

Thanks, merged

docs/HTTP3.md

bagder · 2019-04-13T21:30:09Z

I pushed a rebased update of this branch, but now I fail to build it with quiche. configure won't detect it and says this in config.log:

conftest.c:74:1: warning: function declaration isn't a prototype [-Wstrict-prototypes]
 char quiche_connect ();
 ^~~~
/usr/bin/ld: /home/daniel/src/quiche/target/debug/libquiche.a(ring-cb95bbea3dc49340.ring.6d3c26sa-cgu.10.rcgu.o): in function `ring::constant_time::verify_slices_are_equal':
/home/daniel/.cargo/registry/src/github.com-1ecc6299db9ec823/ring-0.14.2/src/constant_time.rs:27: undefined reference to `GFp_memcmp'
/usr/bin/ld: /home/daniel/src/quiche/target/debug/libquiche.a(ring-cb95bbea3dc49340.ring.6d3c26sa-cgu.11.rcgu.o): in function `ring::cpu::cache_detected_features::{{closure}}':
/home/daniel/.cargo/registry/src/github.com-1ecc6299db9ec823/ring-0.14.2/src/cpu.rs:24: undefined reference to `GFp_cpuid_setup'

ghedo · 2019-04-14T10:18:54Z

That'w weird, I can build it fine here. Maybe try to run cargo clean in the quiche directory and rebuild it from scratch? Also, make sure to be on the latest commit.

bagder · 2019-04-16T07:56:55Z

Thanks, yes I was probably drunk or just stupid somehow because today when I just cleaned up and retried I have a successful build with quiche + boringssl again...

ghedo · 2019-07-17T12:50:21Z

@bagder a few more changes to make it build with latest quiche. Though note that we don't have an HTTP/1.1 over QUIC test server anymore, so it's kind of hard to test right now (though that will change once quiche HTTP/3 support in integrated into curl).

Also, we are currently missing support for QUIC timers, so we can't react to packet loss and the like. It's not quite clear to me whether curl already has some kind of timer infrastructure we can use, but ideally we'd use something that's already there :)

diff --git a/configure.ac b/configure.ac
index 364939c8d..eae7f9062 100755
--- a/configure.ac
+++ b/configure.ac
@@ -3461,9 +3461,9 @@ if test X"$want_quiche" != Xno; then
   CLEANCPPFLAGS="$CPPFLAGS"
   CLEANLIBS="$LIBS"
 
-  LIB_QUICHE="-lquiche -ldl -lpthread"
+  LIB_QUICHE="-l:libquiche.a -ldl -lpthread"
   CPP_QUICHE="-I$OPT_QUICHE/include"
-  LD_QUICHE="-L$OPT_QUICHE/target/debug"
+  LD_QUICHE="-L$OPT_QUICHE/target/release"
 
   LDFLAGS="$LDFLAGS $LD_QUICHE"
   CPPFLAGS="$CPPFLAGS $CPP_QUICHE"
diff --git a/docs/HTTP3.md b/docs/HTTP3.md
index 64bd31a6e..c09bf0a76 100644
--- a/docs/HTTP3.md
+++ b/docs/HTTP3.md
@@ -17,7 +17,7 @@ QUIC libraries we're experiementing with:
 
 [ngtcp2](https://github.com/ngtcp2/ngtcp2)
 
-[quiche](https://github.com/ghedo/quiche)
+[quiche](https://github.com/cloudflare/quiche)
 
 ## Experimental!
 
@@ -68,12 +68,12 @@ Build BoringSSL (it needs to be built manually so it can be reused with curl):
 Build quiche:
 
      % cd ../..
-     % QUICHE_BSSL_PATH=$PWD/deps/boringssl cargo build
+     % QUICHE_BSSL_PATH=$PWD/deps/boringssl cargo build --release
 
 Clone and build curl:
 
      % cd ..
-     % git clone -b quiche https://github.com/ghedo/curl
+     % git clone https://github.com/curl/curl
      % ./buildconf
      % ./configure --with-ssl=$PWD/../quiche/deps/boringssl/.openssl --with-quiche=$PWD/../quiche --enable-debug
      % make -j`nproc`
diff --git a/lib/vquic/quiche.c b/lib/vquic/quiche.c
index 2adb47eb8..f6c4ad42c 100644
--- a/lib/vquic/quiche.c
+++ b/lib/vquic/quiche.c
@@ -38,7 +38,7 @@
 
 #define QUIC_MAX_STREAMS (256*1024)
 #define QUIC_MAX_DATA (1*1024*1024)
-#define QUIC_IDLE_TIMEOUT 60 /* seconds? */
+#define QUIC_IDLE_TIMEOUT 60 * 1000 /* milliseconds */
 
 static CURLcode process_ingress(struct connectdata *conn,
                                 curl_socket_t sockfd);
@@ -59,18 +59,18 @@ CURLcode Curl_quic_connect(struct connectdata *conn, curl_socket_t sockfd,
 
   infof(conn->data, "Connecting socket %d over QUIC\n", sockfd);
 
-  qs->cfg = quiche_config_new(QUICHE_VERSION_DRAFT18);
+  qs->cfg = quiche_config_new(QUICHE_PROTOCOL_VERSION);
   if(!qs->cfg)
     return CURLE_FAILED_INIT; /* TODO: better return code */
 
   quiche_config_set_idle_timeout(qs->cfg, QUIC_IDLE_TIMEOUT);
+  quiche_config_set_initial_max_data(qs->cfg, QUIC_MAX_DATA);
   quiche_config_set_initial_max_stream_data_bidi_local(qs->cfg, QUIC_MAX_DATA);
-  quiche_config_set_initial_max_stream_data_bidi_remote(qs->cfg,
-                                                        QUIC_MAX_DATA);
+  quiche_config_set_initial_max_stream_data_bidi_remote(qs->cfg, QUIC_MAX_DATA);
   quiche_config_set_initial_max_stream_data_uni(qs->cfg, QUIC_MAX_DATA);
-  quiche_config_set_initial_max_data(qs->cfg, QUIC_MAX_DATA);
-  quiche_config_set_initial_max_streams_bidi(qs->cfg, 10);
-  quiche_config_set_initial_max_streams_uni(qs->cfg, 10);
+  quiche_config_set_initial_max_streams_bidi(qs->cfg, QUIC_MAX_STREAMS);
+  quiche_config_set_initial_max_streams_uni(qs->cfg, QUIC_MAX_STREAMS);
+  quiche_config_set_application_protos(qs->cfg, (uint8_t *) "\x05hq-20", 6);
 
   result = Curl_rand(conn->data, qs->scid, sizeof(qs->scid));
   if(result)
@@ -143,7 +143,7 @@ static CURLcode flush_egress(struct connectdata *conn, int sockfd)
 {
   ssize_t sent;
   struct quicsocket *qs = &conn->quic;
-  static uint8_t out[1400];
+  static uint8_t out[1200];
 
   do {
     sent = quiche_conn_send(qs->conn, out, sizeof(out));

USe configure --with-ngtcp2 or --with-quiche Using either option will enable a HTTP3 build. Co-authored-by: Alessandro Ghedini <alessandro@ghedini.me>

bagder · 2019-07-20T22:27:32Z

Plan: once this builds green (since nothing tests or uses HTTP/3 in the CI) I plan to merge this as the early and initial foundation to work further on. It should make it easier for everyone to collaborate on getting QUIC and HTTP/3 landed.

HTTP/3 support is experimental and should be treated as such in curl until further notice. We should make sure all documentation says so.

USe configure --with-ngtcp2 or --with-quiche Using either option will enable a HTTP3 build. Co-authored-by: Alessandro Ghedini <alessandro@ghedini.me> Closes curl#3500

JenoP · 2019-10-05T17:34:00Z

Hello folks,

Installed curl 7.66 in my Fedora29 box. http/3 support is not working.

pjeno@jeno:~/experimentation$ curl --http3-direct https://cloudflare-quic.com/
curl: option --http3-direct: is unknown

pjeno@jeno:~/experimentation$ curl --http3 https://cloudflare-quic.com/
curl: (1) Unsupported protocol

Do you've any ideas ?

Thanks!

bagder · 2019-10-06T08:35:19Z

You need to explicitly enable HTTP/3 when you build curl, which no distro-packaged curl does yet because this is still early days in HTTP/3 land.

If you want to experiment with and try out curl's HTTP/3 support, I recommend building curl from git master and following the instructions in docs/HTTP3.md .

JenoP · 2019-10-06T17:44:32Z

Hi Daniel and all, I built curl with http/3 support and got it working with a web server which advertises http/3. Thanks for your suggestion! I want to benchmark transfer speed across http/2 and http/3. Do you guys know about any web server with http/3 which host considerably big size web contents ? Thanks, Jeno

…

On Sun, Oct 6, 2019 at 2:05 PM Daniel Stenberg ***@***.***> wrote: You need to explicitly enable HTTP/3 when you build curl, which no distro-packaged curl does yet because this is still early days in HTTP/3 land. If you want to experiment with and try out curl's HTTP/3 support, I recommend building curl from git master and following the instructions in docs/HTTP3.md <https://github.com/curl/curl/blob/master/docs/HTTP3.md> . — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3500?email_source=notifications&email_token=AGX6MI3G2VEVS7TIFF4E7HLQNGPOJA5CNFSM4GSKOCAKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAOEM3I#issuecomment-538723949>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGX6MI4DVTGPGECWUAING7TQNGPOJANCNFSM4GSKOCAA> .

bagder · 2019-10-06T22:38:39Z

I know several of them provide specific URL for "large" downloads over HTTP/3 but if you want to benchmark you should probably rather run of them by yourself so that you know what you compare and can do it in a controlled way.

curl's HTTP/3 implementation is not polished much and is certainly not yet optimized for speed.

For dedicated client-side HTTP benchmarking, I'd recommend using h2load which also supports HTTP/3 at least in git. That'll give you a much better idea than curl.

bagder added the HTTP/3 h3 or quic related label Jan 25, 2019

simon04 reviewed Feb 11, 2019

View reviewed changes

docs/HTTP3.md Outdated Show resolved Hide resolved

bagder force-pushed the QUIC branch from 73d98ab to ffc364a Compare February 12, 2019 07:11

bagder force-pushed the QUIC branch from ffc364a to 8cee4d8 Compare March 3, 2019 21:21

bagder force-pushed the QUIC branch from 8cee4d8 to 0b34f7f Compare March 22, 2019 15:42

bagder force-pushed the QUIC branch from 0b34f7f to 5c846b1 Compare April 13, 2019 20:53

bagder force-pushed the QUIC branch from 660c24c to a46c81a Compare May 13, 2019 06:30

bagder force-pushed the QUIC branch from a46c81a to c571f20 Compare May 22, 2019 08:15

bagder force-pushed the QUIC branch from c571f20 to 9aa847f Compare June 10, 2019 12:42

bagder force-pushed the QUIC branch from 9aa847f to 693513d Compare June 25, 2019 06:47

bagder force-pushed the QUIC branch from 693513d to 0b6ff0a Compare July 16, 2019 10:14

HTTP3: initial (experimental) support

74c6336

USe configure --with-ngtcp2 or --with-quiche Using either option will enable a HTTP3 build. Co-authored-by: Alessandro Ghedini <alessandro@ghedini.me>

bagder force-pushed the QUIC branch from 0b6ff0a to 74c6336 Compare July 20, 2019 22:25

fixup version numbers, and mark as experimental in docs

f965075

bagder closed this in 3af0e76 Jul 21, 2019

bagder deleted the QUIC branch July 21, 2019 21:49

lock bot locked as resolved and limited conversation to collaborators Jan 4, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[WIP] QUIC and HTTP/3 #3500

[WIP] QUIC and HTTP/3 #3500

bagder commented Jan 25, 2019 •

edited

ghedo commented Jan 25, 2019 •

edited

ghedo commented Jan 25, 2019 •

edited

bagder commented Jan 25, 2019

bagder commented Apr 13, 2019

ghedo commented Apr 14, 2019

bagder commented Apr 16, 2019

ghedo commented Jul 17, 2019

bagder commented Jul 20, 2019

JenoP commented Oct 5, 2019 •

edited

bagder commented Oct 6, 2019

JenoP commented Oct 6, 2019 via email

bagder commented Oct 6, 2019

[WIP] QUIC and HTTP/3 #3500

[WIP] QUIC and HTTP/3 #3500

Conversation

bagder commented Jan 25, 2019 • edited

ghedo commented Jan 25, 2019 • edited

ghedo commented Jan 25, 2019 • edited

bagder commented Jan 25, 2019

bagder commented Apr 13, 2019

ghedo commented Apr 14, 2019

bagder commented Apr 16, 2019

ghedo commented Jul 17, 2019

bagder commented Jul 20, 2019

JenoP commented Oct 5, 2019 • edited

bagder commented Oct 6, 2019

JenoP commented Oct 6, 2019 via email

bagder commented Oct 6, 2019

bagder commented Jan 25, 2019 •

edited

ghedo commented Jan 25, 2019 •

edited

ghedo commented Jan 25, 2019 •

edited

JenoP commented Oct 5, 2019 •

edited