Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] QUIC and HTTP/3 #3500

Closed
wants to merge 2 commits into from

Conversation

@bagder
Copy link
Member

bagder commented Jan 25, 2019

This is work in progress (and incomplete) work on QUIC and HTTP/3 support.

The first steps of the plan;

  • - make all QUIC code conditional on the detection of a QUIC library at build time
  • - make the build able to use different QUIC libraries at build time
  • - detect ngtcp2 in configure
  • - detect quiche in configure
  • - show "HTTP3" as a feature in --version output
  • - show ngtcp2 in the version string for third party libs
  • - show quiche in the version string for third party libs
  • - support QUIC/HTTP3 builds with cmake
  • - figure out how to show the correct ngtcp2/quiche version numbers in that string
  • - support CURLOPT_H3 and the CURLH3_DIRECT flag
  • - support --http3-direct in the command line tool
  • - actually connect using QUIC when CURLH3_DIRECT is set (with quiche)
  • - actually connect using QUIC when CURLH3_DIRECT is set (with ngtcp2)
  • - disconnect QUIC
  • - integrate with alt-svc

In this branch, see docs/HTTP3.md for basic and initial docs.

There's no general HTTP/3 support for ngtcp2 nor quiche available yet.

With the quiche backend a first trial HTTP/1.1 request over QUIC can be done with:

$ ./src/curl --http3-direct https://cloudflare-quic.com/

Wiki page for QUIC implementation thoughts

@bagder bagder added the HTTP/3 label Jan 25, 2019
@ghedo

This comment has been minimized.

Copy link
Member

ghedo commented Jan 25, 2019

🎉

FTR, we started working on HTTP/3 support for quiche (starting with client-side), though it's still quite far from being ready for use.

@ghedo

This comment has been minimized.

Copy link
Member

ghedo commented Jan 25, 2019

Heads up, we just slightly changed the quiche stream recv API to be a bit nicer, so if you use latest quiche master you now need these changes:

diff --git a/lib/vquic/quiche.c b/lib/vquic/quiche.c
index 4cff3c6d9..41e837f1d 100644
--- a/lib/vquic/quiche.c
+++ b/lib/vquic/quiche.c
@@ -167,8 +167,8 @@ static ssize_t quic_stream_recv(struct connectdata *conn,
                                 size_t buffersize,
                                 CURLcode *curlcode)
 {
+  bool fin;
   ssize_t recvd;
-  quiche_rangebuf *data;
   struct quicsocket *qs = &conn->quic;
   curl_socket_t sockfd = conn->sock[sockindex];
 
@@ -177,18 +177,16 @@ static ssize_t quic_stream_recv(struct connectdata *conn,
     return -1;
   }
 
-  data = quiche_conn_stream_recv(qs->conn, 0, buffersize);
-  if(data == NULL) {
+  recvd = quiche_conn_stream_recv(qs->conn, 0, (uint8_t *) buf, buffersize, &fin);
+  if(recvd == QUICHE_ERR_DONE) {
     *curlcode = CURLE_AGAIN;
     return -1;
   }
 
-  recvd = quiche_rangebuf_len(data);
-
-  if(recvd > 0)
-    memcpy(buf, quiche_rangebuf_data(data), recvd);
-
-  quiche_rangebuf_free(data);
+  if(recvd < 0) {
+    *curlcode = CURLE_RECV_ERROR;
+    return -1;
+  }
 
   *curlcode = CURLE_OK;
   return recvd;

The nice thing about this is that there's no need for the additional memcpy() anymore.

@bagder

This comment has been minimized.

Copy link
Member Author

bagder commented Jan 25, 2019

Thanks, merged

docs/HTTP3.md Outdated Show resolved Hide resolved
@bagder bagder force-pushed the QUIC branch from 73d98ab to ffc364a Feb 12, 2019
@bagder bagder force-pushed the QUIC branch from ffc364a to 8cee4d8 Mar 3, 2019
@bagder bagder force-pushed the QUIC branch from 8cee4d8 to 0b34f7f Mar 22, 2019
@bagder bagder force-pushed the QUIC branch from 0b34f7f to 5c846b1 Apr 13, 2019
@bagder

This comment has been minimized.

Copy link
Member Author

bagder commented Apr 13, 2019

I pushed a rebased update of this branch, but now I fail to build it with quiche. configure won't detect it and says this in config.log:

conftest.c:74:1: warning: function declaration isn't a prototype [-Wstrict-prototypes]
 char quiche_connect ();
 ^~~~
/usr/bin/ld: /home/daniel/src/quiche/target/debug/libquiche.a(ring-cb95bbea3dc49340.ring.6d3c26sa-cgu.10.rcgu.o): in function `ring::constant_time::verify_slices_are_equal':
/home/daniel/.cargo/registry/src/github.com-1ecc6299db9ec823/ring-0.14.2/src/constant_time.rs:27: undefined reference to `GFp_memcmp'
/usr/bin/ld: /home/daniel/src/quiche/target/debug/libquiche.a(ring-cb95bbea3dc49340.ring.6d3c26sa-cgu.11.rcgu.o): in function `ring::cpu::cache_detected_features::{{closure}}':
/home/daniel/.cargo/registry/src/github.com-1ecc6299db9ec823/ring-0.14.2/src/cpu.rs:24: undefined reference to `GFp_cpuid_setup'
@ghedo

This comment has been minimized.

Copy link
Member

ghedo commented Apr 14, 2019

That'w weird, I can build it fine here. Maybe try to run cargo clean in the quiche directory and rebuild it from scratch? Also, make sure to be on the latest commit.

@bagder

This comment has been minimized.

Copy link
Member Author

bagder commented Apr 16, 2019

Thanks, yes I was probably drunk or just stupid somehow because today when I just cleaned up and retried I have a successful build with quiche + boringssl again...

@ghedo

This comment has been minimized.

Copy link
Member

ghedo commented Jul 17, 2019

@bagder a few more changes to make it build with latest quiche. Though note that we don't have an HTTP/1.1 over QUIC test server anymore, so it's kind of hard to test right now (though that will change once quiche HTTP/3 support in integrated into curl).

Also, we are currently missing support for QUIC timers, so we can't react to packet loss and the like. It's not quite clear to me whether curl already has some kind of timer infrastructure we can use, but ideally we'd use something that's already there :)

diff --git a/configure.ac b/configure.ac
index 364939c8d..eae7f9062 100755
--- a/configure.ac
+++ b/configure.ac
@@ -3461,9 +3461,9 @@ if test X"$want_quiche" != Xno; then
   CLEANCPPFLAGS="$CPPFLAGS"
   CLEANLIBS="$LIBS"
 
-  LIB_QUICHE="-lquiche -ldl -lpthread"
+  LIB_QUICHE="-l:libquiche.a -ldl -lpthread"
   CPP_QUICHE="-I$OPT_QUICHE/include"
-  LD_QUICHE="-L$OPT_QUICHE/target/debug"
+  LD_QUICHE="-L$OPT_QUICHE/target/release"
 
   LDFLAGS="$LDFLAGS $LD_QUICHE"
   CPPFLAGS="$CPPFLAGS $CPP_QUICHE"
diff --git a/docs/HTTP3.md b/docs/HTTP3.md
index 64bd31a6e..c09bf0a76 100644
--- a/docs/HTTP3.md
+++ b/docs/HTTP3.md
@@ -17,7 +17,7 @@ QUIC libraries we're experiementing with:
 
 [ngtcp2](https://github.com/ngtcp2/ngtcp2)
 
-[quiche](https://github.com/ghedo/quiche)
+[quiche](https://github.com/cloudflare/quiche)
 
 ## Experimental!
 
@@ -68,12 +68,12 @@ Build BoringSSL (it needs to be built manually so it can be reused with curl):
 Build quiche:
 
      % cd ../..
-     % QUICHE_BSSL_PATH=$PWD/deps/boringssl cargo build
+     % QUICHE_BSSL_PATH=$PWD/deps/boringssl cargo build --release
 
 Clone and build curl:
 
      % cd ..
-     % git clone -b quiche https://github.com/ghedo/curl
+     % git clone https://github.com/curl/curl
      % ./buildconf
      % ./configure --with-ssl=$PWD/../quiche/deps/boringssl/.openssl --with-quiche=$PWD/../quiche --enable-debug
      % make -j`nproc`
diff --git a/lib/vquic/quiche.c b/lib/vquic/quiche.c
index 2adb47eb8..f6c4ad42c 100644
--- a/lib/vquic/quiche.c
+++ b/lib/vquic/quiche.c
@@ -38,7 +38,7 @@
 
 #define QUIC_MAX_STREAMS (256*1024)
 #define QUIC_MAX_DATA (1*1024*1024)
-#define QUIC_IDLE_TIMEOUT 60 /* seconds? */
+#define QUIC_IDLE_TIMEOUT 60 * 1000 /* milliseconds */
 
 static CURLcode process_ingress(struct connectdata *conn,
                                 curl_socket_t sockfd);
@@ -59,18 +59,18 @@ CURLcode Curl_quic_connect(struct connectdata *conn, curl_socket_t sockfd,
 
   infof(conn->data, "Connecting socket %d over QUIC\n", sockfd);
 
-  qs->cfg = quiche_config_new(QUICHE_VERSION_DRAFT18);
+  qs->cfg = quiche_config_new(QUICHE_PROTOCOL_VERSION);
   if(!qs->cfg)
     return CURLE_FAILED_INIT; /* TODO: better return code */
 
   quiche_config_set_idle_timeout(qs->cfg, QUIC_IDLE_TIMEOUT);
+  quiche_config_set_initial_max_data(qs->cfg, QUIC_MAX_DATA);
   quiche_config_set_initial_max_stream_data_bidi_local(qs->cfg, QUIC_MAX_DATA);
-  quiche_config_set_initial_max_stream_data_bidi_remote(qs->cfg,
-                                                        QUIC_MAX_DATA);
+  quiche_config_set_initial_max_stream_data_bidi_remote(qs->cfg, QUIC_MAX_DATA);
   quiche_config_set_initial_max_stream_data_uni(qs->cfg, QUIC_MAX_DATA);
-  quiche_config_set_initial_max_data(qs->cfg, QUIC_MAX_DATA);
-  quiche_config_set_initial_max_streams_bidi(qs->cfg, 10);
-  quiche_config_set_initial_max_streams_uni(qs->cfg, 10);
+  quiche_config_set_initial_max_streams_bidi(qs->cfg, QUIC_MAX_STREAMS);
+  quiche_config_set_initial_max_streams_uni(qs->cfg, QUIC_MAX_STREAMS);
+  quiche_config_set_application_protos(qs->cfg, (uint8_t *) "\x05hq-20", 6);
 
   result = Curl_rand(conn->data, qs->scid, sizeof(qs->scid));
   if(result)
@@ -143,7 +143,7 @@ static CURLcode flush_egress(struct connectdata *conn, int sockfd)
 {
   ssize_t sent;
   struct quicsocket *qs = &conn->quic;
-  static uint8_t out[1400];
+  static uint8_t out[1200];
 
   do {
     sent = quiche_conn_send(qs->conn, out, sizeof(out));
USe configure --with-ngtcp2 or --with-quiche

Using either option will enable a HTTP3 build.
Co-authored-by: Alessandro Ghedini <alessandro@ghedini.me>
@bagder bagder force-pushed the QUIC branch from 0b6ff0a to 74c6336 Jul 20, 2019
@bagder

This comment has been minimized.

Copy link
Member Author

bagder commented Jul 20, 2019

Plan: once this builds green (since nothing tests or uses HTTP/3 in the CI) I plan to merge this as the early and initial foundation to work further on. It should make it easier for everyone to collaborate on getting QUIC and HTTP/3 landed.

HTTP/3 support is experimental and should be treated as such in curl until further notice. We should make sure all documentation says so.

@bagder bagder closed this in 3af0e76 Jul 21, 2019
@bagder bagder deleted the QUIC branch Jul 21, 2019
caraitto added a commit to caraitto/curl that referenced this pull request Jul 23, 2019
USe configure --with-ngtcp2 or --with-quiche

Using either option will enable a HTTP3 build.
Co-authored-by: Alessandro Ghedini <alessandro@ghedini.me>

Closes curl#3500
@JenoP

This comment has been minimized.

Copy link

JenoP commented Oct 5, 2019

Hello folks,

Installed curl 7.66 in my Fedora29 box. http/3 support is not working.

pjeno@jeno:~/experimentation$ curl --http3-direct https://cloudflare-quic.com/
curl: option --http3-direct: is unknown

pjeno@jeno:~/experimentation$ curl --http3 https://cloudflare-quic.com/
curl: (1) Unsupported protocol

Do you've any ideas ?

Thanks!

@bagder

This comment has been minimized.

Copy link
Member Author

bagder commented Oct 6, 2019

You need to explicitly enable HTTP/3 when you build curl, which no distro-packaged curl does yet because this is still early days in HTTP/3 land.

If you want to experiment with and try out curl's HTTP/3 support, I recommend building curl from git master and following the instructions in docs/HTTP3.md .

@JenoP

This comment has been minimized.

Copy link

JenoP commented Oct 6, 2019

@bagder

This comment has been minimized.

Copy link
Member Author

bagder commented Oct 6, 2019

I know several of them provide specific URL for "large" downloads over HTTP/3 but if you want to benchmark you should probably rather run of them by yourself so that you know what you compare and can do it in a controlled way.

curl's HTTP/3 implementation is not polished much and is certainly not yet optimized for speed.

For dedicated client-side HTTP benchmarking, I'd recommend using h2load which also supports HTTP/3 at least in git. That'll give you a much better idea than curl.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.