HTTP 3 Documentation
Fedora 37-39
sudo dnf -y install ccache make pkgconfig bison flex gcc-c++ clang autoconf automake libtool sudo git rpm-build distcc-server file wget openssl nghttp2 libnghttp2-devel sudo git rpm-build distcc-server file wget openssl hwloc hwloc-devel libcap-devel nghttp2 libnghttp2-devel sudo git rpm-build distcc-server file wget openssl nghttp2 libnghttp2-devel ImageMagick-devel ImageMagick-c++-devel hiredis-devel zlib-devel libmaxminddb-devel perl-ExtUtils-MakeMaker perl-Digest-SHA perl-URI curl tcl-devel java python3 httpd-tools procps-ng nmap-ncat python3-pip python3-gunicorn python3-requests python3-devel python3-psutil telnet golang openssl-devel pcre-devel initscripts ninja-build cmake
Ubuntu
apt-get update
apt-get -y install sudo
sudo apt-get -y install build-essential llvm ccache make pkgconf bison flex g++ clang gettext libc++-dev autoconf automake libtool autotools-dev git distcc file wget openssl hwloc intltool-debian clang-tools-14 clang-14 libssl-dev libexpat1-dev libpcre3-dev libcap-dev libhwloc-dev zlib1g-dev libjemalloc-dev libluajit-5.1-dev liblzma-dev libhiredis-dev libbrotli-dev libncurses-dev libgeoip-dev libmagick++-dev libmaxminddb-dev libcjose-dev libcjose0 libjansson-dev libunwind-dev ninja-build cmake
Fedora 37-39
sudo dnf -y install libev-devel jemalloc-devel python2-devel libxml2-devel c-ares-devel libevent-devel jansson-devel zlib-devel systemd-devel cargo perl-FindBin perl-IPC-Cmd
Ubuntu
sudo apt -y install libev-dev libjemalloc-dev python2-dev libxml2-dev libpython2-dev libc-ares-dev libsystemd-dev libevent-dev libjansson-dev zlib1g-dev cargo
You'll also need rust and cargo to build Quiche on next step. You may be able to install them by using package managers such as dnf or apt, but rust installed by those package manager may be too old. You need a quite new version of rust to satisfy Quiche's requirement. One of the ways to install the latest version of rust is using rustup. See https://rustup.rs for the detail.
There will be HTTTP/3 versions of curl and h2load under the /opt/bin directory
git clone git@github.com:apache/trafficserver.git || git clone https://github.com/apache/trafficserver.git
cd trafficserver
cd tools
BASE=/opt ./build_h3_tools.sh
cd ..
create the installation directory
sudo mkdir -p /opt/ats
USER=$(whoami) sudo chown $USER: /opt/ats
gcc configure
cmake -DOPENSSL_ROOT_DIR=/opt/boringssl -DOPENSSL_INCLUDE_DIR=/opt/boringssl/include -Dquiche_ROOT=/opt/quiche -DENABLE_QUICHE=TRUE -DCMAKE_INSTALL_PREFIX=/opt/ats -B build -G Ninja
clang configure
CC=/bin/clang CXX=/bin/clang++ cmake -DOPENSSL_ROOT_DIR=/opt/boringssl -DOPENSSL_INCLUDE_DIR=/opt/boringssl/include -Dquiche_ROOT=/opt/quiche -DENABLE_QUICHE=TRUE -DCMAKE_INSTALL_PREFIX=/opt/ats -B build -G Ninja
build and install
cmake --build build
cmake --install build
You will need to generate TLS certificates (see below) if you don't have any already. Below is an example configuration that will work with the benchmarking example.
ETC_DIR=/opt/ats/etc/trafficserver
RECORDS_CONFIG=$ETC_DIR/records.yaml
REMAP_CONFIG=$ETC_DIR/remap.config
/opt/ats/bin/traffic_ctl config set proxy.config.http.server_ports "8080 4443:ssl 4443:quic" -c $RECORDS_CONFIG
/opt/ats/bin/traffic_ctl config set proxy.config.udp.threads 1 -c $RECORDS_CONFIG
/opt/ats/bin/traffic_ctl config set proxy.config.diags.show_location 2 -c $RECORDS_CONFIG
/opt/ats/bin/traffic_ctl config set proxy.config.quic.initial_max_streams_bidi_in 100000 -c $RECORDS_CONFIG
/opt/ats/bin/traffic_ctl config set proxy.config.quic.initial_max_streams_bidi_out 100000 -c $RECORDS_CONFIG
/opt/ats/bin/traffic_ctl config set proxy.config.http.insert_response_via_str 3 -u -c $RECORDS_CONFIG
echo map / http://127.0.0.1/ @plugin=generator.so >> $REMAP_CONFIG
echo "dest_ip=* ssl_cert_name=$ETC_DIR/localhost.crt ssl_key_name=$ETC_DIR/localhost.key" >> $ETC_DIR/ssl_multicert.config
Create a certificate configuration file
tee -a apache.conf << EOF
[req]
distinguished_name = req_distinguished_name
x509_extensions = v3_req
prompt = no
[req_distinguished_name]
C = US
ST = CA
L = Mountain View
O = Traffic Server
CN = trafficserver.org
[v3_req]
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names
[alt_names]
DNS.1 = *.trafficserver.org
EOF
Create the private and public keys and move them to Traffic Server's configuration directory
openssl req -nodes -x509 -newkey rsa:4096 -keyout private-key.pem -out cert.pem -config apache.conf -sha256 -days 730
ETC_DIR=/opt/ats/etc/trafficserver
mv private-key.pem $ETC_DIR/localhost.key
mv cert.pem $ETC_DIR/localhost.crt
Installing as root or in a docker container
sudo chown -R nobody: /opt/ats/var/*
Start ATS
/opt/ats/bin/trafficserver start
Test one request
/opt/bin/curl -k --http3 https://127.0.0.1:4443/cache/1024
Run h2load
/opt/bin/h2load -n 500000 -c 100 --npn-list=h3 https://127.0.0.1:4443/cache/1024/asdfasdf
You should see output like this below
starting benchmark...
spawning thread #0: 100 total client(s). 500000 total requests
TLS Protocol: TLSv1.3
Cipher: TLS_AES_128_GCM_SHA256
Server Temp Key: X25519 253 bits
Application protocol: h3
progress: 10% done
progress: 20% done
progress: 30% done
progress: 40% done
progress: 50% done
progress: 60% done
progress: 70% done
progress: 80% done
progress: 90% done
progress: 100% done
finished in 8.53s, 58611.15 req/s, 66.12MB/s
requests: 500000 total, 500000 started, 500000 done, 500000 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 500000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 564.07MB (591470968) total, 72.93MB (76470968) headers (space savings 35.47%), 488.28MB (512000000) data
UDP datagram: 753325 sent, 1500566 received
min max mean sd +/- sd
time for request: 132us 86.27ms 1.67ms 1.03ms 91.72%
time for connect: 14.81ms 39.15ms 28.11ms 6.83ms 61.00%
time to 1st byte: 54.53ms 105.88ms 75.15ms 11.71ms 69.00%
req/s : 586.36 623.81 595.43 8.92 83.00%
HTTP/3 is implemented by ourselves although Quiche has HTTP/3 implementation, because we want to have full control on this part.
Unlike HTTP/1.x and HTTP/2, there is only Http3Session (no Http3ClientTransaction) at the moment because HTTP/3 is only supported on client side connections. HQSession is a super class for all HTTP sessions that runs on QUIC connection. HTTP/0.9 (Http09Session) is supported for test with other implementation. Although all sessions indirectly inherit VConnection, the interface between session and transaction varies because of characteristics of HTTP versions and underlying NetVConnection.
Similarly, HQTransaction is a super class for all HTTP transactions that runs on QUIC Connection (or HTTP/3 session). HTTP/0.9 is supported for the same reason as Http09Session. All transactions inherit ProxyTransaction, and the interface between HttpSM and each transaction is VConnection (VIO).
Although SSLNetVConnection and QUICNetVConnection have similarity in terms of secure connection that uses TLS, underlying implementations are completely different and it's difficult to have a common super class due to historical complicated implementation SSLNetVConnection that involves even UnixNetVConnection.
Also, many components relating TLS assume that TLS is only available on SSLNetVConnection and it's always possible to cast a NetVConection to SSLNetVConnection. This is no longer true with QUIC support.
Several mix-in classes were introduced to resolve this complexity and hard dependency. With this design, we never want to cast a NetVConnection instance to SSLNetVConnection nor QUICNetVConnection. You should always check if a specific feature you need is available instead. However, we also have interfaces/abstract classes for components that depend on QUIC transport. For example, HTTP/3 requires QUIC by design. You can use those QUIC specific abstract classes from components for HTTP/3. But again, even in HTTP/3 module, you should not use QUICNetVConnection directly. This allows us to test HTTP/3 module without having real QUIC connections.
You may see code that against this design because refactoring is not completed. We may have to violate the design sometimes because of that, but the violation should not be permanent.
Fedora 37
git clone git@github.com:apache/trafficserver-ci.git
cd trafficserver-ci
cd docker/http3/fedora37
sed -i "s/UID=1000/UID=$(id -u)/" Dockerfile
sed -i "s/GID=1000/GID=$(id -g)/" Dockerfile
sed -i "s/username/$(whoami)/" Dockerfile
docker-compose up -d
ssh config
echo -e "\nHost fedora_h3" >> ~/.ssh/config
echo "HostName localhost" >> ~/.ssh/config
echo "Port 666" >> ~/.ssh/config
ssh to Docker
ssh fedora_h3
Jump to Section 3 below and continue with those steps.