Observed slower performance of HTTP/3 vs HTTP/2 in libcurl tests

[pkropachev]

Hello Team,

I performed a simple test to compare the performance of HTTP/2 and HTTP/3 using libcurl. The test measures the total time (CURLINFO_TOTAL_TIME_T) of requests for each protocol.

I used a locally running Caddy server on the same machine as the test client. The server serves a 10MB file, and the configuration is minimal:

{
    servers {
        protocols h1 h2 h3
    }
}

caddy.crc.com:443 {
    log
    root * /home/pk/projects/caddy/files
    tls internal
    file_server browse
}

I am using libcurl from the master branch:

curl 8.21.0-DEV (x86_64-pc-linux-gnu) libcurl/8.21.0-DEV OpenSSL/4.0.0 zlib/1.2.11 c-ares/1.34.6 libpsl/0.21.0 nghttp2/1.69.0 ngtcp2/1.22.1 nghttp3/1.15.0
Release-Date: [unreleased]
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns mqtt mqtts pop3 pop3s rtsp smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS HSTS HTTP2 HTTP3 HTTPS-proxy HTTPSRR IPv6 Largefile libz PSL SSL threadsafe TLS-SRP UnixSockets

The test code is straightforward: I perform 10 sequential requests (the connection is established during the first request and reused for subsequent requests). Each protocol test was run multiple times.

#include <chrono>
#include <fstream>
#include <iostream>
#include <optional>
#include <thread>

#include <curl/curl.h>

struct TransferContext {
    std::ofstream file;

    TransferContext(const std::string &filename) {
        file.open(filename, std::ios::binary);
    }
};

size_t write_callback(void* contents, size_t size, size_t nmemb, void* userp) {
    size_t realsize = size * nmemb;
    TransferContext* context = static_cast<TransferContext*>(userp);

    if (context->file.is_open()) {
        context->file.write(static_cast<const char*>(contents), realsize);
    }

    return realsize;
}

void dump(CURL* curl) {
    long response_code = 0;
    curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &response_code);

    curl_off_t namelookup = 0.0;
    curl_off_t connect = 0.0;
    curl_off_t appconnect = 0.0;
    curl_off_t total = 0.0;

    curl_easy_getinfo(curl, CURLINFO_NAMELOOKUP_TIME_T, &namelookup);
    curl_easy_getinfo(curl, CURLINFO_CONNECT_TIME_T, &connect);
    curl_easy_getinfo(curl, CURLINFO_APPCONNECT_TIME_T, &appconnect);
    curl_easy_getinfo(curl, CURLINFO_TOTAL_TIME_T, &total);

    long http_ver;
    curl_easy_getinfo(curl, CURLINFO_HTTP_VERSION, &http_ver);
    const char* ver_str = "Unknown";
    if (http_ver == CURL_HTTP_VERSION_1_0) ver_str = "1.0";
    else if (http_ver == CURL_HTTP_VERSION_1_1) ver_str = "1.1";
    else if (http_ver == CURL_HTTP_VERSION_2_0) ver_str = "2";
    else if (http_ver == CURL_HTTP_VERSION_3) ver_str = "3";

    fprintf(stdout, "res_code=%ld, dns=%ld, tcp=%ld, tls=%ld, total=%ld, http_ver=%s\n",
            response_code, namelookup, connect, appconnect, total, ver_str);
}

int main(int argc, char* argv[]) {
    using namespace std::chrono_literals;

    std::optional<long> http_version;

    for (int i = 1; i < argc; ++i) {
        std::string_view arg = argv[i];
        if (arg == "-1") {
            http_version = CURL_HTTP_VERSION_1_1;
        } else if (arg == "-2") {
            http_version = CURL_HTTP_VERSION_2_0; 
        } else if (arg == "-3") {
            http_version = CURL_HTTP_VERSION_3;
        } else {
            std::cerr << "Unknown argument: " << arg << ", ignoring...\n";
            std::cerr << "Usage: " << argv[0] << " [-1 | -2 | -3]\n";
        }
    }

    const char* url = "https://caddy.crc.com/10MB.txt";
    const char* out_filename = "output.txt";

    curl_global_init(CURL_GLOBAL_ALL);
    CURL* curl = curl_easy_init();

    if (!curl) {
        std::cerr << "Failed to init curl!" << std::endl;
        curl_global_cleanup();
        return 1;
    }

    TransferContext context(out_filename);

    curl_easy_setopt(curl, CURLOPT_URL, url);
    if (http_version.has_value()) {
        curl_easy_setopt(curl, CURLOPT_HTTP_VERSION, http_version.value());
    }
    curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_callback);
    curl_easy_setopt(curl, CURLOPT_WRITEDATA, &context);
    // curl_easy_setopt(curl, CURLOPT_BUFFERSIZE, 512 * 1024L);

    curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 0L);
    curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0L);

    std::cout << "Start requests " << url << "...\n";
    for (int i = 0; i < 10; i++) {
        curl_easy_perform(curl);
        dump(curl);
        std::this_thread::sleep_for(1s);
    }
    std::cout << "done!" << std::endl;

    curl_easy_cleanup(curl);
    curl_global_cleanup();

    return 0;
}

Observed results on my computer:
HTTP/2:

Start requests https://caddy.crc.com/10MB.txt...
res_code=200, dns=99888, tcp=100237, tls=110955, total=135092, http_ver=2
res_code=200, dns=0, tcp=0, tls=0, total=26557, http_ver=2
res_code=200, dns=0, tcp=0, tls=0, total=28790, http_ver=2
res_code=200, dns=0, tcp=0, tls=0, total=25609, http_ver=2
res_code=200, dns=0, tcp=0, tls=0, total=26908, http_ver=2
res_code=200, dns=0, tcp=0, tls=0, total=27271, http_ver=2
res_code=200, dns=0, tcp=0, tls=0, total=25395, http_ver=2
res_code=200, dns=0, tcp=0, tls=0, total=23111, http_ver=2
res_code=200, dns=0, tcp=0, tls=0, total=26801, http_ver=2
res_code=200, dns=0, tcp=0, tls=0, total=26144, http_ver=2
done!

HTTP/3:

Start requests https://caddy.crc.com/10MB.txt...
res_code=200, dns=97662, tcp=103969, tls=110311, total=140119, http_ver=3
res_code=200, dns=0, tcp=0, tls=0, total=34015, http_ver=3
res_code=200, dns=0, tcp=0, tls=0, total=31879, http_ver=3
res_code=200, dns=0, tcp=0, tls=0, total=33339, http_ver=3
res_code=200, dns=0, tcp=0, tls=0, total=31396, http_ver=3
res_code=200, dns=0, tcp=0, tls=0, total=33335, http_ver=3
res_code=200, dns=0, tcp=0, tls=0, total=39715, http_ver=3
res_code=200, dns=0, tcp=0, tls=0, total=31878, http_ver=3
res_code=200, dns=0, tcp=0, tls=0, total=34271, http_ver=3
res_code=200, dns=0, tcp=0, tls=0, total=35821, http_ver=3
done!

From these results, HTTP/3 appears slower than HTTP/2.

I also tried running the Caddy server on a separate machine over a real network, but the outcome was the same: HTTP/2 consistently outperformed HTTP/3.

I find this behavior surprising.

Has anyone observed similar performance characteristics with HTTP/3? Could there be something in my setup or test methodology that is causing HTTP/3 to underperform?

[icing]

My observations from 2 years ago: https://eissing.org/icing/posts/curl-h3-performance/ . There have been some curl improvement since, but HTTP/1.1 and HTTP/2 outperform on throughput still.