Skip to content

http-kit: add Clojure framework - #1090

Merged
MDA2AV merged 2 commits into
MDA2AV:mainfrom
Ramblurr:pr/clojure-http-kit
Aug 8, 2026
Merged

http-kit: add Clojure framework#1090
MDA2AV merged 2 commits into
MDA2AV:mainfrom
Ramblurr:pr/clojure-http-kit

Conversation

@Ramblurr

@Ramblurr Ramblurr commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Description

Adds the http-kit Clojure framework.


PR Commands — comment on this PR to trigger (requires collaborator approval):

Command Description
/benchmark -f <framework> Run every test the framework subscribes to
/benchmark -f <framework> -t <test> Run one test only
/benchmark -f <framework> --save Run and save results (updates the leaderboard on merge)
/benchmark -f <framework> -t <test> --save Run one test and save results
/benchmark -f <framework> --compare <other> Measure the deltas against another framework instead of this one
/benchmark-multiple -f <fw1>,<fw2>,... Benchmark several frameworks in one run — takes -t and --save too; saved results land in a single commit
/benchmark-multiple --save No -f needed: benchmark and save every framework the PR touches
/benchmark-test -t <test> Benchmark all enabled frameworks subscribed to <test> and save the results

For /benchmark, always specify -f <framework>; the flags combine in any order. Results come back as a comment with a per-profile table of RPS, p99, CPU and memory — one table per framework on multi runs. A new benchmark comment while a run is in flight queues behind it (one deep) instead of cancelling it. For multi-framework PRs (dependency bumps, same-language refactors) prefer /benchmark-multiple, which runs everything in a single job and commits all saved results together, so no run overwrites another. --compare works on single-framework runs only.

What the deltas are measured against. By default, this framework's own results published on main - answering "did this change help?". When you are tuning a variant or a successor entry, --compare re-bases them on another entry instead:

/benchmark -f genhttp-11 --compare genhttp

The reply states which baseline it used, and profiles the other framework does not run show n/a rather than a delta.


Run benchmarks locally

You can validate and benchmark your framework locally with the lite script — no CPU pinning, fixed connection counts, all load generators run in Docker.

./scripts/validate.sh <framework>
./scripts/benchmark-lite.sh <framework> baseline
./scripts/benchmark-lite.sh --load-threads 4 <framework>

Requirements: Docker Engine on Linux. Load generators (gcannon, h2load, h2load-h3, wrk, ghz) are built as self-contained Docker images on first run.

@MDA2AV

MDA2AV commented Aug 8, 2026

Copy link
Copy Markdown
Owner

/benchmark -f http-kit --save

@github-actions

github-actions Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

👋 Benchmark request received. A collaborator will review and approve the run.

@github-actions

github-actions Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: http-kit | Test: all tests

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 157,646 520.9% 1.8GiB ~0% ~0%
baseline 4096 154,873 518.4% 2.0GiB ~0% ~0%
pipelined 512 287,743 804.4% 1.0GiB ~0% ~0%
pipelined 4096 294,008 800.0% 1.1GiB ~0% ~0%
limited-conn 512 114,067 417.5% 1.5GiB ~0% ~0%
limited-conn 4096 113,158 414.0% 1.5GiB ~0% ~0%
json 4096 87,746 6201.6% 2.3GiB ~0% ~0%
json-comp 512 44,557 6242.7% 3.3GiB ~0% ~0%
json-comp 4096 46,074 6345.7% 3.9GiB ~0% ~0%
json-comp 16384 43,836 6234.8% 4.0GiB ~0% ~0%
upload 32 288 132.4% 4.7GiB ~0% ~0%
upload 256 242 149.6% 9.1GiB ~0% ~0%
static 1024 102,853 1751.4% 2.7GiB ~0% ~0%
static 4096 98,271 1719.2% 2.5GiB ~0% ~0%
static 6800 99,956 1653.4% 2.8GiB ~0% ~0%
async-db 1024 56,713 6445.6% 3.0GiB ~0% ~0%
Full log
[info] server ready

[run 1/3]
Running 5s test @ http://localhost:8080
  64 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     8.90ms    9.42ms 284.35ms   95.75%
    Req/Sec     3.03k     2.49k   12.19k    71.70%
  474855 requests in 5.10s, 28.20GB read
  Socket errors: connect 0, read 0, write 0, timeout 196
Requests/sec:  93103.31
Transfer/sec:      5.53GB
[info] CPU 2296.8% | Mem 2.5GiB

[run 2/3]
Running 5s test @ http://localhost:8080
  64 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    38.74ms  154.76ms   1.70s    96.81%
    Req/Sec     2.36k     1.97k   13.25k    72.89%
  501204 requests in 5.10s, 29.76GB read
  Socket errors: connect 0, read 0, write 0, timeout 118
Requests/sec:  98271.73
Transfer/sec:      5.84GB
[info] CPU 1719.2% | Mem 2.5GiB

[run 3/3]
Running 5s test @ http://localhost:8080
  64 threads and 4096 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    54.15ms  169.01ms   1.81s    96.34%
    Req/Sec     1.70k     0.99k   12.47k    64.26%
  479341 requests in 5.10s, 28.47GB read
  Socket errors: connect 0, read 0, write 0, timeout 110
Requests/sec:  93978.54
Transfer/sec:      5.58GB
[info] CPU 1567.6% | Mem 3.0GiB

=== Best: 98271 req/s (CPU: 1719.2%, Mem: 2.5GiB) ===
[info] saved results/static/4096/http-kit.json
httparena-bench-http-kit
httparena-bench-http-kit

==============================================
=== http-kit / static / 6800c (tool=wrk) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
Running 5s test @ http://localhost:8080
  64 threads and 6800 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    11.16ms   10.17ms 286.84ms   95.90%
    Req/Sec     2.76k     2.45k   16.59k    80.33%
  468538 requests in 5.10s, 27.82GB read
Requests/sec:  91870.09
Transfer/sec:      5.46GB
[info] CPU 1903.5% | Mem 2.3GiB

[run 2/3]
Running 5s test @ http://localhost:8080
  64 threads and 6800 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    21.98ms   81.37ms   1.83s    98.66%
    Req/Sec     2.75k     2.60k   10.93k    79.02%
  491011 requests in 5.10s, 29.16GB read
Requests/sec:  96269.89
Transfer/sec:      5.72GB
[info] CPU 1666.1% | Mem 2.8GiB

[run 3/3]
Running 5s test @ http://localhost:8080
  64 threads and 6800 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    15.79ms   23.43ms 941.93ms   99.43%
    Req/Sec     2.69k     2.73k   12.69k    80.23%
  509792 requests in 5.10s, 30.27GB read
  Socket errors: connect 0, read 0, write 0, timeout 183
Requests/sec:  99956.08
Transfer/sec:      5.94GB
[info] CPU 1653.4% | Mem 2.8GiB

=== Best: 99956 req/s (CPU: 1653.4%, Mem: 2.8GiB) ===
[info] saved results/static/6800/http-kit.json
httparena-bench-http-kit
httparena-bench-http-kit
[info] skip: http-kit does not subscribe to static-tls

==============================================
=== http-kit / async-db / 1024c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     1024 (16/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 5
  Expected:  200
  Duration:  10s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   10.20ms   7.86ms   19.40ms   47.70ms   107.70ms

  540569 requests in 10.00s, 539826 responses
  Throughput: 53.97K req/s
  Bandwidth:  206.76MB/s
  Status codes: 2xx=539826, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 539826 / 539826 responses (100.0%)
  Reconnects: 21310
  Per-template: 107770,108137,108233,107974,107712
  Per-template-ok: 107770,108137,108233,107974,107712
[info] CPU 5924.6% | Mem 2.9GiB

[run 2/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     1024 (16/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 5
  Expected:  200
  Duration:  10s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   10.39ms   8.54ms   19.00ms   33.40ms   59.60ms

  567800 requests in 10.00s, 567139 responses
  Throughput: 56.70K req/s
  Bandwidth:  217.50MB/s
  Status codes: 2xx=567139, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 567138 / 567139 responses (100.0%)
  Reconnects: 22501
  Per-template: 112896,113338,113771,113840,113293
  Per-template-ok: 112896,113338,113771,113840,113293
[info] CPU 6445.6% | Mem 3.0GiB

[run 3/3]
gcannon v0.5.3
  Target:    localhost:8080/
  Threads:   64
  Conns:     1024 (16/thread)
  Pipeline:  1
  Req/conn:  25
  Templates: 5
  Expected:  200
  Duration:  10s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   11.94ms   9.88ms   21.50ms   38.10ms   70.20ms

  567210 requests in 10.00s, 566555 responses
  Throughput: 56.64K req/s
  Bandwidth:  217.16MB/s
  Status codes: 2xx=566555, 3xx=0, 4xx=0, 5xx=0
  Latency samples: 566555 / 566555 responses (100.0%)
  Reconnects: 22445
  Per-template: 112798,113324,113810,113591,113032
  Per-template-ok: 112798,113324,113810,113591,113032
[info] CPU 6479.3% | Mem 3.0GiB

=== Best: 56713 req/s (CPU: 6445.6%, Mem: 3.0GiB) ===
[info] input BW: 3.79MB/s (avg template: 70 bytes)
[info] saved results/async-db/1024/http-kit.json
httparena-bench-http-kit
httparena-bench-http-kit
[info] skip: http-kit does not subscribe to crud
[info] skip: http-kit does not subscribe to fortunes
[info] skip: http-kit does not subscribe to baseline-h2
[info] skip: http-kit does not subscribe to static-h2
[info] skip: http-kit does not subscribe to baseline-h2c
[info] skip: http-kit does not subscribe to json-h2c
[info] skip: http-kit does not subscribe to baseline-h3
[info] skip: http-kit does not subscribe to static-h3
[info] skip: http-kit does not subscribe to gateway-64
[info] skip: http-kit does not subscribe to gateway-h3
[info] skip: http-kit does not subscribe to production-stack
[info] skip: http-kit does not subscribe to unary-grpc
[info] skip: http-kit does not subscribe to unary-grpc-tls
[info] skip: http-kit does not subscribe to stream-grpc
[info] skip: http-kit does not subscribe to stream-grpc-tls
[info] skip: http-kit does not subscribe to echo-ws
[info] skip: http-kit does not subscribe to echo-ws-pipeline
[info] skip: http-kit does not subscribe to echo-ws-limited
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/results/http-kit.json - 16 new, 16 total
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
[info] restoring loopback MTU to 65536

@MDA2AV
MDA2AV merged commit 3b22738 into MDA2AV:main Aug 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants