Repository navigation
Performance Baseline
This page records SPACK's local performance baseline for the Refine AOT benchmark. The goal is regression tracking, not a production SLA.
The benchmark compares two runtime modes with the same generated frontend build output:
| Mode | Description |
|---|---|
| Direct directory runtime |
spack-runtime serves the frontend dist directory directly. |
| AOT bundle runtime |
spack-compiler compiles the same dist directory into app.spack; spack-runtime serves the bundle. |
Each round runs four k6 workloads:
| Scenario | Meaning |
|---|---|
| direct page | Full frontend page-load model against direct directory runtime. |
| AOT page | Full frontend page-load model against AOT bundle runtime. |
| direct static | Hot static asset request against direct directory runtime. |
| AOT static | Hot static asset request against AOT bundle runtime. |
$env:PATH='D:\msys64\usr\bin;' + $env:PATH
$env:SPACK_RUNTIME_BENCH_BUILD='false'
$env:SPACK_COMPILER_BENCH_BUILD='false'
$env:SPACK_PERF_ROUNDS='3'
$env:K6_VUS='256'
$env:K6_DURATION='2m'
task perf:aot:refine:baselineThe image matrix variables are forwarded to the compiler container. Keep SPACK_IMAGE_ENABLE=false for a pure static baseline, or enable it to measure compiler-generated image variants.
Generated files:
tmp/k6/results/refine-aot-baseline-summary.md
tmp/k6/results/refine-aot-baseline-summary.json
tmp/k6/results/*-baseline-rN.json
tmp/k6/results/refine-aot-startup-baseline-rN.json
Run date: 2026-06-26.
| Item | Value |
|---|---|
| Host OS | Microsoft Windows 11 Pro for Workstations, version 10.0.26200, build 26200 |
| Host CPU | AMD Ryzen 7 9800X3D 8-Core Processor |
| Host CPU topology | 8 physical cores, 16 logical processors |
| Host memory | 61.56 GiB |
| Docker Desktop | Client 29.5.3, Server 29.5.3, Compose 5.1.4 |
| Docker engine OS | Docker Desktop Linux VM, x86_64, WSL2 kernel 6.18.33.1-microsoft-standard-WSL2 |
| Docker VM resources | 16 CPUs, 32,360,771,584 bytes memory, about 30.13 GiB |
| Go | go1.26.4 windows/amd64 |
| Node.js | v24.15.0 |
| pnpm | 10.33.4 |
| Bash | MSYS2 Bash 5.3.15 |
| Runtime benchmark image |
spack-k6-bench:local, sha256:86d46ba8b02ece272d54e3b92d35508ebc2c45223eddea5adaef3ea1ec944f56
|
| Compiler benchmark image |
spack-compiler-bench:local, sha256:5d35e5926fb24e493379720505851d0a8ed34d697a742798f15a8fa6b6ec7247
|
| k6 image |
grafana/k6:latest, sha256:632ddbc81a4a9fdc9e597da91ab1d8fcf1916dd988b43b4a4559d2f8d8e73d47
|
The Refine AOT benchmark now also records startup readiness samples immediately after Docker Compose starts the direct and AOT runtime containers. Each round writes:
tmp/k6/results/refine-aot-startup-baseline-rN.json
Each file contains direct runtime ready time and AOT runtime ready time in milliseconds. These samples are intended to catch regressions in bundle startup extraction and direct catalog bootstrap time.
Configuration: 3 rounds, 256 VUs, 2 minutes per k6 workload.
| Scenario | Failed max | Requests/s mean | Requests/s stdev | p95 mean | p99 mean |
|---|---|---|---|---|---|
| direct page | 0.000000 | 1272.67 | 220.76 | 3170.03 ms | 4460.87 ms |
| AOT page | 0.000000 | 1780.89 | 180.80 | 2368.22 ms | 3117.54 ms |
| direct static | 0.000000 | 1533.28 | 238.38 | 836.82 ms | 1696.54 ms |
| AOT static | 0.000000 | 2202.64 | 46.09 | 597.17 ms | 1345.95 ms |
| Comparison | Requests/s delta | p95 delta | p99 delta |
|---|---|---|---|
| AOT page vs direct page | +39.94% | -25.29% | -30.11% |
| AOT static vs direct static | +43.66% | -28.64% | -20.66% |
The AOT path has a stable advantage in this local Docker Desktop baseline:
- Page-load throughput is about 40% higher.
- Hot static asset throughput is about 44% higher.
- p95 and p99 latencies are lower for both page and static workloads.
- All scenarios completed with zero failed requests across all three rounds.
The direct runtime remains important as the zero-compile deployment path. The AOT runtime is the preferred performance path when users can run the compiler in CI or during image build.
These numbers are machine-local and should not be treated as a production performance guarantee.
Important sources of variance:
- Docker Desktop and WSL2 scheduling.
- k6 and SPACK containers running on the same host.
- Windows filesystem and bind-mount behavior.
- Local CPU load during the run.
-
grafana/k6:latestis not pinned by digest in the benchmark compose file yet.
For release-quality baselines, store the raw k6 JSON, machine profile, image digests, configuration, and commit SHA with the release artifact.