The Baseline

The Baseline

The baseline is the set of reference measurements every raw value is compared against. It's what makes a score mean something on its own rather than only in a head-to-head.

reference-v1

The built-in baseline is embedded in the binary at compile time; its source is baseline/reference-v1.toml. The values describe a hypothetical mid-range 2021 thin-and-light laptop — 8-core mobile CPU, dual-channel LPDDR4x, a Gen3 NVMe SSD:

Component	Subtest	Baseline value
cpu	int_single	10000 Mops/s
cpu	int_multi	45000 Mops/s
cpu	float_single	8000 MFLOP/s
cpu	float_multi	70000 MFLOP/s
cpu	hash	4200 MiB/s
cpu	compress	68 MiB/s
cpu	aes_gcm	1800 MiB/s
cpu	sha256	1800 MiB/s
memory	bw_read	22 GiB/s
memory	bw_write	16 GiB/s
memory	bw_copy	14 GiB/s
memory	bw_read_mt	28 GiB/s
memory	latency	95 ns
disk	seq_write	1500 MiB/s
disk	seq_read	3000 MiB/s
disk	rand_read	15000 IOPS
disk	rand_write	40000 IOPS
network	tcp_stream	7.0 GiB/s
network	tcp_parallel	20.0 GiB/s
network	tcp_rtt	14.0 µs
network	udp_pps	450 Kpps

A run that hits every one of these scores 1000 on every component and grades straight B.

It is provisional

These numbers are synthetic anchors, not an average of real hardware. They're internally consistent and put a 2021-ish ultrabook around 1000, but they haven't been calibrated against a fleet. So:

• Treat a single machine's absolute score and letter as indicative — "around baseline", "clearly faster", "clearly slower".
• For anything that has to be defensible, use compare, which doesn't touch the baseline.
• If you have representative hardware, build your own baseline and the absolute scores become meaningful for your context.

If a benchmark subtest ever lacks a baseline entry, scoring fails with a hard error rather than silently skipping it — the baseline can't quietly fall out of sync with the code.

Recalibrating

loadbearer baseline with no arguments prints the embedded baseline. Given result files, it emits a new baseline whose every value is the geometric mean of that metric across the inputs:

# collect runs from machines you consider "the standard"
loadbearer run --output ref-1.json
loadbearer run --output ref-2.json
loadbearer run --output ref-3.json

# average them into a baseline and replace the shipped one
loadbearer baseline ref-1.json ref-2.json ref-3.json \
  --name reference-v2 \
  --description "our 2026 standard-issue laptops" \
  > baseline/reference-v1.toml

cargo build --release

Subtests missing from some inputs (e.g. one run used --only) are still averaged over whatever files have them, with a warning on stderr.

Guidance for a good baseline: use runs at the same --duration (ideally thorough), all built the same way (all target-cpu=native or all portable), on machines that genuinely represent your "par" hardware — the median of your fleet, not the best or worst. Three to five machines is plenty; the geometric mean is stable.

Result files

loadbearer run --output result.json writes a versioned document. schema is "loadbearer.result/1"; a future breaking change bumps the number.

Field	Contents
schema	"loadbearer.result/1"
tool_version	the loadbearer version that produced it
timestamp	RFC 3339, UTC
machine	full inventory — identical to loadbearer info --json (host, OS, CPU model/vendor/cores, RAM, disks)
config	profile, duration_preset, curve_k, seed, threads, baseline name, only
raw	every benchmark, every subtest: direction, unit, the median value, full stats (every timed run, median, mean, min, max, stddev, cv), and the confidence flag
components	scored: per component a score, grade, confidence, a graded flag (whether it counts toward the overall — false for network), and per subtest the value, baseline, ratio, score, confidence
overall	score, grade, profile, and the why lines — computed from the graded components only
link	present only when --net-target was used: target, tcp_upload_gibps, tcp_rtt_us, udp_send_kpps — ungraded, not part of raw/components

Because raw is preserved in full, a result file is a permanent record: it can be re-scored later against a different baseline or curve (the scored components are just one view of it), and compare and loadbearer baseline both work straight off raw without trusting the scores.

# what did this machine actually do, ignoring the grade?
jq '.raw[] | {id, subtests: [.subtests[] | {id, value, unit, confidence}]}' result.json

# just the headline
jq '{cpu: .components[0].grade, overall: .overall.score, why: .overall.why}' result.json