Accuracy Notes

Accuracy Notes

loadbearer is a relative instrument. Every number it produces is only as meaningful as the conditions it was measured under, and a comparison is only valid between runs made the same way. This page is the list of things that change a result and the things loadbearer deliberately doesn't try to measure.

Build flags

The CPU integer and float kernels compile to whatever vector width the target CPU allows. A default cargo build --release targets the portable SSE2 baseline (2-wide f64). Building with:

RUSTFLAGS="-C target-cpu=native" cargo build --release

lets them use AVX2 or AVX-512 where the machine has them, which raises the CPU numbers — sometimes substantially on the float and hash subtests.

This is fine as long as both machines you're comparing use the same build. The released binaries are portable builds; if you build native on one machine, build native on the other too, or compare a released binary against a released binary. compare can't detect this mismatch — it's on you.

Filesystem choice for the disk benchmark

• Point --target-dir at the actual disk you care about. The scratch file goes there.
• Not a RAM disk. On Linux, tmpfs accepts O_DIRECT as a no-op, so a scratch file on /tmp (if /tmp is tmpfs, which is common) measures memory bandwidth and labels it "disk". loadbearer detects a tmpfs/ramfs target on Linux and adds a note; on macOS/Windows it can't.
• Unbuffered I/O support. NTFS, ReFS, ext4, xfs and btrfs all honour O_DIRECT / FILE_FLAG_NO_BUFFERING. Some network, overlay, and encrypted filesystems reject it; loadbearer falls back to buffered I/O and records a note that the read figures may be cache-influenced. A buffered result is still useful for same-filesystem comparisons.
• Free space. A normal run needs ~1 GiB free in --target-dir (2 GiB at thorough); the run fails cleanly if there isn't enough.

Thermals and background load

A thin laptop that throttles under sustained all-core load is slower under sustained all-core load — that's a real property worth grading. But if you want a clean, repeatable number rather than a whatever-state-it-was-in number:

• Run on mains power, not battery (power profiles cap clocks).
• Let the machine cool between runs.
• Close other work — a browser doing background tabs will show up as variance.
• Use --duration thorough; the longer budget and larger iteration count average over short spikes.

The low / medium confidence flags exist precisely to tell you when a result was jittery. A low-confidence component in the "Why" line means "don't lean on this number" — re-run under better conditions.

Queue depth

The disk random-I/O subtests run at queue depth 1 — one I/O outstanding at a time. This is a latency-bound measurement and produces smaller numbers than a QD32 tool (fio, CrystalDiskMark, ATTO), which measure how much parallel I/O the drive and its controller can absorb.

QD1 was chosen because it's simple, single-threaded, portable, and still cleanly separates storage classes (NVMe ≫ SATA SSD ≫ HDD for random 4K). It is not the right tool if you specifically care about high-queue-depth throughput for a database or a busy server — benchmark that workload directly.

Reproducibility

• --seed N fixes the pseudo-random data the workloads use (buffer contents, random offsets). Two runs with the same seed, machine and build differ only by measurement noise.
• Seeding does not freeze timings. Wall-clock rates always vary run to run with thermal and scheduling state; that's what the repeated iterations, the median, and the confidence flag handle.
• The default seed is fixed, so runs are already seed-stable unless you change it.

What loadbearer does not measure

• GPU. No compute, graphics, or video-decode benchmark.
• Network. By design — a network result depends on the far end and the path, not the machine, and loadbearer makes no network calls at all.
• Sustained thermal behaviour as its own metric. A thorough run is a few minutes per component, long enough to catch throttling as lower scores and higher variance, but there's no dedicated "how does it do after 30 minutes flat out" test.
• Multi-threaded memory bandwidth. The memory subtests are single-threaded (see Benchmark Methodology).
• Real-world application performance. The kernels are representative microbenchmarks, not a compile, an encode, or a game. They rank hardware; they don't predict how long your specific workload takes.

When two runs disagree

If the same machine grades differently on two runs:

1. Check the confidence flags on both — a low component explains most of it.
2. Check they used the same --duration and the same build.
3. Check nothing else was running, and it wasn't on battery / warm from the last run.
4. If it still moves more than a few percent at thorough under good conditions, that spread is itself a finding about the machine.