Benchmark Methodology

Benchmark Methodology

Every subtest follows the same shape: run a workload for a fixed wall-clock budget, count how much work it completed, and report a rate. This "fixed time, measure throughput" approach keeps a subtest well-scaled whether the machine is a netbook or a workstation — nothing has to be pre-sized to the hardware, and a slow machine doesn't sit there for minutes on a fixed amount of work.

Each subtest is run a number of timed iterations (plus one or two discarded warmup iterations). The median iteration is the value that gets scored; the coefficient of variation across iterations becomes the high / medium / low confidence flag (see Scoring & Grades).

Duration presets

--duration picks the budget, the iteration count, and a workload-size multiplier:

Preset	Budget / iteration	Warmup + timed	Working-set scale	Rough wall-clock
short	350 ms	1 + 3	0.5×	~10 s per benchmark
normal (default)	800 ms	1 + 5	1.0×	~30 s per benchmark
thorough	1500 ms	2 + 9	2.0×	~2 min per benchmark

--runs N overrides just the timed-iteration count. A noisy result (low confidence) is almost always fixed by moving up a preset.

CPU

Six subtests. All rates are relative measures — the absolute op-counts below are labels, and since the baseline is captured with the same kernels, the ratio is what matters.

Subtest	Unit	What it does
Integer, single-core	Mops/s	Eight independent LCG/xorshift accumulator lanes (multiply, add, shift-xor, multiply, rotate, xor). Eight lanes so the measurement reflects pipeline throughput, not the latency of one dependency chain.
Integer, all cores	Mops/s	The same kernel on every logical CPU at once, rates summed.
Float, single-core	MFLOP/s	Eight lanes of x * c1 + c2 — a plain multiply and add, no FMA, so it measures the portable SSE2 path unless you build with target-cpu=native.
Float, all cores	MFLOP/s	The float kernel on every logical CPU, summed.
BLAKE3 hash	MiB/s	blake3::hash over a fixed 1 MiB buffer, repeatedly. Exercises the SIMD hash path.
DEFLATE compress	MiB/s	zlib/DEFLATE level 6 over a 256 KiB buffer that is roughly half repeated words (compressible) and half random. Input bytes consumed per second.

All-core subtests spawn one OS thread per logical CPU (std::thread::available_parallelism()), run the single-core kernel on each for the budget, and sum the per-thread rates. On an 8-thread machine you'd expect the multi figure to land somewhere between 4× and 8× the single figure depending on how many physical cores there are and how hyperthreading and turbo behave under all-core load — that spread is a real property of the chip.

Build flags matter here. The integer and float kernels compile to whatever vector width the target allows. A default build is SSE2 (2-wide f64); RUSTFLAGS="-C target-cpu=native" lets them use AVX2 or AVX-512 where present, which raises the absolute numbers. Only compare builds made the same way — see Accuracy Notes.

Memory

Four subtests, all single-threaded.

Subtest	Unit	What it does
Sequential read	GiB/s	Sums a large buffer with eight independent accumulators, letting the compiler vectorise.
Sequential write	GiB/s	Fills the buffer with a loop-varying value.
Copy (memcpy)	GiB/s	copy_from_slice from one buffer to another, reported as payload bytes moved (the memcpy convention — not counting the read and write halves separately, as the STREAM triad does).
Random access latency	ns	A pointer chase around a single random cycle.

Working-set size. The bandwidth buffers are 256 MiB at normal (scaled by the preset multiplier), which is far past any consumer last-level cache, so the numbers reflect DRAM rather than cache. On a machine with little RAM the buffer is capped at RAM/8 (floor 16 MiB) and a note is recorded, since a capped buffer may partly fit in cache.

Latency builds a random permutation of the buffer that forms a single cycle covering every slot (Sattolo's algorithm), then walks it: p = cycle[p], repeated. Each step depends on the previous one, so the CPU can't pipeline them and can't prefetch — you get true load-to-use latency, including a TLB miss on most steps because the working set far exceeds TLB coverage. The walk cursor persists across timed batches so it keeps moving through the whole array instead of re-treading a cache-resident prefix. A typical DDR4 laptop lands around 90–130 ns; an LPDDR5 / desktop machine lower.

Memory bandwidth is deliberately single-threaded in this version. One modern core already saturates a large fraction of a laptop's DRAM channel, and a single-thread number is easier to interpret; a multi-threaded aggregate may be added later.

Disk

Four subtests, all at queue depth 1 (one outstanding I/O at a time), single-threaded.

Subtest	Unit	What it does
Sequential write	MiB/s	Writes the whole scratch file in 1 MiB chunks, then fsync. The fsync is included, so this is durable-write throughput.
Sequential read	MiB/s	Reads the whole scratch file back in 1 MiB chunks.
Random 4K read	IOPS	4 KiB reads at uniformly random 4 KiB-aligned offsets.
Random 4K write	IOPS	4 KiB writes at random offsets, with an fsync every 64 writes.

Cache bypass. Reads and random I/O use unbuffered I/O — O_DIRECT on Linux, FILE_FLAG_NO_BUFFERING (plus FILE_FLAG_WRITE_THROUGH for writes) on Windows — through 4096-byte-aligned buffers, so the numbers reflect the device and not the OS page cache. A one-block probe read confirms the filesystem actually honours it; if it doesn't (some network and overlay filesystems reject O_DIRECT), loadbearer falls back to buffered I/O, calls posix_fadvise(DONTNEED) where it can, and records a note that the read figures may be cache-influenced.

The scratch file (.loadbearer-scratch.<pid> in --target-dir, 1 GiB at normal) is created once, filled with pseudo-random bytes so filesystem-level transparent compression can't shortcut it, reused by every subtest, and deleted when the run finishes. A run killed with SIGKILL may leave one behind; a normal q-cancel does not.

Queue depth 1 means these are latency-bound numbers, not the peak-throughput figures a QD32 tool like fio or CrystalDiskMark reports. QD1 random 4K is still very discriminating between storage classes — an NVMe SSD does roughly 10–20k IOPS QD1, a SATA SSD 7–10k, a spinning disk 100–200 — it just isn't the big number on the box. See Accuracy Notes.

Don't point --target-dir at a RAM disk. On Linux, tmpfs accepts O_DIRECT as a no-op, so a scratch file there measures memory bandwidth and reports it as disk. loadbearer detects a tmpfs/ramfs target on Linux and records a note; on other platforms it can't, so that one's on you.