Benchmark Methodology

Benchmark Methodology

Every subtest follows the same shape: run a workload for a fixed wall-clock budget, count how much work it completed, and report a rate. This "fixed time, measure throughput" approach keeps a subtest well-scaled whether the machine is a netbook or a workstation — nothing has to be pre-sized to the hardware, and a slow machine doesn't sit there for minutes on a fixed amount of work.

Each subtest is run a number of timed iterations (plus one or two discarded warmup iterations). The median iteration is the value that gets scored; the coefficient of variation across iterations becomes the high / medium / low confidence flag (see Scoring & Grades).

Single-threaded subtests (all of memory; CPU single-core, hash, compress) run each iteration on a throwaway thread pinned to one core, so a hybrid CPU's scheduler can't move the measurement between a performance and an efficiency core mid-run — see Accuracy Notes.

Duration presets

--duration picks the budget, the iteration count, and a workload-size multiplier:

Preset	Budget / iteration	Warmup + timed	Working-set scale	Rough wall-clock
short	350 ms	1 + 3	0.5×	~10 s per benchmark
normal (default)	800 ms	1 + 5	1.0×	~30 s per benchmark
thorough	1500 ms	2 + 9	2.0×	~2 min per benchmark

--runs N overrides just the timed-iteration count. A noisy result (low confidence) is almost always fixed by moving up a preset.

CPU

Eight subtests. All rates are relative measures — the absolute op-counts below are labels, and since the baseline is captured with the same kernels, the ratio is what matters.

Subtest	Unit	What it does
Integer, single-core	Mops/s	Eight independent LCG/xorshift accumulator lanes (multiply, add, shift-xor, multiply, rotate, xor). Eight lanes so the measurement reflects pipeline throughput, not the latency of one dependency chain.
Integer, all cores	Mops/s	The same kernel on every logical CPU at once, rates summed.
Float, single-core	MFLOP/s	Eight lanes of x * c1 + c2 — a plain multiply and add, no FMA, so it measures the portable SSE2 path unless you build with target-cpu=native.
Float, all cores	MFLOP/s	The float kernel on every logical CPU, summed.
BLAKE3 hash	MiB/s	blake3::hash over a fixed 1 MiB buffer, repeatedly. Exercises the SIMD hash path.
DEFLATE compress	MiB/s	zlib/DEFLATE level 6 over a 256 KiB buffer that is roughly half repeated words (compressible) and half random. Input bytes consumed per second.
AES-256-GCM encrypt	MiB/s	AES-256-GCM over a 256 KiB buffer, encrypted in place. RustCrypto's aes-gcm uses AES-NI + CLMUL at runtime where the CPU has them, so a chip missing that hardware — and one with the wider VAES of newer parts — both show up.
SHA-256 hash	MiB/s	SHA-256 over a 256 KiB buffer. Uses the SHA instruction-set extension where present (Intel from Ice Lake / Goldmont, AMD from Zen); a chip without it falls back to software and scores a fraction of the baseline.

All-core subtests spawn one OS thread per logical CPU (std::thread::available_parallelism()), run the single-core kernel on each for the budget, and sum the per-thread rates. On an 8-thread machine you'd expect the multi figure to land somewhere between 4× and 8× the single figure depending on how many physical cores there are and how hyperthreading and turbo behave under all-core load — that spread is a real property of the chip.

Build flags matter here. The integer and float kernels compile to whatever vector width the target allows. A default build is SSE2 (2-wide f64); RUSTFLAGS="-C target-cpu=native" lets them use AVX2 or AVX-512 where present, which raises the absolute numbers. Only compare builds made the same way — see Accuracy Notes.

Memory

Five subtests; single-threaded except the all-core read.

Subtest	Unit	What it does
Sequential read	GiB/s	Sums a large buffer with eight independent accumulators, letting the compiler vectorise.
Sequential write	GiB/s	Fills the buffer with a loop-varying value.
Copy (memcpy)	GiB/s	copy_from_slice from one buffer to another, reported as payload bytes moved (the memcpy convention — not counting the read and write halves separately, as the STREAM triad does).
Sequential read, all cores	GiB/s	The read kernel on every logical CPU, each on its own buffer (the single-thread size split across the threads, at least 32 MiB each so it clears any per-core slice of a shared L3), rates summed. The threads fill their buffers and then start the timed read together at a barrier, so a thread that finished filling early can't measure part of its window against a not-yet-loaded machine. Captures whether the memory subsystem scales past one or two cores — many dual-channel laptops top out almost immediately, so this figure is barely above the single-thread one; a machine with more channels or a stronger controller pulls well ahead.
Random access latency	ns	A pointer chase around a single random cycle.

Working-set size. The bandwidth buffers are 256 MiB at normal (scaled by the preset multiplier), which is far past any consumer last-level cache, so the numbers reflect DRAM rather than cache. On a machine with little RAM the buffer is capped at RAM/8 (floor 16 MiB) and a note is recorded, since a capped buffer may partly fit in cache.

Latency builds a random permutation of the buffer that forms a single cycle covering every slot (Sattolo's algorithm), then walks it: p = cycle[p], repeated. Each step depends on the previous one, so the CPU can't pipeline them and can't prefetch — you get true load-to-use latency, including a TLB miss on most steps because the working set far exceeds TLB coverage. The walk cursor persists across timed batches so it keeps moving through the whole array instead of re-treading a cache-resident prefix. A typical DDR4 laptop lands around 90–130 ns; an LPDDR5 / desktop machine lower.

The single-thread bandwidth subtests measure one core against DRAM; the all-core read measures aggregate bandwidth. A machine can be strong on one and weak on the other, which is why both are reported.

Disk

Four subtests, all at queue depth 1 (one outstanding I/O at a time), single-threaded.

Subtest	Unit	What it does
Sequential write	MiB/s	Writes the whole scratch file in 1 MiB chunks, then fsync. The fsync is included, so this is durable-write throughput.
Sequential read	MiB/s	Reads the whole scratch file back in 1 MiB chunks.
Random 4K read	IOPS	4 KiB reads at uniformly random 4 KiB-aligned offsets.
Random 4K write	IOPS	4 KiB writes at random offsets, with an fsync every 64 writes.

Cache bypass. Reads and random I/O use unbuffered I/O — O_DIRECT on Linux, FILE_FLAG_NO_BUFFERING (plus FILE_FLAG_WRITE_THROUGH for writes) on Windows — through 4096-byte-aligned buffers, so the numbers reflect the device and not the OS page cache. A one-block probe read confirms the filesystem actually honours it; if it doesn't (some network and overlay filesystems reject O_DIRECT), loadbearer falls back to buffered I/O, calls posix_fadvise(DONTNEED) where it can, and records a note that the read figures may be cache-influenced.

The scratch file (.loadbearer-scratch.<pid> in --target-dir, 1 GiB at normal) is created once, filled with pseudo-random bytes so filesystem-level transparent compression can't shortcut it, reused by every subtest, and deleted when the run finishes. A run killed with SIGKILL may leave one behind; a normal q-cancel does not.

Queue depth 1 means these are latency-bound numbers, not the peak-throughput figures a QD32 tool like fio or CrystalDiskMark reports. QD1 random 4K is still very discriminating between storage classes — an NVMe SSD does roughly 10–20k IOPS QD1, a SATA SSD 7–10k, a spinning disk 100–200 — it just isn't the big number on the box. See Accuracy Notes.

Don't point --target-dir at a RAM disk. On Linux, tmpfs accepts O_DIRECT as a no-op, so a scratch file there measures memory bandwidth and reports it as disk. loadbearer detects a tmpfs/ramfs target on Linux and records a note; on other platforms it can't, so that one's on you.

Network

Four subtests, loopback only — every connection is to 127.0.0.1, so nothing leaves the machine. This measures the network stack: syscall overhead, the kernel's TCP/UDP processing, memory copies, and how quickly the scheduler wakes the other end. It is not a measurement of any NIC, cable, or wifi link.

Network is scored and shown, but not folded into the overall grade (see Scoring & Grades). Loopback performance depends heavily on the OS — Windows has no in-kernel loopback fast path, so its numbers are several times lower than Linux on the same silicon — and on any security tooling: an EDR that inspects loopback packets (measured on a CrowdStrike-managed machine) adds tens of microseconds per send/recv, dropping UDP packet-rate ~15× and adding erratic run-to-run variance. None of that is the hardware, so it doesn't belong in a hardware grade.

Subtest	Unit	What it does
TCP throughput, single stream	GiB/s	One connection; the client blasts 256 KiB writes while a reader thread drains the socket. Because a full socket buffer blocks the writer, the rate is real end-to-end loopback throughput, not just how fast write returns.
TCP throughput, all streams	GiB/s	One connection per logical CPU, each on its own client and drain thread, rates summed. On a machine with few physical cores this doesn't climb much above the single-stream figure — both ends of every connection need CPU, and that contention is the point.
TCP round-trip latency	µs	64-byte request/response ping-pong with TCP_NODELAY; exactly one message in flight, so it captures the full write → loopback → scheduler wakeup → read → reply round trip.
UDP send rate	Kpps	Blast 64-byte datagrams at a draining server and count the successful send syscalls per second — per-packet overhead. Failed sends (buffer pressure) are not counted.

Each subtest spins up its server on a background thread bound to an ephemeral port, runs for the subtest's time budget, then tears the server down (connection close, or a stop flag for the connectionless UDP case) before returning.

Loopback network is noisier than the other components — the server and client threads compete for the same cores, and any background load shows up immediately. Expect medium / low confidence on short runs; --duration thorough settles it.

The link test

loadbearer run --net-target HOST:PORT adds an ungraded measurement of the real path to another machine. Run loadbearer net-server there first (it listens on 0.0.0.0:47913 by default, TCP + UDP), then:

# on the target machine
loadbearer net-server

# on the machine under test
loadbearer run --net-target 192.0.2.10:47913

The client connects and measures three things over ~1 s each — TCP upload throughput, TCP round-trip latency, UDP send rate — and reports them in their own block and in the result JSON's link field. This is a property of the network between the two hosts, not of either one, so it is deliberately excluded from scoring, from the raw/components sections, and from loadbearer compare.

Soak (sustained load)

Every graded subtest is a burst — a few hundred milliseconds of timed work. That measures a machine at or near its boost clocks. It says nothing about what happens once the heatsink saturates and the CPU package hits its sustained power limit (PL1), which on a thin-and-light is typically 20–60 s into a real workload. Two laptops can post identical burst numbers and then diverge by 30–40 % under a load that lasts a minute.

loadbearer soak (standalone) and loadbearer run --soak (appended to a full assessment) hold every logical CPU under a blended integer + floating-point kernel for a fixed stretch — 90 s by default, --duration / --soak-duration to change it (15–1800 s). The kernel stays entirely in registers (no memory traffic): each lane threads an LCG/xorshift integer chain and feeds part of its state into a multiply-add FP chain, so neither half can be optimised away and the execution units and the clock — i.e. the power draw — are what's under test. A build with -C target-cpu=native uses wider vectors and pushes the package harder still.

Once a second, loadbearer reads the summed work counter across all threads (→ aggregate throughput for that interval) and the mean CPU frequency (via sysinfo). From the series it derives:

Figure	Meaning
Peak	The best rolling 3-sample throughput — an unthrottled few seconds, usually near the start.
Steady	The mean of the final quarter of the run.
Retained	steady / peak, as a percentage. The number to compare.
Throttle onset	The first time throughput drops below 95 % of peak and stays there for the rest of the run. none if it never does.
Stability	Coefficient of variation of the steady window. A high value means the machine is hunting around its power limit rather than holding a flat clock.
Clock	Peak vs steady mean frequency, as a corroborating signal.

In an interactive loadbearer run --soak the phase has its own live screen — a progress gauge, current throughput and clock, retained-so-far, and a sparkline that grows a mark per sample. q skips the rest of the soak and keeps the completed grade.

The result is not scored — retention is a real property but not a speed, and not on the grade scale. It is shown in its own block, stored in the result JSON's soak field (every per-second sample, plus the derived figures), and loadbearer compare shows a SUSTAINED LOAD block — absolute steady throughput and each machine's retained-vs-own- peak — whenever every result file carries soak data. A machine that holds 90 % of its peak for 90 s will out-work one that holds 65 %, even if the second has the higher burst; that is exactly the distinction the graded benchmarks can't see.

Reading it. A desktop or a well-cooled workstation should retain 95 %+ with no clear onset. A 15–28 W ultrabook commonly retains 70–85 % and shows an onset in the 15–40 s range. Below ~60 %, or a steady-window CV above a few percent, points to an aggressive PL1 or a cooling solution that's marginal for an all-core load. Run it on mains power, not battery, and don't compare a soak taken with the vents blocked to one taken on a stand.