Scoring and Grades

Scoring & Grades

Scoring turns raw measurements into one number per component and one for the machine. It is deliberately simple and fully documented — a baseline division, a power curve, and geometric means.

The pipeline

1. Ratio to baseline

Each subtest's median value is divided by the matching number in the baseline:

ratio = value / baseline_value              (higher-is-better metrics)
ratio = baseline_value / value              (lower-is-better: latency)

A ratio of 1.0 means "exactly the baseline"; 2.0 means "twice as good"; 0.5 means "half as good". Inverting the latency metric means a faster (lower) latency still produces a ratio > 1, so every metric points the same way.

2. The display curve

score = 1000 · ratio ^ k

k is set by --curve-k (default 0.5, allowed range 0.05–3.0). The square root at k = 0.5 compresses the extremes: being twice as fast as the baseline is worth ~1414, not 2000; half as fast is ~707, not 500. This keeps a single very fast or very slow component from dominating the overall.

ratio	k = 0.3	k = 0.5	k = 1.0
0.25	660	500	250
0.5	812	707	500
1.0	1000	1000	1000
2.0	1231	1414	2000
4.0	1516	2000	4000

Lower k is more forgiving of a weak component and squashes a strong one; higher k spreads scores out and rewards outliers. k is stored in the result file, and compare warns if two files used different values.

3. Component score

component_score = geometric_mean(subtest_scores)

The geometric mean (multiply the values, take the nth root) is the standard choice for combining benchmark ratios: it treats "2× on one metric, 0.5× on another" as a wash, which the arithmetic mean does not, and it can't be gamed by one enormous subtest.

4. Overall score

overall_score = weighted_geometric_mean(component_scores, profile_weights)

i.e. exp( Σ wᵢ·ln(scoreᵢ) / Σ wᵢ ) over the CPU, memory and disk components that ran. The weights come from the profile; with the default general profile all weights are 1 and this is a plain geometric mean.

Network is left out of this step. It is still scored, graded and shown (and compare uses its raw metrics), but its loopback figures depend far too much on the host OS — Windows has no in-kernel loopback fast path — and on any security tooling: an EDR that inspects loopback packets adds tens of microseconds per syscall. That's a property of the software stack, not the hardware, so folding it into a hardware grade produced nonsense (a fast Windows laptop grading F on network and dropping two letters overall). Each component in the result file carries a graded: true|false flag.

The --net-target link probe and the --soak sustained-load test are not scored at all. The link probe measures the path between two hosts, not either one; the soak test reports throughput retention under a long load — how well a machine holds its clocks once it heats up — which is a real and useful property but not a speed, and not on the same scale as the rest. Both are measured, shown in their own report block, stored in the result JSON (link, soak), and used by compare in a separate block — but they never touch a grade.

Grades

The score maps to a letter:

Grade	Score
S	≥ 1400
A	≥ 1150
B	≥ 850
C	≥ 600
D	≥ 400
F	below 400

The bands are centred so that a machine matching the baseline everywhere scores exactly 1000 and lands in the middle of B. B is therefore "as good as a mid-range 2021 ultrabook"; A and S are genuinely faster than that; C and below are slower. Because the baseline is provisional, read a single machine's letter as a rough position, not a verdict — the comparison between two machines is the solid part.

Confidence

Each subtest's coefficient of variation (standard deviation ÷ mean) across its timed iterations sets a flag:

Flag	CV
high	< 3%
medium	< 8%
low	≥ 8%

A component takes the weakest flag among its subtests. Any component that ends up low is named in the run's "Why" section. Low confidence usually means thermal throttling mid-run, background load, or a preset too short for that subtest to settle — random 4K write in particular is jittery on SSDs because of the drive's own garbage collection. Re-run with --duration thorough, on mains power, with other work closed.

Reading a report line

  CPU         921  [B]   ███████████████░░░░░░░░░
    Integer, all cores           37325.3 Mops/s     0.83x     911  high

• 921 / [B] — the component score and its grade.
• 37325.3 Mops/s — the raw median measurement.
• 0.83x — ratio to baseline: this machine did 83% of the baseline's all-core integer throughput.
• 911 — that subtest's score, 1000 · 0.83^0.5.
• high — confidence: the timed iterations agreed closely.

The component score (921) is the geometric mean of all six CPU subtest scores, which is why it doesn't equal any single row.

Profiles

"Which machine is better" depends on what you'll do with it, so the overall score is a weighted geometric mean of the CPU, memory and disk component scores. --profile picks the weights:

Profile	CPU	Memory	Disk	For
general (default)	1.0	1.0	1.0	No particular workload in mind.
dev-workstation	1.4	1.0	1.3	Compiling, containers, version control — CPU-heavy with a lot of small file I/O.
content-creation	1.4	1.3	0.8	Video encode, 3D render, large-image editing — CPU and memory bandwidth bound, disk less so.
server	1.3	1.0	1.4	Sustained throughput under load — disk I/O and CPU dominate.

Network carries no weight because it isn't in the overall (see step 4 above); --net-target link results aren't scored at all.

Weighting is at the component level, not the subtest level — a profile can't currently say "favour multi-core CPU specifically". The profile is recorded in the result file, and compare notes when two files were scored with different profiles (their stored overall scores weight things differently, though compare's own verdict ignores the stored scores entirely — see Comparing Machines).

Only general changes nothing; the other three will move a machine's overall grade up or down depending on where its strengths are. A machine with a fast CPU and a slow disk grades better under content-creation than under server.