Debugging Tool Correctness Hashes

Correctness hash comparison

← Debugging Tools and Commands

The xemu performance-test hash helper compares deterministic baseline and candidate result records without treating missing evidence as success.

Property	Details
Supported OS	Windows, Linux, macOS with Python 3.
Install	Clone the exact xemu-perf-tests revision used by the campaign.
Verify	python3 --version
Primary output	See the workflow below.

When to use it

Performance improvements are unusable if they silently change deterministic guest results.

The repository helper:

xemu-perf-tests/utils/hash_compare.py

compares deterministic hashes such as framebuffer, work, and result checksums from baseline/candidate result records.

Compare one candidate

python3 utils/hash_compare.py \
  /results/baseline \
  /results/candidate

Compare several candidates and save JSON

python3 utils/hash_compare.py \
  /results/baseline \
  /results/candidate-a \
  /results/candidate-b \
  --json-out hash-comparison.json

Read the output

Human output resembles:

BASELINE ...
RUN ...
  test-name framebuffer=1 work=1 result=1
VERDICT PASS checked=3 mismatch=0 missing=0 extra=0
OVERALL PASS

Meaning:

• 1 = eligible hash matched;
• 0 = eligible hash differed;
• - = that hash was not eligible/available;
• PASS requires checked hashes and no missing/extra/mismatched records;
• UNVERIFIED means no eligible hashes were actually checked.

Do not convert UNVERIFIED into PASS.

---

Boundaries and common mistakes

A hash can validate only the data included in its contract. It does not replace image inspection, gameplay validation, audio checks, or platform-specific tests outside that contract.

UNVERIFIED, missing records, extra records, and ineligible hashes must remain visible. Never promote them to PASS.

Evidence to retain

• Tool name and exact version.
• Source commit/tree and executable SHA-256.
• The complete command or saved profile used.
• Input artifact hashes and the explicitly selected time or address range.
• Raw output plus any derived report; do not retain only a screenshot or conclusion.
• Admission failures, warnings, dropped events, incomplete rows, or unsupported operations.

Official references

• xemu-perf-tests

Related xemu guides

• Debugging Tools and Commands
• Debugging Lag and Hot Paths
• Performance Methodology