Repository navigation
ArchXBench v1.5
The task set and complexity hierarchy are unchanged from v1. This release corrects and hardens the specifications, testbenches, and tooling that check those tasks, based on a full audit of every benchmark across all seven levels.
Testbench correctness fixes
Several benchmarks had bugs that could fail a correct design or pass an incorrect one regardless of implementation quality. These included off-by-one expected-value sequences, a reset/clock-edge race condition, a vacuous pass condition that accepted any output, and a fixed random seed that made "randomized" tests deterministic and non-representative.
Missing benchmark data
FIR filter tasks referenced coefficient parameters without specifying actual values in the spec files, while the golden output was generated from a particular fixed set. A correctly-implemented filter using a different (but equally valid) set of coefficients would not match. Coefficients are now specified explicitly, with scripting to prepare synthesis-ready RTL from them.
Several other Level 4-6 benchmarks (convolution, FFT, quantized matmul) similarly gained dedicated scripts to generate reference data and memory files that previously didn't exist.
Golden-model/testbench consistency fixes
A few Level 5-6 tasks fed the DUT different parameters (e.g. convolution kernels) than the Python golden model used to generate expected output, which could cause even a correct implementation to not match.
Broken build and file references fixed
Several benchmarks referenced input files, module names, or Makefile targets that didn't exist, and couldn't run at all prior to this release.
Scoring-script hardening
Comparison scripts across multiple benchmarks silently truncated on output-length mismatches, returned exit code 0 after printing a failure message (invisible to automated pipelines), or used tolerances that didn't make sense for the data type being compared. These are now consistent, hard-failing checks.
Simulator compatibility
Testbenches using SystemVerilog-only constructs (int in for-loops, ++, automatic functions, inline variable declarations in procedural blocks) were made portable to strict Verilog-2001 simulators.
Specification clarifications
Several tasks left signed/unsigned conventions, port semantics, or protocol timing (e.g. handshake signal behavior) undefined, causing correct implementations under one reasonable interpretation to fail under another. These are now stated explicitly.
Levels 2 and 3
Levels 2 and 3 received the same correctness-fix treatment as the others (pipelined multiplier/adder fixes, AES ground-truth regeneration, floating-point and iterative-solver testbench fixes), even though earlier internal audit notes for these levels weren't published separately.
Not yet included
Synthesis and PPA (power/performance/area) tooling exists internally for most benchmarks but isn't fully integrated into this release's public toolchain yet. This is planned for a future release.