Add CI performance benchmark suite

[Max042004]

Summary

Add a reproducible two-tier performance benchmark suite for detecting
regressions in pull requests.

• Tier 1 runs pinned lmbench workloads for syscall, process, and
  filesystem latency.
• Tier 2 measures Python startup, Git status, ripgrep, zstd, and Make
  against a deterministic committed corpus.
• Results include medians and raw samples in structured JSON.
• Baseline columns are prepared for elfuse, QEMU-based native Linux,
  and OrbStack.

CI integration

Add a benchmark leg to the self-hosted Apple Silicon runtime matrix.
The leg runs exclusively on the host to avoid interference from sibling
jobs, compares results in report-only mode, and uploads the JSON artifact.

Missing baseline metrics are treated as failures, preventing broken or
timed-out benchmarks from silently passing. Benchmark preparation and
cleanup commands are also bounded by timeouts.

The checked-in baseline has not yet been captured on the designated
self-hosted CI machine. It will be refreshed in a follow-up after the
benchmark leg is deployed and runner variance has been characterized.
Comparison therefore remains report-only for now.

Validation

• Full cold run: approximately 10m 19s
• Cached CI-equivalent run: approximately 9m 20s
• Baseline match exits successfully
• An injected 2x regression exits non-zero
• A missing baseline metric exits non-zero
• Benchmark output contains 9 lmbench and 6 application metrics

Closes #195

---

Summary by cubic

Adds a two-tier performance benchmark suite with a deterministic workload to catch syscall, process-startup, and app-level regressions. Runs on the self-hosted Apple Silicon runner under an exclusive host lock and posts a sanitized PR comment with benchmark deltas. (Issue #195)

• New Features

  • tests/bench-suite.sh: Tier 1 lmbench (lat_syscall/lat_proc/lat_fs) and Tier 2 app workloads (git status, rg, make, zstd) with warmups; guest-side timing for qemu-aarch64 via tests/bench-timeit.c; writes medians+samples to build/bench-results*.json.
  • Deterministic tests/bench-corpus for stable Tier 2 inputs; excluded from formatting to preserve baselines. Fixtures fetch app tools and cross-compile pinned lmbench.
  • Baselines in tests/bench-baseline.json for elfuse-aarch64, qemu-aarch64, and orbstack. scripts/bench-compare.py flags regressions (tunable via BENCH_REGRESSION_THRESHOLD) and treats missing metrics as NEW by default; scripts/bench-promote.py refreshes baselines.
  • CI: make bench / make bench-ci; BENCH_ENV selects elfuse-aarch64 (default) | qemu-aarch64 | orbstack | native. Benchmark leg isolates measurements with scripts/ci-host-lock.sh. .github/workflows/bench-report.yml uses scripts/bench-report.py to publish a sanitized advisory PR comment from the uploaded report artifact. Docs in docs/testing.md.

• Migration

  • Run: make bench (or BENCH_ENV=qemu-aarch64 make bench-ci).
  • Compare: scripts/bench-compare.py --report-only.
  • Refresh baselines: run make bench-ci per env, then scripts/bench-promote.py; commit the updated tests/bench-baseline.json.

^{Written for commit c2c8cbe. Summary will update on new commits.}

[jserv]

Once the benchmark is complete, upload the results to GitHub Pages, as rv32emu-bench does.

[henrybear327]

In general, LGTM. Just some nits, as the text / comment is bloating a bit I feel.

Thanks.

[henrybear327]

nit: rewrite the entire comment to make it clearer

[henrybear327]

ditto

[henrybear327]

nit: make the text concise

I think the current testing.md has grown to the size that's already too big and too complicated to read

[jserv]

Once the benchmark is complete, upload the results to GitHub Pages, as rv32emu-bench does.

I create elfuse-bench and @Max042004 can maintain it directly.