Reject mixed stdout benchmark report formats#340
Conversation
- Prevent JSON/CSV and human-readable reports from sharing stdout - Exit with an error when one of the reports needs an output file
📝 WalkthroughWalkthroughAdded early validation in the benchmark runner to prevent mixed structured and human-readable formats from writing to stdout simultaneously, detecting this conflict before rendering and terminating with an error. Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Possibly related PRs
Suggested labels
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Comment |
Benchmark Results386 benchmarks Interpreted: 🟢 35 improved · 🔴 270 regressed · 81 unchanged · avg -6.1% arraybuffer.js — Interp: 🟢 2, 🔴 7, 5 unch. · avg +0.6% · Bytecode: 🟢 3, 🔴 3, 8 unch. · avg +0.6%
arrays.js — Interp: 🔴 17, 2 unch. · avg -5.3% · Bytecode: 🟢 7, 12 unch. · avg +1.1%
async-await.js — Interp: 🔴 3, 3 unch. · avg +10.8% · Bytecode: 🟢 4, 2 unch. · avg +3.6%
base64.js — Interp: 🔴 6, 4 unch. · avg -7.6% · Bytecode: 🟢 2, 8 unch. · avg +0.6%
classes.js — Interp: 🟢 2, 🔴 19, 10 unch. · avg -3.4% · Bytecode: 🟢 6, 25 unch. · avg +2.1%
closures.js — Interp: 🔴 10, 1 unch. · avg -7.7% · Bytecode: 🟢 1, 10 unch. · avg +0.7%
collections.js — Interp: 🔴 8, 4 unch. · avg -3.1% · Bytecode: 🟢 9, 3 unch. · avg +3.2%
csv.js — Interp: 🔴 12, 1 unch. · avg -12.1% · Bytecode: 🟢 4, 9 unch. · avg +1.1%
destructuring.js — Interp: 🔴 21, 1 unch. · avg -8.1% · Bytecode: 🟢 4, 18 unch. · avg +1.3%
fibonacci.js — Interp: 🔴 8 · avg -8.0% · Bytecode: 🟢 1, 7 unch. · avg +0.7%
float16array.js — Interp: 🟢 4, 🔴 21, 7 unch. · avg -2.8% · Bytecode: 🟢 5, 🔴 9, 18 unch. · avg -0.4%
for-of.js — Interp: 🔴 7 · avg -7.3% · Bytecode: 🔴 1, 6 unch. · avg -0.6%
helpers/bench-module.js — Interp: 0 · Bytecode: 0
iterators.js — Interp: 🟢 11, 🔴 14, 17 unch. · avg -0.5% · Bytecode: 🟢 16, 🔴 4, 22 unch. · avg +1.4%
json.js — Interp: 🔴 20 · avg -10.0% · Bytecode: 🟢 2, 🔴 3, 15 unch. · avg +0.1%
jsx.jsx — Interp: 🔴 21 · avg -9.4% · Bytecode: 🟢 7, 14 unch. · avg +1.3%
modules.js — Interp: 🔴 9 · avg -11.5% · Bytecode: 🔴 5, 4 unch. · avg -1.2%
numbers.js — Interp: 🔴 8, 3 unch. · avg -7.9% · Bytecode: 🔴 2, 9 unch. · avg -1.1%
objects.js — Interp: 🔴 6, 1 unch. · avg -10.6% · Bytecode: 🟢 1, 🔴 1, 5 unch. · avg +0.9%
promises.js — Interp: 🔴 10, 2 unch. · avg -5.7% · Bytecode: 🟢 6, 🔴 1, 5 unch. · avg +1.7%
regexp.js — Interp: 🔴 11 · avg -12.6% · Bytecode: 🔴 8, 3 unch. · avg -3.2%
strings.js — Interp: 🔴 11, 8 unch. · avg -18.8% · Bytecode: 🟢 5, 🔴 9, 5 unch. · avg -4.4%
tsv.js — Interp: 🔴 9 · avg -16.1% · Bytecode: 🔴 4, 5 unch. · avg -0.9%
typed-arrays.js — Interp: 🟢 13, 🔴 2, 7 unch. · avg +5.1% · Bytecode: 🟢 1, 🔴 12, 9 unch. · avg -1.5%
uint8array-encoding.js — Interp: 🟢 3, 🔴 10, 5 unch. · avg -12.2% · Bytecode: 🟢 1, 🔴 8, 9 unch. · avg -7.4%
Measured on ubuntu-latest x64. Benchmark ranges compare cached main-branch min/max ops/sec with the PR run; overlapping ranges are treated as unchanged noise. Percentage deltas are secondary context. |
Suite Timing
Measured on ubuntu-latest x64. |
Summary
GocciaBenchmarkRunnerthat detects mixed stdout report targets and exits with an error.--outputinstead.Fixes #335