Context
scripts/Measure-InstallPerformance.ps1 records elapsed time even when a measured installation action fails. The console summary omits the recorded Error property, allowing partial work to look like a successful benchmark result.
Observed behavior
The action exception is caught and stored, but a normal result object is still appended to JSONL and displayed with DurationS. The -All measurements used for #100 stopped on font-file collisions yet appeared in the timing table.
Expected behavior
Failed scenarios are visibly reported as failures and are excluded from successful timing comparisons.
Reproduction
- Run the harness with an action that throws, such as a known colliding
-All installation.
- Observe that a duration is written and displayed as though the scenario completed.
- Inspect the JSONL manually to discover the hidden
Error value.
Environment
PowerShell 7 on Windows. The behavior is platform-independent.
Regression
Introduced with the performance harness in #100.
Workaround
Inspect the Error field in every JSONL record before comparing durations.
Acceptance criteria
- Failed scenarios are unmistakably marked as failed in console and persisted output.
- Failed or partial runs are excluded from successful timing comparisons.
- Successful scenario output retains elapsed time and module-version metadata.
- The harness exits nonzero, or otherwise exposes a machine-readable failure state, when any requested scenario fails.
Technical decisions
Preserve failure evidence and elapsed diagnostics, but do not classify failed work as a successful benchmark sample.
Implementation plan
Context
scripts/Measure-InstallPerformance.ps1records elapsed time even when a measured installation action fails. The console summary omits the recordedErrorproperty, allowing partial work to look like a successful benchmark result.Observed behavior
The action exception is caught and stored, but a normal result object is still appended to JSONL and displayed with
DurationS. The-Allmeasurements used for #100 stopped on font-file collisions yet appeared in the timing table.Expected behavior
Failed scenarios are visibly reported as failures and are excluded from successful timing comparisons.
Reproduction
-Allinstallation.Errorvalue.Environment
PowerShell 7 on Windows. The behavior is platform-independent.
Regression
Introduced with the performance harness in #100.
Workaround
Inspect the
Errorfield in every JSONL record before comparing durations.Acceptance criteria
Technical decisions
Preserve failure evidence and elapsed diagnostics, but do not classify failed work as a successful benchmark sample.
Implementation plan