PluginBench v0.4.0

PluginBench now runs every coding evaluation as a native Harbor task. Codex receives instructions, complete skills, and plugins through its native interfaces, and every treatment gets the same task prompt.

• Freeze treatment contents, file modes, runtime settings, and native discovery receipts. Isolate account plugins, and track root and child session usage without treating missing usage as zero.
• Grade, measure, review, and reuse vanilla results from one reconstructed canonical submission, with readiness and cleanup evidence.
• Report the code each submission adds: net cognitive and cyclomatic complexity, functions, files, hotspots, changed-line coverage, redundant added tests, and duplicated test lines. Measure each task's solution.patch offline as a reference baseline and report ratios to it.
• Rework the report with fixed arm order, grouped bars, a key-results table, passing-only summaries, and optional dated prices.json cost rows.
• Remove the local and Pier runners, bundled benchmark tasks, the multi-stage review flow, whole-snapshot complexity maxima, and the leave-one-out test-overlap measure. Report regeneration no longer reads multi-stage standards or ledger archives, and saved settings that select test overlap fail validation.

Install the attached Python wheel or source archive; see the README for Docker and Codex prerequisites and examples/evaluation.yaml. Run pluginbench run --config evaluation.yaml --dry-run before a paid evaluation. This remains experimental: the reference baseline and per-test redundancy have not yet run on a full saved evaluation.