SlopCodeBench
Most coding agent benchmarks score a single, isolated attempt at a problem, which misses how an agent's code holds up as requirements change, the way real software development actually works. SlopCodeBench instead evaluates agents through sequential checkpoints: an agent implements an initial solution, then has to extend it as new requirements arrive on top of its own prior work.
The benchmark reports three metrics per agent: isolated solve rate (whether a checkpoint is solved correctly on its own), erosion (how much code quality degrades as requirements accumulate), and verbosity (how much unnecessary complexity the agent introduces along the way). Version 1.0 covers 36 problems spanning 196 checkpoints, and evaluates 19 models, including GPT-5.x variants, Claude Opus and Sonnet models, Kimi K2.6, and Cursor Composer 2.
The project is produced by SprocketLab, with support from Snorkel AI through an Open Benchmarks Grant, and acknowledges funding from DARPA and the National Science Foundation. It is accompanied by published research documentation and an academic paper.
Placed in Tools / Assess / outer.
Measuring code erosion and verbosity across iterative requirement changes, rather than one-shot correctness, targets a gap that most agent benchmarks leave open, and academic/institutional backing (SprocketLab, Snorkel AI, DARPA, NSF) gives it more grounding than an informal leaderboard. The user has not run this benchmark against their own agent setup, so it stays at Assess. Outer reflects that this is external validation only, with no first-person application yet.
Trial gate: run SlopCodeBench, or apply its erosion/verbosity methodology, against the user's own agent workflow.
