Repository navigation
AgentLab is a reproducible laboratory for empirical research on multi-agent LLM systems. Version 1.0 is not a set of benchmark numbers; it is a methodology, demonstrated end to end on Claude Haiku 5.5 across three studies:
- Artifact-resistant evaluation. A pilot that saturated because agents self-verified internally; removing the artifact (blind execution) exposed a real effect: external verification improves correctness by +0.075 [+0.044, +0.106] over matched orchestration, flat across team sizes — per-subtask insurance, not a scaling effect.
- Calibration gates and mechanistic benchmarking. A pre-registered test of error accumulation whose offline calibration gate rejected the first task generator before any live run. The experiment then showed the intervention failed to induce the error regime required to identify the proposed mechanism — the platform rejected its own hypothesis for the correct reason, with every number replay-verified.
- Model-calibrated benchmark generation. A closed-loop controller that calibrates a family's first-attempt failure rate to a requested band: 5% ± 2% converged in 2 iterations and validated on held-out seeds at 0.050 [0.034, 0.073]. For the 10% band, the calibration procedure correctly diagnosed that the current generator family cannot represent that operating point, and declared failure rather than approximating.
Every experiment ships as an immutable, sealed bundle with first-class provenance, deterministic replay, and analysis that consumes only recorded metrics.
AgentLab does not guarantee that every benchmark can be calibrated. It guarantees that unsupported operating regimes are detected and reported rather than silently approximated.