Release v1.0.2 for templates/template_autoscientists.
Publication
- Version: 1.0.2
- GitHub release: https://github.com/docxology/template_autoscientists/releases/tag/v1.0.2
- DOI: https://doi.org/10.5281/zenodo.20931927
- Zenodo: https://zenodo.org/records/20931927
- PDF SHA-256:
0af391375b14eb397812a8050657e2980fbc3a768e6fb108aa2f7eff46773e16
Abstract
Recent work on AutoScientists coordinates self-organizing teams of language-model agents through a small set of shared mechanisms: a champion-and-experiment-log shared state, a registry of retired dead-end directions, effect-size ranking of candidate directions, noise-band confirmation of claimed improvements, and stagnation-driven reorganization of teams. This exemplar provides a deterministic, standalone reference implementation of those mechanisms and studies them honestly as a testbed rather than as a performance claim.
We make the comparison fair by holding the total number of objective evaluations fixed: coordinated teams partition a single sequential experiment budget rather than adding parallel compute. Under that matched budget, coordination cannot — and in our results does not — beat a single-thread baseline on the final champion metric; we report the actual numbers and claim no speedup. What the testbed does demonstrate are two distinct, independently measurable benefits. First, noise-robustness: because the objective is stochastic, a single observed gain can be a draw of evaluation noise, so we separate the reported champion metric from the clean noise-free ground truth and show that noise-band confirmation shrinks the gap between them by roughly an order of magnitude — with confirmation on, the final champion's reported metric sits