Repository navigation
A training eval where one score hid the side effect: rewarded to win, the model ended the world more #1852
KyleClouthier
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
I built a deterministic multi-agent environment (a real-time world war with pacts and a nuclear doomsday clock) and used it as a held-out eval for a training recipe.
Two copies of gpt-oss-20b, three rounds of expert iteration on their own best games, one line of the reward different. On 10 sealed wars:
The restraint copy looks perfect on the headline metric and doubles betrayal. Every turn logs the public message, the private plan and the orders, so said-versus-did is scored too. Seeds are committed before play; any model runs through an HTTP API, MCP or
pip install secondstrike.Small sample, a game, stated on the page. Write-up and data: https://secondstrike.io/research/rewarded-to-win-ended-the-world-more
Happy to hear how people here would score side effects like this.
All reactions