SecretMafia is an environment for evaluating agents on social deduction, persuasion, and strategic deception in a Mafia-style game. This environment wraps the SecretMafia implementation from TextArena, a framework for text-based game environments.
- Social deduction and role inference
- Persuasive argumentation and defense
- Strategic voting and coalition building
- Deception detection and counter-deception
SecretMafia does not require a sandbox. It has minimal compute requirements.
MIT.
There are two splits: train (150 tasks) and test (150 tasks). Each split contains 50 tasks across each of 3 variants:
- SecretMafia-v0
- SecretMafia-v0-train
- SecretMafia-v0-raw
Each task is seeded for reproducibility.
This is a sparse reward environment. Rewards are mapped from TextArena's native range of {-1, 0, 1} to {0.0, 0.5, 1.0} via (raw + 1) / 2.
We do not use LLM graders for this environment; reward is determined programmatically.
Game state is generated procedurally by the TextArena engine using seeded randomness. No external data files are required.
Agents are given a single tool:
send_message(message): Send a message, vote [vote X], or night action [kill X].
SecretMafia is a multi-turn environment.
High. Agents must navigate complex social dynamics with hidden information, persuading others and identifying opponents while concealing their own role, across alternating day and night phases with 6 players.
This environment requires an OpenAI API key (passed via secrets) to power the LLM opponents.
Agents in SecretMafia interact only with a social deduction game and have no access to external systems, the internet, or sensitive data. The environment does not present safety risks.
@software{textarena2024,
author = {Guertler, Leon and Banting, Wilfried and Pignatelli, Eduardo},
title = {TextArena},
year = {2024},
publisher = {GitHub},
url = {https://github.com/LeonGuertler/TextArena}
}