Multi-Agent LLM Behavioural Experiment Comparison
malbec is a lightweight, open Python testbed for running experiments on groups of interacting LLM agents. Install it, configure agents and an environment, and run a reproducible multi-agent experiment. Easily compare results across models, population sizes, and conditions.
Under development. This is an early placeholder release; the API is not yet stable and there is no usable functionality at this version.
- Agents composed of an LLM (frontier via API, or open-weight served locally) and a fully configurable harness: prompts, tools, information, and memory.
- Environments defining how agents and other entities interact, including principals that deploy and control agents.
- A controller that runs experiments from simple configuration files, with reproducible seeds and easy parameter sweeps.
- Automatic logging at multiple levels of detail, supporting open-data practices.
- A growing library of experiment templates to get researchers most of the way to a new experiment.
malbec grows out of research on selection dynamics in populations of LLM agents. See the pilot study: The Risks of Evolving AI are Near-Term and Broad.
Charlie Pilgrim (Google Scholar)
Apache 2.0