Security Arena v1.0.0

Security Arena evaluates how LLM agents handle adversarial instructions while carrying out legitimate tasks. This is the first public release of the benchmark code, research and competition dataset, and supporting analyses.

Revised manuscript (September 2026) · arXiv record · Dataset guide

Downloads

• adaptive-adversaries-1.0.0.tar.gz: complete dataset and reproduction bundle, with source code, standalone wheel, schemas, Croissant metadata, and file manifests.
• adaptive_adversaries_benchmark-1.0.0-py3-none-any.whl: installable code package; numerical reproduction also needs the dataset.
• adaptive-adversaries-paper-2026-09.pdf: revised 26-page manuscript, including the appendices.
• SHA256SUMS: checksums for the three downloads.

The dataset contains the 945-battle frontier matrix, 540-battle three-attacker replication, provenance/history/defender-state controls, expanded fixed-attacker evaluation, 62-source/558-trial replay, 18,422 held-out competition battles, 93,291 development battles, and the benign-task records. Shared study membership, scoring profiles, and missing historical fields are documented.

Reproduce without model calls

tar -xzf adaptive-adversaries-1.0.0.tar.gz
cd adaptive-adversaries-1.0.0/code_release
python -m pip install '.[analysis]'
adaptive-adversaries-reproduce --data ../dataset --out reproduced
python ../dataset/validate_release.py

All 480 bundle entries were verified against the file manifests. The release preserves the validated candidate's executable code and derived scientific results; the final wheel installs independently of the source checkout.

Code is MIT with upstream notices; the dataset is CC BY 4.0. Cite Devina Jain, David Hartmann, and Chuan Li and the artifact version using the supplied citation files. Participant harness source and private identity maps are excluded. The organizer confirmed that the competition participation terms or existing permissions cover publication of the included interaction traces and benign-task records.