SHE: Trajectory-driven Safety Harness Evolution for LLM Agents

SHE evolves an explicit safety harness for tool-using LLM agents from execution trajectories. Rather than treating safety as a single monolithic prompt, it separates the harness into a System Prompt, Rule Bank, Safety Memory, and Tool Policy. Diagnosed failures are routed to the smallest responsible artifact, revised, and evaluated against both safety and task-completion signals.

This repository contains the code and released resources for the paper SHE: Trajectory-driven Safety Harness Evolution for LLM Agents.

Overview

For each evolution round, SHE:

1. runs agents under clean and adversarial conditions;
2. diagnoses trajectory-level failures using normalized risk dimensions;
3. attributes each failure to the responsible harness artifact;
4. proposes bounded edits and retains the best validated harness.

The released setup uses a fixed 15-task Agent-SafetyBench evolution split and evaluates on the remaining 185 task IDs under six conditions: clean, context poisoning, indirect injection, tool tampering, memory injection, and composite attack.

Repository layout

artifacts/                   Seed safety-harness artifacts
configs/                     Default experiment configuration
data/agent_safety_bench/     Fixed evolution and held-out task splits
evaluate/                    Rollout judge and evaluation entry points
scripts/                     Evolution, rollout, analysis, and evaluation scripts
she/                         SHE runtime and evolution library
tests/                       Smoke tests

Generated artifacts, rollout outputs, credentials, and external benchmark packages are intentionally excluded.

Installation

Python 3.10 or later is required.

git clone https://github.com/RainbowQTT/SHE.git
cd SHE

python -m venv .venv
. .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -e '.[llm]'

Run the included smoke tests:

python -m unittest discover -s tests

Configuration

SHE reads model endpoints and keys from environment variables. Configure the endpoints appropriate to your deployment before running experiments:

export SHE_GATEWAY_BASE_URL="https://your-gateway.example/v1"
export SHE_GATEWAY_API_KEY="..."
export SHE_REFLECT_BASE_URL="https://your-reflection-endpoint.example/v1"
export SHE_REFLECT_API_KEY="..."

Do not commit API keys or generated run outputs. The default experiment settings are in configs/default.yaml.

Evaluation

Evaluate seed or evolved harness artifacts on the held-out split:

python scripts/run_safety_bench_evaluation.py \
  --tasks data/agent_safety_bench/heldout185_same_ids.json \
  --benchmark-root "$BENCHMARK_ROOT" \
  --artifacts-dir artifacts \
  --use-judge

For an evolved harness, replace artifacts with its generated best_harness/ directory.

Responsible use

This code is intended for research on the safety of tool-using agents. Run it only in controlled environments and review generated policies before deployment. Do not use the repository to enable unauthorized access, harmful tool execution, or collection of sensitive data.

License

This project is released under the terms in LICENSE.

Citation

Citation information will be added with the paper release.