Run of the Week #2: deterministic Incident Response World

[tonyhyworld]

A rejected action is part of the result

TraceArena's public Incident Response World is a small, deterministic, non-financial benchmark for long-horizon agent evaluation.

Two roles share the same synthetic evidence and move through:

acknowledge → mitigate → health check → resolve

The reference fixture deliberately attempts to declare the incident resolved before a passing health check. The world rejects that action with:

passing_health_check_required

The replay then records the accepted health check and settles the final state as:

• incident_status: resolved
• service_health: 100
• evidence_quality: 100
• response_progress: 100

Run it locally:

python3 examples/incident_response_world/scripts/replay.py \
  examples/incident_response_world/tests/fixture.json
python3 -m unittest discover -s examples/incident_response_world/tests -p 'test_*.py'

Links:

• Incident Response World
• Benchmark Card
• Scenario proposal form

What dependency, evidence rule, or failure category should a long-horizon agent benchmark make explicit next?