ClaimPilot now evaluates trajectories, not only final answers
v0.3.0 adds a deterministic tool-use Trace Arena for insurance claim AI agents.
Highlights
- Replay every policy lookup, evidence retrieval, document request, escalation, and decision.
- Score required tool coverage, evidence coverage, action ordering, approval gates, forbidden actions, verdict, and step budget.
- Block dangerous payment and denial tools unless an explicit human approval token is present.
- Compare an evidence-first cautious trajectory (100/100) with a shortcut trajectory (5/100).
- Execute the bodily injury domain workflow as a testable case contract.
- Explore the Trace Arena in the interactive GitHub Pages product demo.
Verified release surface
- 10 adversarial insurance cases
- 18 risk tags
- 31 automated tests
- Deterministic JSON and standalone HTML trace artifacts
- No simulated external-model results presented as measured model performance