Home

safelabs-eval Wiki

Open-source evaluation framework for red-teaming AI agents (LangChain, CrewAI, AutoGen, and others) against the OWASP Agentic Security Initiative Top 10.

pip install safelabs-eval
safelabs run --target <your-agent-url> --category all

Reference

• OWASP ASI Coverage — what each ASI01–ASI10 category tests, and how to run it
• Detectors — how verdicts (PASS/FAIL/UNCERTAIN/VULNERABLE) are scored, and known detector limitations
• Model vs. Framework Findings — does orchestration framework choice affect agent security? (Short answer: no.)

Learn more

• Blog — evaluation writeups, research findings, incident postmortems
• Research — peer-reviewed and preprint work built on this framework
• Discussions — questions, ideas, community
• Report an issue

---

📦 PyPI · 📄 Apache 2.0 License