Auly — AgentRisk

Adversarially test your AI agent and get a content-free, framework-mapped risk rating — a letter grade, per-attack-class findings, and a fix → score remediation path. Auly runs its own jailbreak and prompt-injection probes against your agent and keeps only pass/block rates and case-ID pointers — your agent's responses never leave your machine.

Two ways to run it, same engine, same scorecard:

1. CLI — auly-scan (any agent behind an HTTP endpoint)

Describe your agent in a target.yaml (see demo/target.refund-bot.yaml), then:

npx auly-scan demo/target.refund-bot.yaml --out .
# → agentrisk-scorecard.json + a console scorecard

2. Python SDK — auly (an agent you hold as a callable in code)

No endpoint, no target.yaml, no auth wiring:

pip install auly

from auly import rate

def my_agent(prompt: str) -> str:
    return my_stack.run(prompt)            # any function that answers a prompt

scorecard = rate(
    agent=my_agent,
    name="my-support-agent",
    scope="What it may and must-never do (grades injected-instruction attacks).",
    depth="balanced",                      # quick | balanced | thorough
    grader="google:gemini-2.5-flash",
    out=".",
)
print(scorecard["composite"]["grade"])

A TypeScript in-process adapter lives in src/agentrisk/redteam/inProcess.ts (rate({ agent, ... })).

What you get

• Content-free by construction — only Auly's own probes are sent; your agent's responses are held transiently to grade pass/block, then discarded. Only rates + pointers persist.
• Self-attested — a self-run rating is honestly labeled self-attested (not Auly-verified).
• Framework-mapped — block-rate atoms crosswalk to OWASP LLM Top 10, MITRE ATLAS, NIST AI RMF, ISO, and the EU AI Act.

Requirements

• Node ≥ 20 (the CLI + the packaged battery).
• A grader LLM key in the environment (e.g. GEMINI_API_KEY) — grading runs on your machine.
• The Python SDK additionally needs Python ≥ 3.10 (stdlib only, no pip deps).

License

Apache-2.0. "Auly" and "AgentRisk" are trademarks reserved by the project (Apache-2.0 §6 grants no trademark rights) — see NOTICE. More at tryauly.com.