v0.2.1

• Case study: a real garak 0.17 run against Llama 3.2 3B, assessed with --no-excerpts, plus a human review of the jailbreak detector's calls.
• garak 0.13+ reports: the target is read from plugins.target_type / plugins.target_name (it showed as "unknown").
• garak 0.17 probe modules are mapped: web_injection.* → RT-07 (the old xss.* name still maps), sysprompt_extraction.* → RT-04, propile.* → RT-05, and fitd.*, dra.*, smuggling.* → RT-03.

Install: pip install llm-redteam-evidence==0.2.1

Artifacts carry SLSA build provenance; PyPI uploads use trusted publishing with attestations.

v0.2.0: one-command demo and readable sample output

• redteam-evidence demo assesses the bundled examples and compares v1 → v2 right after pip install; no clone needed.
• Committed sample-output/ so the reports can be read without running anything; CI checks it stays current.
• Reproducible output: set SOURCE_DATE_EPOCH for byte-identical evidence, summaries and OSCAL (deterministic OSCAL UUIDs, LF line endings on every OS).
• Sources inside a scanned folder are recorded relative to it, so evidence hashes don't depend on the install path.
• Dependencies pinned by hash; OpenSSF Scorecard workflow; contributing guide and issue forms.

Install

pip install "git+https://github.com/Mihirajmera/llm-redteam-evidence@v0.2.0"
redteam-evidence demo

The bundled examples are synthetic.

v0.1.0: first release

First release of llm-redteam-evidence: turns LLM red-team results into audit evidence and a release gate.

What it does

• Reads garak report.jsonl and promptfoo results JSON
• Maps every attack to 10 controls, each with a risk tolerance (maximum attack success rate) and a minimum number of attacks
• Reports attack success rate with a 95% confidence upper bound, so small clean samples aren't mistaken for proof
• Writes a reviewer report with a release recommendation (APPROVE / REVIEW / BLOCK) and sign-off, SHA-256-hashed evidence tied to the raw tool output, and OSCAL 1.1.2 assessment results
• compare flags regressions between model versions, for use as a CI release gate
• Maps to the OWASP Top 10 for LLM Applications 2025, MITRE ATLAS, NIST AI 600-1, NIST AI RMF and NIST 800-53

Install

pip install "git+https://github.com/Mihirajmera/llm-redteam-evidence@v0.1.0"
redteam-evidence assess path/to/results --system "My Assistant" --version v2

The bundled examples are synthetic. This is an ongoing project; issues and catalog suggestions are welcome.