v1.0.0

First public release of unsafe-eval, a framework for reproducible prompt-safety evaluation built on Inspect AI. It scores safety classifiers and general-purpose models on the same benchmarks with the same rules, so their results can be compared directly.

🚀 What's New

• Add adapters for 8 prompt-safety benchmarks, each loaded at a pinned dataset revision: SimpleSafetyTests, HarmBench Text Test, Aegis 2.0, OpenAI Moderation, ToxicChat 0124, WildGuardMix, CoSApien and DynaGuardrail.
• Add 15 versioned protocols, one per model family, that fix the prompt format, generation settings, allowed options, scorer and metrics: shieldstral-1.0-3b, scope-guard-2601, scope-guard-2609, nemotron-3.5-content-safety, polyguard-qwen, qwen3-guard-gen, gpt, claude, gemini, jev, d1, clef, laya, gliner2.5-decide and gliguard.
• Ship 502 packaged run configurations covering 64 models, which pin the dataset revision and, where the provider allows it, the model revision (an exact hub commit for local models, a dated snapshot for hosted ones).
• Add Inspect model providers for models that no stock provider reaches: TypeSafe's Jev, Kev, Laya, Clef, d1, CLM, GLiNER2 and Orbitals' ScopeGuard. Each provider and runtime is an optional extra (vllm, hf, openai, openrouter, anthropic, google, typesafe, orbitals, laya, clef, clm, gliner).
• Add the unsafe-eval CLI: run executes packaged or local configurations with --resume, list shows benchmarks, protocols and configurations, config path locates a packaged configuration, and report summarizes the metrics files of a log directory.
• Record the setup of every run (unsafe-eval and inspect-ai versions, uv.lock hash, git commit and dirty state) in its metrics file, and ask before mixing runs from different setups in one log directory or reusing them with --resume.
• Write an HTML report (report.html) to the log directory after every run, or on demand with report --format html: a single file that needs no network, styled like the public leaderboard, with per-benchmark metrics and median latency.

🔧 Internal

• Derive the package version from git tags with hatch-vcs.
• Publish to PyPI through trusted publishing when a GitHub release is published (.github/workflows/publish.yaml).
• Document every benchmark, protocol, metric and provider under docs/, with guides for adding a benchmark, a model and a metric.