v0.13.2-beta — live-agent compliance judging + MAF eval-feature integration
Highlights
Two community/feature landings on top of the judge-primary grading line — no grader-behaviour changes (judge-primary Composite Judges shipped in 0.13.0-beta and are unchanged).
Compliance: live-agent judging + a silent judge-parse fix (#57, @Javierif) 🙌
- Live-agent compliance judging (
AgentScenarioEval) — GDPR / EU AI Act benchmarks drive the real agent-under-test with each scenario's own prompt and grade its real answer; an agent failure surfaces as a distinct "error" leaf, never a confirmed violation. - Silent compliance-judge parse fix — the verdict parser didn't bridge
snake_case↔camelCase, so every GDPR / EU AI Act verdict was being parsed to score0. Fixed. - Plus the
EvaluationFailedhonesty primitive, richer compliance findings, and red-team scan truncation-salvage (ScanOptions.OverallTimeout).
Microsoft Agent Framework evaluation-feature integration (#66)
AgentEval.MAF plugs AgentEval into MAF's built-in agent.EvaluateAsync(...) — score a MAF agent with AgentEval metrics or a whole benchmark composite, in one call, rendered as a self-contained HTML report.
AgentEvalAgentEvaluator— nativeIAgentEvaluatorthat forwards the fullEvalItem.Conversation(tool-call turns included), so code-based tool metrics see the real calls.AgentEvalCompositeEvaluator— runs an AgentEval composite as one MEAIIEvaluator; captures the rich tree for rendering, flattens for MAF's roll-up.MeaiToEvalResultBridge+AgentEvaluatorExtensions+ theMafEvalLightPathsample + the using-agenteval-with-maf-evals guide.
NuGet: AgentEval + AgentEval.Cli 0.13.2-beta. Full details: CHANGELOG.