Red-teaming: adversarial test generation & attack replay

[abhishekmanyam]

Summary

Bring adversarial testing / red-teaming into EvalCore the same way we do everything else: as config-first suites you can record once and replay in CI for $0. Think of the Promptfoo red-team surface, but built on our record/replay cassette so an attack suite runs offline and deterministically once recorded.

Why this fits EvalCore

Red-teaming is just evals with adversarial inputs and safety-oriented scorers. We already have the engine (targets, scorers, gates, cassettes, HTML reports). The missing piece is generating the adversarial cases and grading refusals/policy adherence.

Proposed shape (YAML-first, as always)

1. Attack generators — a plugin (subprocess protocol, like custom scorers) that emits adversarial cases as JSONL: jailbreaks, prompt injection, PII exfiltration, harmful-content elicitation, indirect injection via retrieved context.
2. Record the responses — attacks hit the target once, live, and land in the cassette. CI then replays them byte-for-byte, offline, no keys.
3. Safety scorers — refusal detection / policy-adherence grading (a judge rubric plus deterministic guards), surfaced through the same gates and reports.

Open questions

• Generator as a bundled binary vs. pure subprocess protocol (leaning protocol — no one should have to write Rust).
• Which attack families ship in v1.
• How much overlaps with existing judge + subprocess scorers vs. needs new scorer types.

This is a tracking epic — comment if you want to own an attack family or the generator protocol. Related topics: red-teaming, pentesting, vulnerability-scanners.