[Spec Change] Should llm_judge prompt-injection mitigations be a MUST, not a SHOULD?

[adhabnr-ux]

Flagged in spec/CRITIQUE.md #3 and explicitly marked "Partially fixed" — this is the one open security item from the critique that hasn't moved since 1.0.

The problem: a malicious test case's input can inject instructions into an llm_judge grader's prompt, causing the judge to always return score: 1.0 regardless of the actual output — silently making a broken system appear to pass. The spec currently says runners:

• SHOULD use structured output (JSON schema) to constrain judge responses
• SHOULD delimit the output being evaluated from instructions
• SHOULD cap judge LLM output length

All three are recommendations, not requirements, and there's no spec-level way to detect that a runner skipped them.

Why this is genuinely a judgment call, not just "make it stricter": the spec's role is data format, not runner implementation — the CRITIQUE's own original resolution reasoned "the spec cannot enforce prompt injection resistance at the data format level." Structured-output constraint (SHOULD #1) is something a reference implementation controls; delimiting (SHOULD #2) depends on how each runner assembles its prompt template, which the spec doesn't currently standardize.

Concrete options for this thread:

1. Leave as-is (three SHOULDs), but add a spec-level test fixture — an llm_judge test case with an obvious injection attempt in input — to the conformance suite (see the linked discussion on that) so at least detection of the vulnerability is standardized even if mitigation stays a SHOULD.
2. Promote SHOULD Adapter wanted: Ragas (to_openeval() / from_openeval()) #1 (structured output / JSON schema-constrained judge response) to a MUST, since evalport-sdk's own llm_judge implementation already does this by default — codifying existing reference-implementation behavior rather than inventing new requirements.
3. Add a new reserved metadata key, e.g. openeval.judge_hardening: "structured_output+delimited", so a Result can self-report which mitigations its runner actually applied — turns an invisible property into a queryable, auditable one without forcing every runner into the same implementation.

Security-minded folks, especially anyone who's dealt with prompt injection in production judge/grading pipelines, this is the thread to weigh in on.

[adhabnr-ux]

Leaning toward option 3 (openeval.judge_hardening self-report metadata) after building 22 adapters, for a reason that's less about security policy and more about a pattern that's already proven itself repeatedly in practice: adapters keep needing an honest "here's what we actually know, and here's what we don't" metadata channel for exactly this class of opaque-judge problem, and this would just be the security-specific instance of a channel that already exists and works.

Concrete precedent from the adapters themselves:

• giskard-openeval-adapter can't extract LLMJudge's actual prompt template (Giskard builds it internally), so it documents the gap honestly in params.prompt rather than fabricating one.
• llamaindex-openeval-adapter has the identical problem for FaithfulnessEvaluator et al. — most llama_index judges don't expose one literal prompt string at all (FaithfulnessEvaluator queries a query engine built over context, no template fill), so to_openeval() synthesizes an honest human-readable description instead, and fills params.model with a documented placeholder ("llamaindex-configured-llm") since the evaluator resolves its LLM from Settings.llm at call time and never stores it on itself.
• arthur-bench-openeval-adapter hits the mirror case on the results side: several categorical scorers need a live Arthur-hosted API or LLM judge this adapter has no credentials to run, so those get exported as custom with the real scorer name/config captured (a faithful record of what ran) rather than something runnable.

Every one of these is "the adapter can prove X truthfully and can't prove Y, so it says so explicitly instead of inferring or omitting" — which is exactly the shape openeval.judge_hardening: "structured_output+delimited" would need: a runner honestly self-reporting which mitigations it actually applied, not the spec asserting they were applied. Given that pattern already has three independent, real implementations solving the same underlying honesty problem for a different field, adding it for judge-hardening specifically feels like the lowest-friction of the three options — it doesn't require standardizing prompt-assembly (which SHOULD #2 would need), and it doesn't need reference-implementation behavior promoted to a MUST before every alternate implementation is confirmed to match it (which SHOULD #1 would need).

I'd pair it with option 1 regardless (an injection-attempt fixture in the conformance suite — see #9) since self-reporting hardening and testing it are different problems; a runner could claim judge_hardening and still be wrong. But as a spec change on its own, 3 seems like the one that's already been informally validated by how every adapter handling opaque LLM internals has independently converged on the same "self-report honestly" shape.

[adhabnr-ux]

Shipped in 1.0.0-rc.3, and it's option 3 — the one you were leaning toward above: metadata.openeval.judge_hardening, a free-text self-report convention (+-joined mitigation names, e.g. "structured_output+delimited+length_capped") that a runner sets on a GraderResult to declare which of the three SHOULDs it actually applied to that llm_judge call. No schema change — GraderResult.metadata was already additionalProperties: true — documented in SPEC.md's Extension Mechanism → Judge Hardening Self-Report, and in spec/CRITIQUE.md §3's update.

Fixture: judge_hardening_self_report.json.

Being precise about what did and didn't move here, since this is the security thread: the three underlying SHOULDs (structured output, delimiting, output-length caps) are still SHOULDs. This does not promote SHOULD #1 to a MUST (option 2), and it does not standardize prompt assembly — the CRITIQUE.md update calls the MUST-vs-SHOULD question itself "won't-fix," for the same reason your original post gave: a spec-level MUST isn't tractable without also standardizing how a runner assembles its prompt template. What this does ship is turning "did this runner apply any hardening" into a queryable, honest, per-result fact instead of an invisible property — matching your own framing that self-reporting and testing are different problems, and this only ships the former.

Your comment also floated pairing this with option 1 — an injection-attempt fixture in the conformance suite (#9). That part is not done. judge_hardening_self_report.json validates that the self-report metadata shape is well-formed and accepted; it does not attempt an actual prompt injection or check whether a runner's judge_hardening claim is true. Flagging that explicitly rather than letting the fixture's existence read as covering more than it does — a real injection-attempt fixture (adversarial input designed to hijack an llm_judge prompt, checked against whatever hardening a runner claimed) is genuinely open work, and would live in spec/conformance/ alongside this one if someone wants to build it.

Marking the self-report-convention question resolved; leaving this thread open specifically for the injection-attempt-fixture idea, since that's real security-test coverage this rc didn't add.