fix: build judge input as string; strip legacy judge config messages

[jsonbailey]

Summary

• String input instead of message list: Judge.evaluate() now calls runner.run(input_str) with a plain string ("MESSAGE HISTORY:\n...\n\nRESPONSE TO EVALUATE:\n...") instead of constructing and passing a List[LDMessage].
• Backwards-compatible legacy message stripping: A new _strip_legacy_judge_messages() helper transparently removes the old assistant/user template messages ({{message_history}} / {{response_to_evaluate}}) that appeared in older judge configs. System messages are never stripped.
• Runner.run narrowed to str: The Runner protocol and all four runner implementations (OpenAIModelRunner, LangChainModelRunner, OpenAIAgentRunner, LangChainAgentRunner) now declare input: str. The _coerce_input list-handling path in the model runners has been removed.
• messages guard removed: The early-exit guard that returned an error when messages was None or empty has been removed — configs without messages (new-style) are now handled gracefully.
• chevron / _interpolate_message removed from judge: The Mustache interpolation code is gone from judge/__init__.py; chevron remains a dependency because it is still used in client.py.

Why

Passing a message list through the runner added unnecessary complexity and required all runners to handle both str and List[LDMessage] inputs. Moving to a single string interface simplifies the runner contract, aligns with the direction of new judge configs (system-message-only), and removes the Mustache templating requirement from the judge path.

Test plan

• TestStripLegacyJudgeMessages — unit tests for the helper covering: strip assistant/user template messages, preserve system messages even with template vars, pass-through non-template messages, empty list, new-style system-only config
• TestJudgeEvaluate::test_evaluate_passes_string_input_to_runner — verifies runner.run() receives a str, not a list
• TestJudgeEvaluate::test_evaluate_string_input_format — verifies exact "MESSAGE HISTORY:\n...\n\nRESPONSE TO EVALUATE:\n..." format
• TestJudgeEvaluate::test_evaluate_legacy_config_strips_template_messages — verifies legacy configs still produce string input
• TestJudgeEvaluate::test_evaluate_succeeds_when_messages_is_none — verifies no early-exit error for None messages

🤖 Generated with Claude Code

---

Note

Medium Risk
Medium risk because it narrows the Runner.run contract to str and changes how prompts/config messages are assembled, which can break callers that passed List[LDMessage] and alter judge/model prompting behavior.

Overview
Standardizes runner inputs to plain strings. The Runner protocol and OpenAI/LangChain model + agent runners now declare run(input: str, ...), removing support for passing List[LDMessage] and deleting the _coerce_input paths.

Moves config prompts into runner construction. OpenAIRunnerFactory/LangChainRunnerFactory now pass config.messages into their model runners, which prepend these config messages internally; ManagedModel.run() correspondingly passes only the user prompt string.

Simplifies judge prompting and adds legacy-config cleanup. Judge.evaluate() now builds a single formatted evaluation string (MESSAGE HISTORY / RESPONSE TO EVALUATE) instead of interpolating Mustache templates into message lists, and the client strips legacy non-system template messages via new _strip_legacy_judge_messages; tests were updated/added to cover the new behavior.

^{Reviewed by Cursor Bugbot for commit 133bc10. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 133bc10. Configure here.}