feat: Add evaluations support to ManagedAgent.run()

[jsonbailey]

Summary

• Wires judge evaluations into ManagedAgent.run() via asyncio.Task, mirroring ManagedModel.run() (PR 7 / PR 8)
• run() returns immediately; await result.evaluations guarantees both evaluation and tracker.track_judge_result() complete
• Uses ai_config.evaluator.evaluate(input, content) — resolves to empty list with Evaluator.noop()
• Failed judge results (success=False) do NOT call track_judge_result()
• Adds 6 new tests covering the full evaluations contract

Depends on

• feat: Wire LDAIMetrics tool_calls and duration_ms into tracker #152 (PR 10 — enrich-metrics, which is based on feat: Add ManagedGraphResult, GraphMetricSummary, and AgentGraphRunnerResult types #151, feat: Update LangChain runners to implement Runner protocol returning RunnerResult #150, feat: Update OpenAI runners to implement Runner protocol returning RunnerResult #149, feat!: Add ManagedResult, RunnerResult, and Runner protocol; rename invoke() to run() #148, fix: Replace done_callback with coroutine chain for judge tracking #147)

Test plan

• All existing tests pass (uv run pytest packages/sdk/server-ai/tests/)
• New TestManagedAgentEvaluations tests: run returns before evaluations resolve, collect results, tracking fires on await, noop evaluator returns empty list, failed results not tracked

🤖 Generated with Claude Code

---

Note

Medium Risk
Adds background asyncio evaluation tasks and tracking side effects to ManagedAgent.run(), which can introduce task-lifecycle/awaiting and error-handling edge cases. Core behavior remains the same for the main agent invocation, but results now include an optional evaluations task and new warning logs on evaluation failures.

Overview
ManagedAgent.run() now kicks off judge evaluations via ai_config.evaluator.evaluate(input, output) in a background asyncio.Task and returns a ManagedResult that includes the optional evaluations handle.

When result.evaluations is awaited, successful JudgeResults are recorded via tracker.track_judge_result(); failed results (or tracking exceptions) are logged as warnings without failing the main run. Tests are expanded with a helper config and a new TestManagedAgentEvaluations suite covering task-asynchrony, result collection, and tracking behavior for noop/failed evaluations.

^{Reviewed by Cursor Bugbot for commit 91877d5. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 9f9c880. Configure here.}

[cursor]

Duplicated _track_judge_results logic across managed classes

Low Severity

The _track_judge_results method in ManagedAgent is a character-for-character duplicate of the same method in ManagedModel. Both take tracker, input_text, output_text, call evaluator.evaluate(), wrap it in an async task that iterates results, tracks successful ones, and logs failures. This duplicated logic increases maintenance burden — a bug fix or behavior change in one would need to be manually replicated in the other.

 

^{Reviewed by Cursor Bugbot for commit 9f9c880. Configure here.}

[jsonbailey]

We will consider a refactor in the future if needed. Its light enough that we will leave it as is for the moment.