docs(weave): add "Evaluate your AI agent with Weave" tutorial

[anastasiaguspan]

Summary

New Get Started tutorial, Evaluate your AI agent with Weave (weave/agent-evals.mdx), with a companion Colab (weave/cookbooks/source/agent_evals.ipynb) and a nav entry under the Weave Get Started group.

It teaches agent evaluation with the Agents/Conversations workflow (OTel spans, not Ops/Calls): trace an agent as turns and tool calls, score single-turn and multi-turn task completion with an LLM judge via EvaluationLogger, and compare agent versions.

What's included

• weave/agent-evals.mdx — the page. Inline code is intentionally reduced to the Weave surface; the full runnable source lives in the Colab.
• weave/cookbooks/source/agent_evals.ipynb — companion notebook (agent on Claude Sonnet, judge on Claude Opus).
• docs.json — weave/agent-evals added to the Get Started group (English only; localized nav untouched).

Verification

The code patterns were run end-to-end against the real Weave and Anthropic SDKs (agent trace, single-turn eval, multi-turn eval) publishing to a live Weave project. The notebook was validated (JSON parses; every code cell parses).

Recommendations for technical review

Identified during authoring and style editing; please confirm before merge.

Technical accuracy

• The implicitly_patch_integrations: False rationale (avoids duplicate legacy Op/Call traces alongside the manual Conversation SDK spans) matches current behavior.
• The eval-to-agent-trace deep link is a UI/query-time feature, not an SDK return value — confirm the "link to the transcript" claim renders as described.
• Model IDs claude-sonnet-5 (agent) and claude-opus-4-8 (judge) are correct for the docs' target audience.

Missing content / confirm

• The reduced snippets use placeholders (convo_id, new_id(), run_agent_turn, run_tool, response_tool_calls, task_completion, judge_task) defined in a bulleted list and deferred to the Colab — confirm this is enough for a self-contained read.
• The Colab/GitHub links assume the notebook lands at weave/cookbooks/source/agent_evals.ipynb on main. If the cookbook build renames or generates notebooks, adjust the URLs.
• Cross-link /weave/guides/evaluation/evaluation_logger exists on main; the in-flight branch codex/link-agent-evals-docs edits that same guide (linked agent eval traces) — coordinate if both land.

Related

• Complements weave/tutorial-rag.
• Jira: DOCS-2885.

🤖 Generated with Claude Code

[anastasiaguspan]

Resume prompt (for a fresh Claude Code session)

Continue work on wandb/docs PR #2945 (branch aguspan/docs-agent-evals): the new Weave tutorial weave/agent-evals.mdx + companion Colab weave/cookbooks/source/agent_evals.ipynb, nav entry in docs.json under the Weave "Get Started" group. Jira DOCS-2885.

Context: teaches agent evaluation with the new Agents/Conversations workflow (OTel spans, not weave.Op/Calls) — trace an agent, score single- and multi-turn task completion with an LLM judge via EvaluationLogger, compare versions. Created in addition to weave/tutorial-rag (not a replacement). Agent = claude-sonnet-5, judge = claude-opus-4-8. Inline code is intentionally reduced to the Weave surface; full source is in the Colab. Code was verified end-to-end against the real Weave + Anthropic SDKs.

Verified working source and decision notes live outside the repo at ~/agent-eval-tutorial/ (HANDOFF.md, agent.py, judge.py, 01_trace.py, 02_eval_task_completion.py, 04_multi_turn.py). Persistent project memory: .claude/projects/-Users-aguspan-Weave-weave/memory/ (agent-eval-tutorial.md, agent-eval-mechanism.md, task-completion-meaning.md, harbor-analogy.md).

Open items: the "Recommendations for technical review" checklist in the PR body (implicit-patching rationale, eval→trace link being a UI feature, placeholder sufficiency, Colab path, coordination with branch codex/link-agent-evals-docs). Docs style already applied via the docs:docs skill passes.

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated (UTC)
wandb	🟢 Ready	View Preview	Jul 21, 2026, 3:06 PM

[github-actions]

📚 Mintlify Preview Links

🔗 View Full Preview

✨ Added (1 total)

📄 Pages (1)

File	Preview
weave/agent-evals.mdx	Agent Evals

📝 Changed (1 total)

⚙️ Other (1)

File
docs.json

---

🤖 Generated automatically when Mintlify deployment succeeds
📍 Deployment: 75157be at 2026-07-22 19:34:17 UTC

[github-actions]

🔗 Link Checker Results

⚠️ Some issues were detected

Preview: https://wb-21fd5541-aguspan-docs-agent-evals.mintlify.site

Full Github Actions output

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated (UTC)
wandb	🟡 Building	–	Jul 21, 2026, 3:01 PM

[w-b-hivemind]

HiveMind Sessions

1 session · 1h 19m · $34

Session	Agent	Duration	Tokens	Cost	Lines
Refining Agent Evals Documentation Page 8980c233-4799-4ecc-a696-ef116d75a721	claude	1h 19m	234.4K	$34	+417 -46
Total		1h 19m	234.4K	$34	+417 -46

View all sessions in HiveMind →

Run claude --resume 8980c233-4799-4ecc-a696-ef116d75a721 to pickup where you left off.