Skip to content

Add a docs-lookup subagent for wide documentation lookups - #3

Draft
abernier wants to merge 1 commit into
claude/plugin-role-onxyywfrom
claude/sous-agent-docs-onxyyw
Draft

Add a docs-lookup subagent for wide documentation lookups#3
abernier wants to merge 1 commit into
claude/plugin-role-onxyywfrom
claude/sous-agent-docs-onxyyw

Conversation

@abernier

@abernier abernier commented Aug 10, 2026

Copy link
Copy Markdown
Member

Stacked on #4 — review that one first; this diff is only the agent.

Adds agents/docs-lookup.md and extends #4's test harness to cover agents.

Why

A question that spans a drei helper, the R3F hook under it and the zustand store behind that pulls three index listings and several page bodies into whatever the session was actually working on. drei's index alone is 134 lines. The subagent reads those in its own context and reports back.

It is deliberately not the default path. One index plus one page is smaller than the round trip through another agent, and a summary of a signature is precisely the failure mode this component exists to prevent — the caller would then write code from a paraphrase, which is memory at one remove. So:

  • the agent quotes signatures, prop names and option keys verbatim, links every page, and says plainly when the docs don't cover something rather than filling the gap from memory;
  • SKILL.md keeps the lookup inline by default and hands off only when the reading is wide — 3+ pages, or 2+ libraries.

Read-only (ListMcpResourcesTool, ReadMcpResourceTool, get_page_content), Haiku, capped at 12 turns.

Single source of truth

The agent carries no copy of the coverage table. It preloads the docs skill via the skills: frontmatter field, so SKILL.md stays the one place recording which libraries the server actually serves and when that was last checked. A test enforces the table doesn't get copied.

Tests

Extends the harness to agents, same principle — catch what fails silently at runtime rather than loudly at load:

Invariant Why it matters
tools entries use the scoped mcp__plugin_{plugin}_{server}__{tool} form derived from the manifests, not hardcoded, so a one-sided plugin rename is caught
built-in tool names are spelled correctly a list resolving to nothing stops the agent launching at all
preloaded skills exist a missing one is skipped with only a debug-log warning
hooks / mcpServers / permissionMode absent plugin agents ignore them, so they read as config that does something
the coverage table stays in one file a second copy is a copy that goes stale unnoticed
agent and skill names don't collide @pmndrs:docs-lookup shouldn't read as a twin of /pmndrs:docs

Mutation-checked — breaking an invariant fails the test that claims to guard it, and no other:

bare MCP server key in tools            → every tool name resolves
plugin renamed in one manifest only     → every tool name resolves | marketplace entry agrees…
typo in a built-in tool name            → every tool name resolves
preloaded skill that does not exist     → preloaded skills exist
permissionMode on a plugin agent        → only uses front matter that plugin agents honour
coverage table copied into the agent    → does not restate what the skill owns
SKILL.md stops routing to the agent     → tells Claude when to delegate, and to an agent that exists

23/23 green locally, claude plugin validate . included.

Worth a look during review

  • Staleness. The coverage date test checks the format, not the age. Failing CI on a date drifting past a year would force a re-check, but it also breaks builds nobody touched — happy to add it if you'd rather have the nag.
  • The delegation threshold (3+ pages / 2+ libraries) is a judgement call written in prose, so it's the one thing here no test can pin down.
  • Whether the agent is worth it at all. The honest case against: if it turns out Claude delegates for single-page questions anyway, this costs a Haiku round trip on every lookup and returns a summary where the skill returned a quote. That's measurable rather than arguable — see below.

Not in this stack

An eval — N questions with known documented answers, replayed through claude -p with and without each component, counting lookups triggered and signatures quoted correctly. It's the only thing that would settle both the delegation threshold and whether the skill changes behaviour at all. It needs model calls, so it'd be npm run eval by hand rather than CI.

A question that spans a drei helper, the R3F hook under it and the zustand
store behind that pulls three index listings and several page bodies into
whatever the session was actually working on. `docs-lookup` reads those in
its own context and reports back.

It is deliberately not the default path. One index plus one page is smaller
than the round trip through another agent, and a summary of a signature is
the failure mode this component exists to prevent — so the agent is told to
quote verbatim and link, and SKILL.md keeps the lookup inline unless the
reading is wide (3+ pages, or 2+ libraries).

The agent carries no copy of the coverage table: it preloads the `docs`
skill, which stays the single place recording which libraries the server
serves and when that was last checked.

Extends the test harness to agents, covering what fails silently rather than
loudly: `tools` entries must use the scoped
`mcp__plugin_<plugin>_<server>__<tool>` form derived from the manifests, and
must name real built-in tools — a list resolving to nothing stops the agent
launching. Preloaded skills must exist, since a missing one is skipped with
only a debug-log warning. `hooks`, `mcpServers` and `permissionMode` must be
absent, since plugin agents ignore them. And the coverage table must stay in
one file.

Each invariant was mutation-checked: breaking it fails the test that claims
to guard it, and no other.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TnvYiz7aGhnBTK2tzExHis
@abernier
abernier force-pushed the claude/sous-agent-docs-onxyyw branch from 4e5ab69 to 4935a15 Compare August 10, 2026 06:04
@abernier
abernier changed the base branch from main to claude/plugin-role-onxyyw August 10, 2026 06:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants