feat(mcp): make capability discovery an intent-aware primary agent workflow

[rickylabs]

Summary

The docs MCP currently exposes list_docs, search_docs, and get_doc, but retrieval is a flat lexical pass:

• tokenize the query into literal terms;
• count occurrences in title ×12, headings ×5, body ×1;
• sort by score;
• return one body-adjacent snippet.

There is no concept/synonym expansion, field-length normalization, section-level ranking, link-graph traversal, task sequence, or code-block extraction. The MCP instructions only route agents to docs search for troubleshooting symptoms; they do not make it the default “understand this unfamiliar framework before implementing” workflow.

Wave-four baseline was 0 docs-MCP calls across three runs. Observational proof after a change belongs in #1090; this issue is the implementable product surface required before that measurement is meaningful.

Required product shape

Introduce an intent-aware guidance flow (name is open; find_guidance is illustrative) that answers a task, not just a bag of terms. A response should be bounded and structured:

• ordered recommended pages and sections;
• why each result matches the intent;
• prerequisite → implementation → verification sequence;
• relevant code blocks with language and source slug/section;
• related capability/reference links;
• confidence/fallback when the corpus cannot support the answer.

The task router landed in #1068/#1079 should become input to this flow, not remain a separate text artifact agents must happen to read.

Retrieval requirements

Implementation may use BM25 + curated concept aliases, a small local embedding model, or another deterministic hybrid. The contract matters more than the algorithm:

• natural-language concept mismatch must work (e.g. “avoid hitting my service every render” → cache-first queries/staleTime);
• results should rank sections, not only whole documents;
• document links should supply next/prerequisite traversal;
• code fences/Vento code examples should be independently retrievable;
• filesystem and embedded corpora must behave the same;
• the result remains token-bounded and offline-capable.

Evaluation corpus

Add a checked-in retrieval evaluation with at least these intents and expected top-three destinations:

Intent	Expected destination
validated route-bound form	web-layer/form or the server-validated-form how-to
keep server data fresh without polling	web-layer/query / live-dashboard cache-first chapters
add a capability NetScript does not ship	custom plugin authoring guide
use a Prisma-supported database NetScript does not wrap	second-database unsupported-driver section
build a real service-backed UI	the #1068 task-router sequence

Measure top-k recall and deterministic result bounds; do not use a future agent run as an acceptance checkbox.

Activation

Update the MCP server instructions and generated agent guidance so an agent asks the guidance flow before inventing an unfamiliar NetScript API or architecture. Keep search_docs for literal lookup and get_doc for exact retrieval.

Acceptance

• An intent-aware guidance tool/flow returns ordered section-level guidance and cited code excerpts.
• A checked-in evaluation corpus defines expected top-k results for the intents above and passes deterministically.
• Retrieval handles concept mismatch without requiring the user to know NetScript's exact symbol names.
• Internal links contribute prerequisite/next-step routing.
• Filesystem and embedded corpora have parity and bounded responses.
• MCP instructions and generated agent guidance activate the flow before unfamiliar implementation work.
• Follow-up observed usage/adoption is tracked only in verify(wave-five): does the shipped agent surface actually change agent behaviour? #1090.