docs: agent design guide — cost model, round semantics, hard vs. soft enforcement; two corrections to existing pages

[sierragolflima]

Summary

Building and cost-tuning a document-extraction agent (it reads a 60–80-page financial-report PDF from a content collection and writes ~60 typed fact nodes into a dimensional model) surfaced a set of generic agent-design lessons. Iterating the same task from ~$14 down to under $2 per run — with correctness improving at the same time — required understanding several framework behaviors that are currently undocumented, and one existing page contains guidance that directly caused failed runs.

This issue proposes: (1) two targeted corrections to existing pages, and (2) a new documentation page, "Designing agents", capturing the transferable patterns. Draft content for all of it is included below — happy to submit it as a PR if the direction is agreed.

What exists today (inventory)

Page	Covers
Architecture/AgenticAI.md	Architecture: agents as mesh nodes, hierarchy/entry agents, MeshPlugin, MCP, multi-agent roles
AI/AgenticAI.md	Concepts: misbeliefs, human-in-the-loop, delegation vs. handoff
AI/Tools/MeshPlugin.md	Tool reference incl. Create/Update workflows
Architecture/ContentChunkNavigation.md	Chunking model, retrieval vs. extraction, the three chunk tools
DataMesh/UnifiedPath.md	Unified paths; the content-transformer table is current (post-#396)

None of these covers the agent cost model, round semantics, or the AgentConfiguration enforcement surface (plugin whitelist, per-method filter, tool budget).

Correction 1 — ContentChunkNavigation.md: where the full text actually is

"Retrieval vs. extraction" currently says: "Read the whole document, not chunks. Use Get on the file's Document node (or the raw content/ file)." The first half of that parenthetical is wrong in an important way:

• Get on the Document node ({collection}/_Documents/{file}) returns metadata plus a ~300-character summary — never the text.
• The full extracted text is served only by Get on the file path ({collection}/{file}), via the registered content transformers.

Observed failure mode (reproduced in three consecutive runs): search returns the Document node (that is what gets indexed), the agent Gets that path, receives metadata, concludes the full text is unavailable, and falls back to stepping through every chunk. For a ~150-chunk report this multiplies read cost by roughly 5× and re-introduces exactly the chunk-boundary table corruption the same section warns against.

Suggested edit: name the two path shapes explicitly, state what each returns, and give the derivation rule (replace the /_Documents/ segment with / to go from the search hit to the readable file). Apply the same fix to step 4 of "Reading a document end to end", which currently says to Get the hit's documentPath.

Correction 2 — MeshPlugin.md, Create: typed-content discipline

The "Critical rules" for Create cover id/name/icon but omit the highest-stakes rule for typed node types: content must carry the exact $type discriminator and the exact field names of the registered type — unknown fields are silently dropped at deserialization. A wrong field name produces a node that looks successfully created but renders without values in every typed view, which is far harder to notice than an error. Suggested additions:

• State the silent-drop behavior explicitly.
• Recommend the form-template pattern: before creating instances of a type the agent has not written before, Get one existing node of that type and copy its content structure exactly.
• Recommend read-back verification: after a batch of creates, Get one node per type and confirm $type, reference fields, and the value fields actually persisted.

Proposed new page: "Designing agents" (suggest AI/AgentDesign.md)

A practices page between the concepts page and the tool reference. Draft outline with the intended content:

1. The cost model: turns × context

Every tool call is a model turn, and every turn re-sends the entire conversation so far — instructions, all prior tool results, all intermediate text. Cost is therefore multiplicative: (number of turns) × (context per turn). The two design levers are reducing turns (batch tool calls, avoid per-item loops) and keeping heavy artifacts (a full document, a large reference table) out of the context during high-turn phases. A worked example: ~60 create calls issued one per turn on top of a context containing a full document re-bills that document ~60 times; the identical work with the document absent and creates batched costs a tenth as much.

2. Round semantics: what survives a round boundary

When a new user message starts a round, the conversation history is rebuilt from the stored message texts only — tool calls and tool results are not replayed. Two consequences:

• Heavy tool results die with the round. This is a feature: end the reading phase with a round boundary and the document no longer burdens the write phase.
• Anything the next round needs must be written into the assistant's answer text. State that lives only in tool results (including facts the agent discovered and acted on) is gone after the boundary.

Also worth stating: token usage is stamped when a round terminates; a round that dies mid-flight (infrastructure error, cancellation) may record no usage at all, so thread-level cost figures are a floor (see #369).

3. The two-phase extract-then-write protocol

For extraction-shaped tasks, split the work across a round boundary: phase 1 reads the source(s) and ends by emitting a complete, self-sufficient worksheet as the answer text; the user reviews it and continues; phase 2 creates all nodes strictly from the worksheet. This yields: the document is billed only during phase 1; the worksheet is a human review gate before anything is written; and the worksheet is durable checkpoint state — after a crash or interruption, a resumed round still has it. Corollary: the worksheet must be genuinely self-sufficient (including any master data needed to create new dimension nodes), because the source is unreachable in phase 2.

4. Hard vs. soft enforcement — affordance beats instruction

Document the AgentConfiguration enforcement surface: the plugins whitelist (whole plugins and per-method filters, e.g. Mesh restricted to Get, Search, Create, Update, Patch) and maxToolCallsPerRound. Then the design rule learned the hard way: when an agent repeatedly takes a forbidden action, remove or restrict the enabling tool instead of escalating prompt wording. In our case the model ignored three successive prompt-level prohibitions of a tool-usage pattern (each fix stronger and closer to the action); removing the enabling plugin from the whitelist ended it instantly. Prompt text competes with tool affordances and loses. Also: keep the prompt's description of the tool surface consistent with the actual whitelist — a prompt that names tools the agent does not have (or vice versa) invites unpredictable behavior.

5. Writing instructions that survive rounds and failures

• Consistency across round boundaries. A rule like "load the document exactly once" reads differently after a reset: the new round has no memory of the load, so the model reads it as "not yet done" and loads again. Scope such rules explicitly to the phase they belong to.
• Rules at the action site. A path-derivation rule buried in a general section lost to the affordance of a search result three runs in a row; the same rule as the first sentence of the step that performs the action, plus a recovery clause for the observed failure signature ("if a Get returned only metadata, you fetched the wrong node — correct the path before doing anything else"), held.
• Countable rules for batching. "Batch your calls" is ignored; "every response in this step MUST contain at least 10 parallel create calls; a response with exactly one is a protocol violation" produced real batches. Even then, expect a compliance ceiling — treat instruction-level batching as an optimization, not a guarantee.
• A stale-state exception where it matters. If a checkpoint artifact can be stale (crash between phases), allow exactly one re-verification — existence of dimension nodes immediately before creating them — and keep everything else bound to the checkpoint.

6. Write-phase patterns for fact/dimension models

• Deterministic ids ({shortKey}-{period}-{position}) make interrupted runs resumable: re-creates collide instead of duplicating.
• The dimension node's id is the short key used in fact ids and reference paths — never an invented long name. Two spellings of the same dimension split the dataset silently: facts referencing the wrong one disappear from every view keyed on the right one.
• Existence checks by listing, not by name-substring. Dimension counts are small; list them all in one search and match key AND display name. A substring search on a name variant returns a false negative, and the agent then creates a duplicate.
• Verification = one count-check per created type plus one read-back spot check per type — not per-node reads.
• Do not verify through the search index immediately after writing. The index is eventually consistent: a count-search issued right after a batch of creates can return zero, and an agent then silently downgrades its verification instead of failing it (observed in two consecutive otherwise-clean runs). With deterministic ids the robust check is direct Get on the expected paths; otherwise defer the count or reconcile explicitly against the checkpoint.
• Reconcile writes against the checkpoint. The closing summary should enumerate which checkpoint rows were intentionally not written and why (e.g. values the source does not report). An unexplained delta between "rows in the worksheet" and "nodes created" is the signature of silent under-creation — observed as worksheets claiming full coverage while a third fewer nodes were written, with only a fraction of the gap explained.

7. Operational notes

• Measuring true cost: sum every _Usage satellite across the thread tree (delegation sub-threads record separately and may be priced as unknown-model, see Delegation sub-thread token usage is recorded under model (unknown) — sub-thread cost is unpriced ($0) #369); error-terminated rounds may be missing entirely.
• Filename hygiene: until path canonicalization ships (Content collection route does not URL-decode path segments — files/folders with spaces (or other encoded characters) return "not found" #364), prefer ASCII filenames for uploads; decomposed-Unicode names index fine but fail by-name reads.

Measured effect (same task shape, same model class)

Configuration	Cost per report
Naive: single-phase, per-value reads, per-node creates	$14–23, frequent failures
Two-phase protocol, but document re-read in phase 2 + sequential creates	~$13.5
+ full-text read via one Get (transformers, #396) + chunk tools removed + batched creates	$1.9–2.0 per clean run (~$4 when a round was interrupted and redone)

Relationship to existing issues

• Content transformers: restore PDF/XLSX + describe images on save (get never returns bytes) #396 (content transformers) made the single-Get read path real; the docs corrections above align the guidance with it.
• Delegation sub-thread token usage is recorded under model (unknown) — sub-thread cost is unpriced ($0) #369 (usage attribution) and Content collection route does not URL-decode path segments — files/folders with spaces (or other encoded characters) return "not found" #364 (path normalization) are referenced by the operational notes; this issue does not duplicate them.

Out of scope (possible follow-up)

A bulk Create accepting an array of nodes would collapse the write phase to one or two calls and remove the dependency on model batching behavior entirely. If there is interest, that can be filed separately as an enhancement.