feat(engine): typed input/output schemas per agent (#58) - #79
Merged
Conversation
Replaces the hardcoded ROLE_FIELDS lookup with per-agent declared contracts: ``input_schema`` and ``output_schema`` are now ``list[str]`` subsets of ``PipelineState.model_fields``. Types are derived from the Pydantic model itself — no second source of truth. Foundation for the rest of v0.5 (#59 cohesion validation, #60 Jinja scoping, #61 form pickers, #62 designer edge annotations). ## Type changes - ``Agent``, ``AgentCreate``, ``AgentUpdate`` schemas: ``dict[str, Any]`` → ``list[str]``. New ``field_validator`` rejects names that don't exist in ``PipelineState.model_fields`` and rejects duplicates. - ORM ``AgentVersionORM.input_schema`` / ``output_schema``: typed ``list[str]`` (column type stays ``JSON`` — accepts both shapes). - TS ``Agent`` / ``AgentCreate`` / ``AgentUpdate`` mirror the change. ## Output parsing - New ``agent_output_model(agent)`` and shared ``resolve_output_validator(role, output_schema)`` in ``dap_types.role_outputs``. Per-agent ``output_schema`` wins; falls back to ``ROLE_FIELDS[role]`` for legacy agents; returns ``None`` for custom roles with no declared schema. - ``parse_node_output`` and ``_parse_role_output`` renamed to ``parse_node_output(role, output_schema, output)`` and ``_parse_agent_output(ctx, output)`` to dispatch on the new helper. Behaviour unchanged when ``output_schema`` is empty (full backward compat with v0.4). ## Migration / back-compat - No DB migration needed — schema is code-driven (SQLAlchemy ``create_all``) and JSON columns accept both dict and list. - ``Agent`` model and ``AgentCreate`` / ``AgentUpdate`` carry a ``mode="before"`` validator that coerces legacy dict payloads (``{}`` and placeholder JSON-Schema-ish blobs) to ``[]``. Removed in v0.6 once clients are on the new shape. ## Tests - ``test_agent_schema_validation.py`` (new): legacy dict coercion, unknown-field rejection, duplicate rejection, non-string element, default factory, None handling. - ``test_crud_agents.py``: 7 new tests for create/update field-list validation (422 on unknown name, 422 on duplicate, persistence round-trip, legacy-dict coercion, default empty list). - ``test_output_parser.py``: existing tests retargeted to the new ``parse_node_output(role, output_schema, output)`` signature; 4 new tests for per-agent ``output_schema`` (overrides role default, enables custom-role validation, widens contract, empty falls back to ROLE_FIELDS). - ``test_pipeline_runner.py`` / ``test_run_lifecycle.py``: ``{}`` → ``[]`` for ORM seed data. Closes #58. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR introduces per-agent typed input/output “contracts” by changing Agent.input_schema/output_schema to list[str] of PipelineState field names, using them for validation and output parsing while keeping legacy role-based defaults for backward compatibility.
Changes:
- Switch agent input/output schemas from dict-shaped blobs to
list[str]with validation (unknown fields / duplicates) and legacy{}/dict/Nonecoercion to[]. - Update engine output parsing to resolve validators via per-agent
output_schemafirst, then fall back toROLE_FIELDS. - Add/extend smoke tests to cover schema coercion + API 422s + output parser behavior.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/smoke/test_run_lifecycle.py | Updates seeded agent versions to use [] schemas. |
| tests/smoke/test_pipeline_runner.py | Updates agent seeding helper to use [] schemas. |
| tests/smoke/test_output_parser.py | Retargets output parser tests to the new parse_node_output(role, output_schema, output) signature and adds per-agent schema tests. |
| tests/smoke/test_crud_agents.py | Adds API-level tests for persistence, coercion, and 422 validation of schema field lists. |
| tests/smoke/test_agent_schema_validation.py | New unit tests for dap_types.Agent schema coercion + field-list validation. |
| packages/types/src/dap_types/role_outputs.py | Adds per-agent validator resolution + shared resolver, and refactors model compilation. |
| packages/types/src/dap_types/agent.py | Changes schemas to list[str] and adds coercion/validation helpers + validators. |
| packages/types/src/dap_types/init.py | Re-exports the new output-validator helpers. |
| apps/engine/src/dap_engine/persistence/models.py | Updates ORM column typing/defaults for input_schema/output_schema to list[str]. |
| apps/engine/src/dap_engine/execution/output_parser.py | Updates parsing logic to accept an explicit output_schema and use the new resolver. |
| apps/engine/src/dap_engine/execution/node_executor.py | Wires node execution to parse output using the agent version’s output_schema. |
| apps/engine/src/dap_engine/api/schemas.py | Updates API request schemas + validators to the new list-of-fields shape and legacy coercion. |
| apps/dashboard/src/lib/api/types.ts | Updates dashboard API types for the new string[] schema fields. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
1. ``node_executor._parse_agent_output`` defends against legacy dict-shaped ``output_schema`` rows: coerce anything that isn't a list to ``[]`` so ``list(dict)`` (= dict keys) doesn't accidentally land in the parser as a per-agent contract. Old rows reliably fall back to ``ROLE_FIELDS`` as intended. 2. ``resolve_output_validator`` sorts ``output_schema`` before lookup so two agents that declare the same field set in different orders share a single cached compiled class (was: cache name was sorted but the fields tuple wasn't, so cache reuse missed). 3. ``_build_model_from_field_subset`` returns ``None`` when a field doesn't exist in ``PipelineState.model_fields`` instead of raising ``KeyError``. Stale or hand-edited rows fall back to permissive merge instead of crashing the run. 4. ``output_parser`` module docstring updated — references :func:`resolve_output_validator` (the actual entry point) instead of the legacy :func:`agent_output_model`. 5. ``coerce_legacy_field_list`` no longer silently coerces non-string elements via ``str(item)``. Returns the value untouched so Pydantic's ``list[str]`` annotation rejects bad input with the standard validation error rather than producing a confusing "unknown field '42'" message later. 6. ``coerce_legacy_field_list`` and ``validate_field_list`` are now public (no leading underscore) so the engine API schemas can import them without coupling to internal module details. New test ``test_stale_field_name_falls_back_to_skipped`` exercises (3). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
rlagowski
approved these changes
Apr 27, 2026
5 tasks
rlagowski
pushed a commit
that referenced
this pull request
Apr 27, 2026
) * feat(prompt-dsl): scope Jinja to declared input_schema fields (#60) When an agent declares ``input_schema`` (v0.5+), the prompt template sees only the fields it lists — referencing anything else fails fast with a descriptive ``PromptBuildError`` pointing at the schema as the fix. Why this matters: - **Documentation**: prompt template + ``input_schema`` together describe what an agent needs. Reading either tells you the contract. - **Refactor safety**: removing a field from ``PipelineState`` fails at template compile time, not at runtime. - **Less coupling**: an agent template can't peek at unrelated state (``run_id``, ``final_status`` etc.) and break determinism. ## Changes - ``build_prompt(template, context, *, input_schema=None)`` — optional projection. ``None`` / empty list → full state visible (backward compat with v0.4 agents that haven't declared a contract). - New ``_project_context`` and ``_format_undefined_error`` helpers — the error surfaces both the missing field name and the currently declared inputs so users know exactly what to add. - ``node_executor`` passes ``ctx.agent_version.input_schema`` (with legacy dict / non-list rows coerced to ``None`` so they pass through with full visibility, matching the #79 fallback strategy). - ``render-preview`` endpoint loads the full agent (not just the template) and forwards ``input_schema`` so the dashboard preview matches runtime behaviour exactly. ## Tests - ``test_prompt_builder.py`` (6 new tests) — declared field renders, undeclared field fails with descriptive scoping error, empty + None pass through, extra context dropped silently, declared-but- missing field still triggers StrictUndefined with the field name. - ``test_render_preview.py`` (2 new tests) — preview honours scoping end-to-end (422 with hint when undeclared, 200 when declared). Closes #60. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * review: distinguish two undefined-error modes (#80) ``_format_undefined_error`` was pointing every undefined-variable failure at ``input_schema`` even when the field WAS declared but the caller forgot to supply a value — misleading hint that sent users on the wrong fix. Now we extract the offending name from the Jinja message and branch: 1. Name not in ``input_schema`` → fix is to extend the schema (or remove the template reference). Existing hint preserved. 2. Name in ``input_schema`` but missing from the render context → fix is to pass the value, NOT to touch the schema. New hint: "The field is declared in agent.input_schema but no value was supplied — pass it in the render context." Falls back to the generic schema reminder when the name can't be parsed (exotic message shapes from attribute access etc.). Updated ``test_input_schema_field_declared_but_missing_in_context_distinct_hint`` to lock in the new branch + assert the misleading hint is absent. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Rafal Lagowski <rla@Rafals-MacBook-Pro.local> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the hardcoded `ROLE_FIELDS` lookup with per-agent declared contracts. Each agent now states exactly which `PipelineState` fields it reads as input and writes as output; the engine uses both for validation and dispatch.
Foundation for the rest of v0.5: #59 (cohesion validation), #60 (Jinja scoping), #61 (form pickers), #62 (designer edge annotations).
Schema (option D — list of field names)
```python
class Agent(BaseModel):
input_schema: list[str] = Field(default_factory=list)
output_schema: list[str] = Field(default_factory=list)
```
Types live in `PipelineState` itself — no parallel type table to keep in sync. Validators reject names that don't exist in `PipelineState.model_fields` and reject duplicates with a 422 listing the offending field(s).
Output parsing
Migration / back-compat
Out of scope (covered by sibling issues)
Test plan
🤖 Generated with Claude Code