Implement workflow pipeline execution surface (spec 058)

[enricopiovesan]

Governing Spec

• 058-workflow-pipeline-execution
• 007-workflow-registry-traversal
• 041-workflow-composition-api
• 009-expedition-example-artifacts
• 042-mcp-library-surface

Project Item

Project 1 item for issue #558 (Status: In Progress, claimed with agent:claude).

Summary

Implements spec 058-workflow-pipeline-execution: multi-capability pipelines executable through the existing workflow traversal engine and the existing execute endpoint, returning merged output under namespaced step keys.

Stacked on #599 (traverse-starter validate/summarize agents, issue #554) — the pipeline integration test executes those agents. Merge #599 first; I will rebase this branch onto main afterwards.

Runtime/registry mechanics (additive, backward compatible)

• WorkflowNodeOutput.publish_to_state_as: Option<String> — stores the entire step output object in workflow state under a namespaced key, alongside existing to_workflow_state hoisting (serde-defaulted; absent in existing artifacts).
• WorkflowDefinition.output_projection: Vec<String> — when non-empty, the final workflow output contains exactly these state keys, producing the spec's merged pipeline shape (e.g. {validate, process, summarize}); empty preserves current whole-state output.
• Registry validation: publish_to_state_as must be non-empty when present; output_projection keys must be non-empty and unique.
• FR-003 (execute endpoint accepts pipeline id) and FR-004 (runtime.submit) work through the existing workflow-backed capability path (implementation_kind: workflow + workflow_ref) — no endpoint changes needed.
• FR-005 (stop on step failure with failed step id) and FR-008 (per-step trace with index and capability id) are existing traversal behavior, now covered by pipeline tests.

Pipeline artifacts

• workflows/examples/traverse-starter/pipeline/workflow.json — validate → process → summarize per spec 058, namespaced outputs, projection.
• contracts/examples/traverse-starter/capabilities/pipeline/contract.json — workflow-backed pipeline capability (traverse-starter.pipeline).
• examples/traverse-starter/registry-bundle/manifest.json + runtime-requests/pipeline.json — registerable bundle and canonical request.
• drafts/workflows/examples/doc-approval/pipeline/workflow.json + drafts/contracts/examples/doc-approval/capabilities/pipeline/contract.json — extract → analyze → recommend, quarantined under the top-level drafts/ (the contractual-enforcement gate's designed quarantine) because the step capabilities belong to Add doc-approval.extract capability contract, WASM agent, and app manifest #538/Add doc-approval.analyze capability contract, WASM agent, and app manifest #556/Add doc-approval.recommend WASM agent for pipeline showcase #555 (Add doc-approval.recommend WASM agent for pipeline showcase #555 blocked) and their contracts are not merged yet, so the workflow cannot register. Wiring is conservative (input_from: initial_input — every step reads document). Move out of drafts/ alongside those tickets; revisit if the recommend step needs analysis fields.

Validation

• cargo test --workspace — all 25 suites pass.
  • Runtime: pipeline_workflow_merges_namespaced_step_outputs_deterministically (same input twice → identical result, exact merged output, per-step trace records), pipeline_workflow_stops_on_failed_step_with_failed_step_id_in_trace.
  • Registry: rejection tests for empty publish_to_state_as and duplicate/empty output_projection keys; parse-compat test that old workflow JSON (without the new fields) still parses with defaults.
  • CLI end-to-end: traverse_starter_pipeline_bundle_executes_with_merged_namespaced_output loads the shipped bundle manifest, real contracts, and real workflow JSON, executes traverse-starter.pipeline twice via Runtime::execute, and asserts the exact deterministic merged output.
  • HTTP surface: execute_endpoint_runs_pipeline_workflow_capability_with_merged_output registers step capabilities + workflow + workflow-backed pipeline capability, then POST /v1/workspaces/{id}/execute with the pipeline capability id returns 200 with the namespaced merged output (FR-003, response shape unchanged).
• cargo clippy --workspace --all-targets -- -D warnings — clean.
• cargo fmt --all — applied.
• bash scripts/ci/spec_alignment_check.sh — passes.

Unblocks

reference-apps #110 (traverse-starter pipeline) now; #111 (doc-approval) once #538/#556/#555 agents land.

Closes #558.

🤖 Generated with Claude Code