fix(providers): recover from schema-shape failures instead of failing the run (#343)

[jrob5756]

Fixes #343.

An agent returning syntactically valid JSON with a wrong-typed field killed the workflow immediately with zero recovery attempts, even though max_parse_recovery_attempts exists for exactly this class of contract violation.

Verified against the real failure: on main the reported payload fails with Output field 'decision' has wrong type: expected string, got dict — the issue's error verbatim. On this branch it succeeds, driven through the actual WorkflowEngine rather than the provider in isolation.

Why the fix has to live in the provider

Schema validation ran a layer above the providers, in executor/agent.py:318, after the provider had already returned. For Copilot that is also after finally: await session.disconnect() — so by the time the error surfaced, the session that could have been re-prompted was already gone. The executor layer could not recover even in principle.

Claude had the same gap

Contrary to the table in the issue, claude was not failing fast by design. Its except ValidationError: raise at claude.py:1271 is scoped to the network-retry loop and means "don't burn API retries on a deterministic error" — sensible, and a separate concern. But its actual recovery loop returned the moment _extract_structured_output or _extract_json_fallback yielded anything. It never checked shape either.

Changes

• copilot — validates inside the recovery loop against output_schema. (The snippet proposed in the issue passes schema_for_prompt, a prompt-facing dict of type/description strings that would AttributeError on field_def.type.)
• claude — _evaluate_structured_response classifies each response. It validates the emit_output and JSON-fallback paths but deliberately not the MCP tool-use path, which returns to the agentic loop rather than being a final answer. A tool_use-origin failure is replayed as text, because a bare tool_use block without a matching tool_result violates the Anthropic message contract.
• hermes — already validated in-loop; gains the unwrap and now honors retry.max_parse_recovery_attempts, which it had been ignoring in favour of a hardcoded 3.
• Exhausted budget re-raises the original ValidationError, so the field name and expected type survive. Syntax failures still raise ProviderError. This is deliberately better than the hermes behaviour the issue asked us to copy — hermes discarded that detail.
• Non-object JSON (a bare 42, null, an array) is re-prompted as a shape failure instead of reaching validate_output and raising an uncaught TypeError.
• normalize_agent_output conservatively unwraps wrapper-shaped scalars: only when the schema declares a scalar and exactly one candidate under the field's own name or a generic value/result key has the expected type. Ambiguity and any other key shape are re-prompted rather than guessed at. Kept out of validate_output, which also validates set and script output where silent reshaping would be a surprise.
• agent_parse_recovery event through provider → console → dashboard. Recovery was previously invisible outside verbose mode — the issue notes "no entry in the log shows a recovery attempt".

Review notes

This went through six review passes; several findings were substantive and are worth calling out for reviewers.

Two live defects were found and fixed after the initial implementation. Non-object JSON raised an uncaught TypeError from a membership test — on Copilot this was a regression introduced by the first commit here, surfacing as a retryable auth error that burned the whole agent retry budget. Separately, Copilot's interrupt/partial path could return a non-dict as AgentOutput.content, which is declared dict[str, Any] and has no recovery loop.

The unwrap heuristic was tightened. It originally included a bare sole-key fallback and would resolve {"answer": {"error": "I could not complete the task"}} into an answer, and flip {"approved": {"not_approved": false}}. It now requires an unambiguous named match. A separate bug meant a field literally named value or result collided with the generic keys and was never unwrapped at all.

A trap worth knowing. parse_json_output wraps JSON syntax errors in ValidationError too, so the two failure kinds cannot be told apart by exception type. Hermes splits them by which call failed. This is documented in AGENTS.md because it is easy to reintroduce.

One behaviour change to a public error surface. test_json_schema_validation_error asserted the old generic ProviderError and now asserts the preserved ValidationError. A companion test pins the ProviderError syntax path.

One divergence pinned rather than fixed. Hermes accepts a bare string where Copilot and Claude reject it, because parse_json_output rewrites non-dicts to {"result": ...} first. Changing that would affect set and script steps, so it is covered by a test with an explanatory docstring instead.

Validation

• 4505 passed, 35 skipped
• make check (lint + typecheck) clean
• Frontend rebuilt via make build-frontend; make test-frontend green
• Rebased onto main after feat(claude-agent-sdk): support workflow MCP servers (#335) #346 landed
• Six behaviours re-verified end-to-end after every restructuring: reported payload, ambiguous wrapper, error-signal, bare scalar, array, syntax-then-recover