fix(agents): provider robustness — OpenAI strict schema, Anthropic refusals, retries, real request timeout - #467
Merged
Conversation
epeicher
disabled the stack merge
July 31, 2026 17:18
OpenAI validates response_format json_schema with strict: true, and
strict mode has no optional fields: 'required' must list EVERY key in
'properties', or the whole request 400s ("'required' is required to
be supplied and to be an array including every key in properties.
Missing 'style'.").
- desktop_mode_agent_answer_schema(): 'style' joins the call-to-action
items' required list and 'call_to_actions' joins the root's. The
lenient parse/sanitize path is unchanged, so pre-filter runtimes and
non-strict providers still work as before.
- desktop_mode_ai_normalize_response_schema() now also repairs partial
'required' lists (recursively, same walk as the additionalProperties
stamp), so a plugin-filtered schema addition cannot reintroduce the
400 on OpenAI. Google's provider already strips response-schema keys
it dislikes, so the repair is provider-safe.
Verified live against the OpenAI connector (Google/Anthropic provider
plugins deactivated to force routing): the previously failing chat now
returns a structured answer with working call-to-action buttons.
…t the turn cap
Two Anthropic-connector failures reproduced live, both one-off flaps
rather than deterministic rejections:
- 'Unexpected Anthropic API response: Missing the "content" key.' —
the provider throws when a 200 arrives with an EMPTY content array
(caught in a raw trace: status=200, no stop_reason, content=[]). A
fresh request immediately succeeds.
- 'Agent stopped after 8 turns without a final answer.' — the model
spends the whole cap calling tools (with thinking enabled) before
answering; the run then discarded everything it gathered.
Fixes, both at the runner (the reliability boundary the HTTP-timeout
work established):
- One bounded retry per generate turn when the failure matches a
transient signature (empty Anthropic content, a failed models-list
fetch ('No models found'), gateway 5xx, transport timeout).
Deterministic rejections (schema 400s, rate limits) are never
retried. desktop_mode_agent_generate_error_is_transient() is the
pure, unit-tested classifier.
- At the turn cap, one forced TOOL-LESS generate over the transcript:
with nothing to call, the model can only produce a final answer from
what it already gathered. Falls back to the original error when even
that fails; turns reports MAX+1.
…refusals - The 'cURL error 28: Operation timed out after 30007 milliseconds' failures were NOT the WordPress HTTP default: Core's WP_AI_Client_Prompt_Builder constructor pins an explicit 30-second timeout via the SDK's RequestOptions, which reaches the transport directly and bypasses the http_request_timeout raise entirely. The timeout wrapper now also raises Core's wp_ai_client_default_request_timeout filter (raise-only, removed in finally, same scoping) so long generations get the full allowance. - The 'Unexpected Anthropic API response: Missing the "content" key.' errors are model REFUSALS, not flaps: a raw-body trace shows 200 responses with stop_reason 'refusal', an empty content array, and stop_details.category 'bio' over innocuous demo content — the provider plugin crashes on the empty content before reaching its own refusal handling. desktop_mode_agent_humanize_generate_error() now translates that parse error into an actionable message (rephrase or switch provider), preserving the original detail in error data; the transient classifier's comments are corrected accordingly (one retry kept for borderline refusals).
epeicher
force-pushed
the
fix/agents-answer-schema-openai-strict
branch
from
July 31, 2026 17:19
0c3ad74 to
bf378bd
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #466.
Testing
What it does
Fixes four provider failures hit while testing #466:
Unexpected Anthropic API response: Missing the "content" key.— a model REFUSAL the Anthropic provider plugin crashes on before its own refusal handling.Agent stopped after 8 turns without a final answer.— the model spending the whole turn cap on tool calls.cURL error 28: Operation timed out after 30007 milliseconds— the AI Client's own 30-second request timeout, which feat(agents): enhance agent capabilities and HTTP timeout management #466'shttp_request_timeoutraise cannot reach.OpenAI: strict structured output 400
Rationale
The OpenAI provider sends the answer schema verbatim with
strict: true, and strict structured output has no optional fields: every key inpropertiesmust also appear inrequired. The agent answer schema listedstyleas optional in the call-to-action items (andcall_to_actionsas optional at the root), which OpenAI rejects before generating anything. #466 already covered the other strict rule (additionalProperties: false) and fixed the Copilot/drafts schemas; the agent answer schema'srequiredlists were the remaining gap.Implementation
desktop_mode_agent_answer_schema()declares fullrequiredlists:[id, label, style, reply]on the items,[text, call_to_actions]at the root. The lenient parse/sanitize path is untouched — a bad or missingstylestill defaults tosecondaryfor pre-filter runtimes and non-strict providers.desktop_mode_ai_normalize_response_schema()(the safety net feat(agents): enhance agent capabilities and HTTP timeout management #466 introduced foradditionalProperties) now also repairs partialrequiredlists in the same recursive walk, so a plugin-filtered schema addition cannot reintroduce the 400. Provider-safe: Google's provider already strips response-schema keys its API rejects.Testing instructions
Manual, with only the OpenAI provider plugin active (deactivate the Google/Anthropic providers to force routing): chat with any agent asking for a proposal that needs approval. Before this fix the answer is the 400 above; after, a structured answer arrives with working call-to-action buttons. Verified live against the OpenAI connector.
Anthropic: refusal handling, retry, forced final answer, real timeout
All reproduced live with only the Anthropic provider active. A raw-body trace of the failing "Missing content" case settled what it actually is:
A model refusal (a safety-classifier false positive over innocuous demo content), returned as a 200 with an empty
contentarray — which the provider plugin throws on ("Missing the "content" key") before reaching its ownrefusalhandling. Not a flap: retries usually reproduce it.desktop_mode_agent_humanize_generate_error()maps that parse error to an actionable message ("its safety system most likely declined this request — rephrase, or switch the provider in Settings → Connectors"), preserving the provider's original text in the error data.No models found …), gateway 5xx, transport timeouts, and the empty-content case (kept because borderline refusals are stochastic and one request is cheap).desktop_mode_agent_generate_error_is_transient()is the pure, unit-tested classifier; deterministic rejections (schema 400s, rate limits) are never retried.DESKTOP_MODE_AGENT_RUNNER_MAX_TURNSturns calling tools, the runner makes one last TOOL-LESS generate over the transcript — with nothing to call, the model can only answer from what it already gathered. Falls back to the original max-turns error if even that fails;turnsreportsMAX + 1. Thedesktop_mode_agent_runner_generatefilter doc notes it can now be invoked more than once per turn.WP_AI_Client_Prompt_Builderconstructor pins an explicit 30-second timeout via the SDK'sRequestOptions, bypassing the WordPress HTTP default that feat(agents): enhance agent capabilities and HTTP timeout management #466's wrapper raises — hence "timed out after 30007 milliseconds" despite the 180s allowance. The wrapper now also raises Core'swp_ai_client_default_request_timeoutfilter, with identical raise-only +finally-removal scoping.Verified live with only the Anthropic provider active: the question that produced the 8-turn error now completes (turns 7 and 8 across runs), and a refused translation request surfaces the explained refusal instead of the parse error.