call_workflow (and other renamed dynamic tools) get a duplicate, unwired generic tool that silently no-ops

[peter-hendy]

Summary

When a workflow configures the built-in call_workflow safe-output feature targeting a specific reusable workflow, the compiler correctly generates a properly-typed tool named after the target workflow (e.g. agent_sandbox_stack for a workflow named agent-sandbox-stack). However, the runtime stdio safe-outputs MCP server (actions/setup/js/safe_outputs_tools_loader.cjs, registerDynamicTools()) also creates a second, generic, unwired tool literally named call_workflow for the same config key — because its "already handled?" dedup check only compares by exact tool name, and the real tool was renamed away from call_workflow.

An agent presented with both tools has no way to know one is fake. Calling the generic call_workflow tool returns a false "executed successfully" response (isError: false) and writes a malformed record with no workflow_name, which the downstream safe-outputs processor then rejects with Workflow name is empty — but by then the agent has already reported success and moved on. This causes a silent, hard-to-diagnose failure: the requested workflow call never happens, and nothing in the agent's own transcript indicates anything went wrong.

Root cause

• pkg/workflow/safe_outputs_call_workflow.go (generateCallWorkflowTool / generateWorkflowToolDefinition) names the generated tool after the target workflow and tags it with _call_workflow_name metadata — not named call_workflow itself.
• actions/setup/js/safe_outputs_tools_loader.cjs, registerDynamicTools(), dedups only by exact tool name:

  if (server.tools[normalizedKey] || tools.find(t => t.name === normalizedKey)) {
    return;
  }

  For config key call_workflow, neither check finds the already-registered agent_sandbox_stack tool (different name), so it falls through and synthesizes a second, generic tool — additionalProperties: true, handler writes {type: "call_workflow", ...args} straight to the output file, bypassing all the metadata-wrapping logic in attachHandlers().
• This only affects the stdio safe-outputs MCP server path (the default/compiler-selected path). The HTTP variant (safe_outputs_mcp_server_http.cjs) uses a different registration loop and is not affected.
• The same root cause (treating every safe-outputs config key as a potential agent-facing tool unless something already has its exact name) also exposes create_report_incomplete_issue as a bogus tool alongside the real report_incomplete tool, and would similarly affect dispatch_workflow, dispatch_repository, and replace_label if configured.
• Degenerate case: if the target workflow itself normalizes to literally call_workflow (e.g. named call-workflow), the exact-name check happens to succeed and no duplicate is created — this only bites "normal" target names.

Reproduction

Live reproduction (not just source reading) against current main:

• config: {call_workflow: {workflows: ["agent-sandbox-stack"]}}
• Registered tools: ["agent_sandbox_stack", "call_workflow"]
• Calling call_workflow with arbitrary args returns:

  {"content":[{"type":"text","text":"{\"result\":\"Safe-job 'call_workflow' executed successfully with arguments: {...}\"}"}],"isError":false}

  and writes raw record {"type":"call_workflow", ...args} — no workflow_name.
• Passing that raw record to the real downstream handler (actions/setup/js/call_workflow.cjs) returns exactly: Workflow name is empty.

This exact failure mode occurred for us in production: an agent run had both agent_sandbox_stack and call_workflow available, called call_workflow, got a false success, and the requested stack-verification handoff silently never happened.

Related issues (not duplicates)

• HTTP safe-outputs server does not register generated call-workflow tools #21074 — fixed the opposite problem (workflow-named tools not registered at all), on the HTTP path only.
• call-workflow is not wired into the consolidated safe_outputs handler-manager path #21205 — fixed unrelated handler-manager wiring.
• [aw-failures] [aw-fix] P1: report_incomplete safe output rejected — "Missing required field: workflow_name" #46376 / [aw-failures] [aw-fix] P1: Test Quality Sentinel — create_report_incomplete_issue rejected: missing incomplete_signals array #47526 (fixed by fix(safe-outputs): fall back to env vars when create_report_incomplete_issue omits workflow_name #46400 / Harden report_incomplete issue handling when incomplete_signals is missing/empty #47533) — made the downstream handler tolerate malformed create_report_incomplete_issue calls, without addressing that the bogus tool itself shouldn't exist.

Suggested fix

Narrow/tactical: extend the dedup check for call_workflow (and dispatch_workflow, dispatch_repository) to also treat any tool carrying the corresponding metadata key (_call_workflow_name, _workflow_name, _dispatch_repository_tool) as "already handled," not just an exact name match. Roughly a one-line change per family, plus a regression test.

Structural: make registerDynamicTools() tool-definition-driven rather than config-key-driven — only synthesize a generic tool for config keys that don't correspond to any known tool family (via metadata), rather than inferring "custom safe-job" status from the mere presence of a config key. This also fixes the create_report_incomplete_issue leak and prevents future config keys from hitting the same trap.

Happy to provide the reproduction script if useful.