Skip to content

[WIP] Fix missing tool issue causing false workflow success#23309

Closed
Copilot wants to merge 1 commit intomainfrom
copilot/fix-missing-tool-workflow-success
Closed

[WIP] Fix missing tool issue causing false workflow success#23309
Copilot wants to merge 1 commit intomainfrom
copilot/fix-missing-tool-workflow-success

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 28, 2026

Thanks for asking me to work on this. I will get started on it and keep this PR's description up to date as I form a plan and make progress.


This section details on the original issue you should resolve

<issue_title>missing-tool: workflow completes successfully when MCP script tools fail</issue_title>
<issue_description>## Summary

When an MCP script tool is unavailable or fails at runtime (e.g., Docker no space left on device), the agent reports output_types: missing_tool but the workflow completes successfully. This makes it impossible to detect when a workflow silently failed to accomplish its task.

Observed Behavior

  1. Agent calls an MCP script tool (cargo) that uses Docker internally
  2. Docker fails with no space left on device (runner disk too small for image)
  3. Agent retries the tool a few times, all fail
  4. Agent reports missing_tool in agent_output.json and exits cleanly
  5. All jobs (activation, agent, safe_outputs, conclusion) pass ✅
  6. Workflow run appears green despite accomplishing nothing

Example run: https://github.com/github/blackbird/actions/runs/23658300309

The agent_output.json shows:

{
    "items": [
        {
            "type": "missing_tool",
            "tool": "mcpscripts-cargo (Docker build container at host.docker.internal:3000)",
            "reason": "The cargo build container service is unreachable — all calls to mcpscripts-cargo time out..."
        }
    ]
}

Yet all 4 jobs passed.

Expected Behavior

The workflow should either:

  • Fail by default when missing_tool is the only output type and no changes were made — the agent could not do its job
  • Provide a frontmatter option (e.g., required-tools: [cargo]) to explicitly fail when specific tools are unavailable
  • At minimum, surface missing_tool as a warning annotation on the workflow run summary so it is visible without downloading artifacts

Context

  • gh-aw version: v0.58.1
  • Tool type: MCP script (run: shell type) that starts a Docker build container
  • Root cause: ubuntu-latest runner had 14GB disk, insufficient for the Docker image
  • The failure was completely invisible in the workflow status — only discoverable by downloading the agent artifact and reading agent_output.json

Related

Comments on the Issue (you are @copilot in this section)

@pelikhan @copilot missing tool should fail the run, with a continue-on-error option to turn into warning.

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

missing-tool: workflow completes successfully when MCP script tools fail

2 participants