[cli-tools-test] audit: misleading "failed with 0 error(s)" and wrong workflow_name for runs that fail before agent activation

[github-actions]

Problem Description

When auditing workflow runs that fail before the agent job starts (e.g., cancelled runs, runs that fail at the infrastructure level with no downloadable logs), the audit tool produces two related issues:

1. Contradictory error message: The key_findings says "Workflow failed with 0 error(s)" — this is logically contradictory. A run either failed or had 0 errors; saying both is confusing to operators trying to diagnose problems.

2. Wrong workflow_name format: For such runs, the audit returns the file path (.github/workflows/daily-cli-tools-tester.lock.yml) instead of the display name (Daily CLI Tools Exploratory Tester). The logs tool correctly returns the display name for the same runs.

Tool

• Tool: audit

Steps to Reproduce

1. Find a workflow run that failed immediately (before any jobs ran or with updated_at == created_at)
2. Use the audit tool with that run ID:
   • Run 23219499044 (Daily CLI Tools Exploratory Tester, push event, branch copilot/fix-copilot-access-azure-apis)
   • Run 23162378624 (Daily CLI Tools Exploratory Tester, push event, branch copilot/recreate-pr-for-current-main)
3. Observe the misleading output

Expected Behavior

For the message: When a run fails without any error details available (no downloadable logs), the message should be:

• "Workflow failed - no error logs were available to analyze" OR
• "Workflow failed before agent activation — infrastructure or pre-activation failure"

For workflow_name: Should return the display name (Daily CLI Tools Exploratory Tester), consistent with how the logs tool returns it for the same runs.

Actual Behavior

{
  "overview": {
    "run_id": 23219499044,
    "workflow_name": ".github/workflows/daily-cli-tools-tester.lock.yml",
    ...
  },
  "key_findings": [{
    "category": "error",
    "severity": "critical",
    "title": "Workflow Failed",
    "description": "Workflow '.github/workflows/daily-cli-tools-tester.lock.yml' failed with 0 error(s)",
    "impact": "Workflow did not complete successfully and may need intervention"
  }],
  "downloaded_files": null
}

Compare with the logs tool for the same run:

{
  "workflow_name": "Daily CLI Tools Exploratory Tester",
  "conclusion": "failure",
  "error_count": 0
}

Environment

• Repository: github/gh-aw
• Testing Run ID: 23272741749
• Date: 2026-03-18
• Affected runs: 23219499044, 23217813123, 23162378624, 23201717127

Impact

• Severity: Medium
• Frequency: Always (reproducible on any run that fails before agent activation)
• Workaround: None — users are left with a confusing "failed with 0 errors" message and an incorrect workflow identifier

Additional Context

A third related issue: auditing a non-existent run ID (e.g., 99999999999) returns a generic MCP internal error (-32603: failed to fetch run metadata) rather than a user-friendly message like "Run ID 99999999999 not found". This makes it hard to distinguish between a bad run ID, a network error, or a permissions issue.

All three issues share the root cause: the audit tool does not handle the "no logs available" and "run not found" cases with appropriate user-facing messages.

Generated by Daily CLI Tools Exploratory Tester · ◷

• expires on Mar 25, 2026, 11:58 PM UTC