Add daily agentic workflow audit agent for monitoring workflow health

[Copilot]

Overview

This PR introduces a new agentic workflow that automatically audits all agentic workflows running in the repository, providing daily health monitoring, error detection, and trend analysis.

What does this workflow do?

The audit agent runs daily at midnight UTC (or can be manually triggered) and performs a comprehensive analysis of all agentic workflow runs from the last 24 hours. It:

1. Builds the CLI - Compiles the gh-aw binary needed for log analysis
2. Collects logs - Downloads workflow logs from the past 24 hours using ./gh-aw logs --start-date -1d --verbose with detailed output
3. Analyzes issues - Scans for missing tools, errors, performance problems, and patterns
4. Stores findings - Uses cache-memory to build persistent knowledge across runs
5. Takes action - Creates GitHub issues for problems found, or pull requests with automated fixes

Key Features

Persistent Knowledge Base

The workflow uses cache-memory to store audit history in /tmp/cache-memory/, maintaining:

• Daily audit summaries indexed by date
• Error pattern database for trend detection
• Missing tool request tracking
• MCP server failure analysis
• Token usage and cost metrics over time

This enables the agent to compare current findings with historical data, identify new vs. recurring issues, and track improvement or degradation over time.

Intelligent Reporting

Based on analysis, the agent intelligently decides the appropriate action:

• Create an issue - When problems are found, generates a detailed report with metrics, affected workflows, and actionable recommendations
• Create a pull request - When it can automatically fix issues (e.g., adding missing tool configurations)
• Silent success - When all workflows are healthy, updates cache memory without creating noise

Comprehensive Analysis

The audit covers:

• Missing Tools - Identifies tools requested but not available, with frequency analysis
• Error Detection - Categorizes errors by type (tool failures, MCP issues, permissions, timeouts)
• Performance Metrics - Tracks token usage, costs, turn counts, and efficiency
• Pattern Recognition - Detects recurring issues and common error signatures

Configuration

on:
  schedule:
    - cron: "0 0 * * *"  # Daily at midnight UTC
  workflow_dispatch:

engine: claude

tools:
  cache-memory: true
  bash:
    - "make build"
    - "./gh-aw logs*"
    - "./gh-aw status"
    - "./gh-aw audit*"

safe-outputs:
  create-issue:
    title-prefix: "[audit] "
    labels: [automation, audit, agentic-workflows]
  create-pull-request:
    title-prefix: "[audit] "
    labels: [automation, audit, improvement]
    draft: true

Security

The workflow follows security best practices:

• Minimal permissions (contents: read, actions: read)
• Safe-outputs pattern for issue/PR creation
• No execution of untrusted code from logs
• Input validation and sanitization

Files Added

• .github/workflows/audit-workflows.md (8.7KB) - Source workflow definition
• .github/workflows/audit-workflows.lock.yml (190KB) - Compiled GitHub Actions workflow

Testing

• ✅ All unit tests pass
• ✅ Workflow compiles successfully
• ✅ YAML validation passes

The workflow is ready to run automatically and will help maintain the health and reliability of all agentic workflows in the repository.

Original prompt

Generate an agentic workflow that audits the agentic workflows from the last 24h. It runs daily.

• steps: make build, and run the "logs" command to get all the logs from the last 24h cached locally

• trigger: on schedule and workflow dispatch

• tools: cache-memory,
  bash: make*,./gh-aw*

• safe outputs: create-issue, create-pull-request

• prompt:

Review the logs table and investigate missing tools, errors. Report analysis in issue or propose improvements in a pull request.

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[pelikhan]

@copilot Add ./gh-aw status and ./gh-aw audit*

[Copilot]

Added ./gh-aw status and ./gh-aw audit* to the bash tools allowed list in commit 81202ca.

[pelikhan]

@copilot add --verbose

[Copilot]

Added --verbose flag to the logs command in commit 81202ca.