[delight] UX Analysis Report - 2026-05-01: Workflow Message Clarity & Documentation Scannability

[github-actions[bot]]

User Experience Analysis Report - 2026-05-01

Executive Summary

Today's analysis focused on:

• 2 documentation files
• 2 CLI commands
• 2 workflow message configurations
• 1 validation test file

Overall Quality: Professional — the codebase maintains a high baseline standard across all reviewed areas.

Key Finding: The dev-hawk.md workflow uses a playful hawk-themed message voice ("Hawk down!", "The skies grow quiet...") that reduces professional clarity when failures occur. This is the highest-impact single-file improvement available.

---

Quality Highlights ✅

Example 1: audit Command Help Text

• File: pkg/cli/audit.go
• What works well: Exhaustive coverage of URL formats accepted by the command, clear multi-run diff capability explanation, practical examples covering all major usage patterns
• Quote: "Each argument accepts: A numeric run ID ... A GitHub Actions job URL with step (e.g., ...#step:7:1)" — the enumeration of every accepted input format is exactly what enterprise users need

Example 2: MemoryOps Guide

• File: docs/src/content/docs/guides/memoryops.md
• What works well: Six clearly-labelled patterns with real workflow names as references, inline code examples, explicit TTL guidance for caching, and a security section warning against PII storage
• Quote: "suggested TTLs: repository metadata (24h), contributor lists (12h), issues/PRs (1h), workflow runs (30m)" — concrete numbers eliminate guesswork

---

Improvement Opportunities 💡

High Priority

Opportunity 1: Playful Hawk Messages Reduce Clarity on Failure - .github/workflows/dev-hawk.md

• File: .github/workflows/dev-hawk.md
• Current State (lines 31–34):

  footer: "> 🦅 *Observed from above by [{workflow_name}]({run_url})*..."
  run-started: "🦅 Dev Hawk circles the sky! [{workflow_name}]({run_url}) is monitoring this {event_type} from above..."
  run-success: "🦅 Hawk eyes report! [{workflow_name}]({run_url}) has completed reconnaissance. Intel delivered! 🎯"
  run-failure: "🦅 Hawk down! [{workflow_name}]({run_url}) {status}. The skies grow quiet..."

• Issue: The military/wildlife-themed language turns a failure notification into a riddle. A developer who sees "Hawk down! The skies grow quiet" must mentally translate before understanding the workflow failed. The {status} placeholder provides the actual signal, but it's buried after theatrical text.
• User Impact: Increases cognitive load during incident response. "Hawk down" reads as humorous, potentially causing users to under-prioritise a real failure.
• Design Principle: Clarity and Precision — messages must communicate state directly, not through metaphor
• Suggested Change: Keep the hawk persona in the workflow name and footer attribution, but make run lifecycle messages functional:

  run-started: "🦅 [{workflow_name}]({run_url}) is monitoring this {event_type}..."
  run-success: "🦅 [{workflow_name}]({run_url}) completed monitoring successfully."
  run-failure: "🦅 [{workflow_name}]({run_url}) {status} during monitoring."

Medium Priority

Opportunity 2: Best Practices Section is a Dense Paragraph - docs/src/content/docs/introduction/how-they-work.mdx

• File: docs/src/content/docs/introduction/how-they-work.mdx
• Current State (lines 67–68):

  Start simple and iterate with clear, specific instructions. Test workflows using gh aw compile --watch and gh aw run, monitor costs with gh aw logs, and review AI-generated content before merging. Use safe outputs (pre-approved GitHub operations) for controlled creation of issues, comments, and PRs.

• Issue: Six distinct recommendations are packed into a single run-on paragraph. A user skimming the page cannot extract individual action items.
• User Impact: First-time users reading this as a quick reference cannot scan for what they need to do next.
• Design Principle: Efficiency and Productivity — reduce cognitive load for scanning
• Suggested Change: Convert to a scannable list:

  - **Start simple**: Write clear, specific instructions and iterate
  - **Compile and test**: Use `gh aw compile --watch` and `gh aw run` during development
  - **Monitor costs**: Use `gh aw logs` to track token usage and cost
  - **Review AI output**: Always review AI-generated content before merging
  - **Control writes**: Use [`safe outputs`](/gh-aw/reference/safe-outputs/) for controlled creation of issues, comments, and PRs

---

Files Reviewed

Documentation

• docs/src/content/docs/introduction/how-they-work.mdx — ⚠️ Needs Minor Work (Best Practices paragraph)
• docs/src/content/docs/guides/memoryops.md — ✅ Professional

CLI Commands

• gh aw audit — ✅ Professional
• gh aw logs — ✅ Professional

Workflow Messages

• .github/workflows/design-decision-gate.md — ✅ Professional
• .github/workflows/dev-hawk.md — ⚠️ Needs Minor Work (themed language in failure messages)

Validation Code

• pkg/workflow/docker_validation_graceful_test.go — ✅ Professional

---

Metrics

• Files Analyzed: 7
• Quality Distribution:
  • ✅ Professional: 5
  • ⚠️ Needs Minor Work: 2
  • ❌ Needs Significant Work: 0

---

🎯 Actionable Tasks

Here are 2 targeted improvement tasks, each affecting a single file:

---

Task 1: Clarify Failure Messages in Dev Hawk Workflow

File to Modify: .github/workflows/dev-hawk.md

Current Experience

Lines 31–34 use hawk-themed language for lifecycle messages. The run-failure message reads:

"🦅 Hawk down! [{workflow_name}]({run_url}) {status}. The skies grow quiet..."

Quality Issue

Design Principle: Clarity and Precision

When a workflow fails, the notification message is the first signal a developer sees. Metaphorical language delays comprehension during incident triage. The message structure is correct ({status} is present) but hidden behind theatrical text.

Proposed Improvement

Before:

run-started: "🦅 Dev Hawk circles the sky! [{workflow_name}]({run_url}) is monitoring this {event_type} from above..."
run-success: "🦅 Hawk eyes report! [{workflow_name}]({run_url}) has completed reconnaissance. Intel delivered! 🎯"
run-failure: "🦅 Hawk down! [{workflow_name}]({run_url}) {status}. The skies grow quiet..."

After:

run-started: "🦅 [{workflow_name}]({run_url}) is monitoring this {event_type}..."
run-success: "🦅 [{workflow_name}]({run_url}) completed monitoring successfully."
run-failure: "🦅 [{workflow_name}]({run_url}) {status} during monitoring."

Why This Matters

• User Impact: Developers can parse failure notifications immediately without mental translation
• Quality Factor: Professional Communication — clear status over themed metaphor
• Frequency: Every workflow run on copilot/* branches posts one of these messages to a PR

Success Criteria

• Changes made to .github/workflows/dev-hawk.md only
• Run-failure message leads with the link and status, not themed text
• Quality rating improves from ⚠️ to ✅

Scope Constraint

• Single file only: .github/workflows/dev-hawk.md
• No changes to other files required

---

Task 2: Convert Best Practices Paragraph to Scannable List

File to Modify: docs/src/content/docs/introduction/how-they-work.mdx

Current Experience

Lines 67–68: the "Best Practices" section contains one dense paragraph with 5–6 distinct recommendations, making it difficult to scan quickly.

Quality Issue

Design Principle: Efficiency and Productivity

This is the closing section of the "How They Work" introduction page — the most-visited page for new users. A single paragraph obscures actionable guidance that should stand out.

Proposed Improvement

Before:

## Best Practices

Start simple and iterate with clear, specific instructions. Test workflows using `gh aw compile --watch` and `gh aw run`, monitor costs with `gh aw logs`, and review AI-generated content before merging. Use [`safe outputs`](/gh-aw/reference/safe-outputs/) (pre-approved GitHub operations) for controlled creation of issues, comments, and PRs.

After:

## Best Practices

- **Start simple**: Write clear, specific instructions and iterate
- **Compile and test**: Use `gh aw compile --watch` and `gh aw run` during development
- **Monitor costs**: Use `gh aw logs` to track token usage and costs
- **Review AI output**: Always review AI-generated content before merging
- **Control writes**: Use [`safe outputs`](/gh-aw/reference/safe-outputs/) for controlled creation of issues, comments, and PRs

Why This Matters

• User Impact: New users can scan to the specific practice they need, especially during onboarding
• Quality Factor: Visual Hierarchy — lists for parallel items
• Frequency: This is the introduction page; it is one of the highest-traffic pages for new adopters

Success Criteria

• Changes made to docs/src/content/docs/introduction/how-they-work.mdx only
• Best Practices section uses a bullet list with bold labels
• Quality rating improves from ⚠️ to ✅

Scope Constraint

• Single file only: docs/src/content/docs/introduction/how-they-work.mdx
• No changes to other files required

---

References:

• §25214712797

📊 User experience analysis by Delight · ● 1.1M · ◷

• expires on May 4, 2026, 12:50 PM UTC

[github-actions[bot]]

This discussion has been marked as outdated by Delight.

A newer discussion is available at Discussion #29775.