[delight] User Experience Analysis Report - 2026-08-30

[github-actions[bot]]

Executive Summary

Today's analysis focused on:

• 2 documentation file(s): docs/src/content/docs/reference/triggering-ci.mdx, docs/src/content/docs/blog/2026-03-30-weekly-update.md
• 0 CLI command(s) — gh-aw binary not available in this environment, so CLI help text could not be sampled this run
• 2 workflow message configuration(s): stale-pr-cleanup.md, breaking-change-checker.md
• 1 validation file: pkg/cli/run_workflow_validation.go

Overall Quality: Generally professional, with one improvement opportunity in AI-generated failure messaging.

Key Finding: The breaking-change-checker.md workflow's run-failure message ends with a vague, unresolved statement ("Compatibility status unknown...") that gives enterprise users no indication of what happened or what to do next.

Quality Highlights ✅

Example 1: Excellent input validation error messages

• File: pkg/cli/run_workflow_validation.go
• What works well: validateWorkflowInputs produces errors that state exactly what's missing, suggests likely-intended input names via fuzzy matching, lists all valid inputs with descriptions/defaults, and appends copy-pasteable gh aw run command examples.
• Quote/Reference: "'%s' -> did you mean '%s'?" and the generated "To set required inputs, use:\n gh aw run %s -F %s=<value>" block (lines ~257–266). This is a model example of clear problem statement + actionable solution + example.

Example 2: Well-organized CI trigger documentation

• File: docs/src/content/docs/reference/triggering-ci.mdx
• What works well: Presents the quickest fix first (magic secret), then progressively more advanced options (PAT, GitHub App, full token override), each with copy-pasteable commands and a troubleshooting tip ("If CI still does not start, inspect workflow runs for the empty-commit SHA...").

Improvement Opportunities 💡

High Priority

Opportunity 1: Vague run-failure message - Single File Improvement

• File: .github/workflows/breaking-change-checker.md
• Current State: Line 336: run-failure: "🔬 Analysis interrupted! [{workflow_name}]({run_url}) {status}. Compatibility status unknown..."
• Issue: The trailing "Compatibility status unknown..." adds no actionable information and ends on an unresolved, slightly unprofessional note (informal ellipsis, no next step).
• User Impact: When a compatibility check fails, enterprise users need to know where to look (the run log) and what happened next — not just that the result is "unknown."
• Suggested Change: Replace the vague trailing clause with a concrete pointer to the run logs, consistent with the tone of the run-success message in the same file.
• Design Principle: Trust and Reliability — error messages should be actionable, not just an acknowledgment of failure.

Medium Priority

Opportunity 2: Failure message lacks next-step guidance - Single File Improvement

• File: .github/workflows/stale-pr-cleanup.md
• Current State: Line 40: run-failure: "❌ Stale PR cleanup failed! [{workflow_name}]({run_url}) {status}. Some PRs may not be processed."
• Issue: States the consequence ("some PRs may not be processed") but gives no direction on what a maintainer should do (e.g., check the run log, re-run manually).
• User Impact: Maintainers seeing this notification have to guess whether to intervene or wait for the next scheduled run.
• Suggested Change: Add a brief pointer to the run log or a suggestion to re-run via workflow_dispatch.
• Design Principle: Trust and Reliability / Professional Communication — actionable status over passive statement.

Files Reviewed

Documentation

• docs/src/content/docs/reference/triggering-ci.mdx - Rating: ✅
• docs/src/content/docs/blog/2026-03-30-weekly-update.md - Rating: ✅

CLI Commands

• (not sampled — gh-aw binary unavailable in this environment)

Workflow Messages

• stale-pr-cleanup.md - Rating: ⚠️
• breaking-change-checker.md - Rating: ⚠️

Validation Code

• pkg/cli/run_workflow_validation.go - Rating: ✅

Metrics

• Files Analyzed: 5
• Quality Distribution:
  • ✅ Professional: 3
  • ⚠️ Needs Minor Work: 2
  • ❌ Needs Significant Work: 0

🎯 Actionable Tasks

Here are 2 targeted improvement tasks, each affecting a single file:

Task 1: Improve run-failure message clarity - .github/workflows/breaking-change-checker.md

File to Modify: .github/workflows/breaking-change-checker.md

Current Experience

Line 336 defines the failure message shown to users when the Breaking Change Checker workflow fails:

run-failure: "🔬 Analysis interrupted! [{workflow_name}]({run_url}) {status}. Compatibility status unknown..."

The message tells the reader the analysis was interrupted, but ends on an unresolved, informal note without directing them anywhere.

Quality Issue

Design Principle: Trust and Reliability

The trailing ellipsis and "status unknown" phrasing reads as unfinished/informal rather than a clear, professional status report. It gives no actionable next step (e.g., check logs, re-run).

Proposed Improvement

Update the run-failure message to point to the run and suggest a concrete next step.

Before:

    run-failure: "🔬 Analysis interrupted! [{workflow_name}]({run_url}) {status}. Compatibility status unknown..."

After:

    run-failure: "🔬 Analysis interrupted! [{workflow_name}]({run_url}) {status}. Check the run log for details, or re-run the workflow to retry the compatibility check."

Why This Matters

• User Impact: Users immediately know to check the run log rather than wondering what "unknown" means.
• Quality Factor: Trust and Reliability — clear, actionable failure communication.
• Frequency: Every time this workflow fails (on PR/push triggers), so this is seen fairly often.

Success Criteria

• Changes made to .github/workflows/breaking-change-checker.md only
• run-failure message includes a concrete next step
• Quality rating improves from ⚠️ to ✅

Scope Constraint

• Single file only: .github/workflows/breaking-change-checker.md
• No changes to other files required
• Can be completed independently

---

Task 2: Add next-step guidance to run-failure message - .github/workflows/stale-pr-cleanup.md

File to Modify: .github/workflows/stale-pr-cleanup.md

Current Experience

Line 40 defines the failure message for the Stale PR Cleanup workflow:

run-failure: "❌ Stale PR cleanup failed! [{workflow_name}]({run_url}) {status}. Some PRs may not be processed."

It states the consequence but not what a maintainer should do about it.

Quality Issue

Design Principle: Trust and Reliability / Professional Communication

Enterprise users receiving this notification have no guidance on whether to wait for the next scheduled run or intervene manually.

Proposed Improvement

Before:

    run-failure: "❌ Stale PR cleanup failed! [{workflow_name}]({run_url}) {status}. Some PRs may not be processed."

After:

    run-failure: "❌ Stale PR cleanup failed! [{workflow_name}]({run_url}) {status}. Some PRs may not be processed until the next scheduled run — check the run log, or trigger a manual re-run via workflow_dispatch."

Why This Matters

• User Impact: Maintainers know exactly what to do next instead of guessing.
• Quality Factor: Professional Communication — actionable guidance over passive statement.
• Frequency: Occurs on each failed weekly run; low frequency but high impact for backlog accuracy.

Success Criteria

• Changes made to .github/workflows/stale-pr-cleanup.md only
• run-failure message includes concrete next-step guidance
• Quality rating improves from ⚠️ to ✅

Scope Constraint

• Single file only: .github/workflows/stale-pr-cleanup.md
• No changes to other files required
• Can be completed independently

📊 User experience analysis by Delight · copilot · auto · 29.6 AIC · ⌖ 7.14 AIC · ⊞ 12.3K · ◷

• expires on Sep 2, 2026, 6:47 AM UTC-08:00