[delight] UX Analysis Report – 2026-04-27

[github-actions[bot]]

Executive Summary

Today's analysis covered:

• 2 documentation files: docs/src/content/docs/setup/quick-start.mdx, docs/src/content/docs/reference/engines.md
• 1 CLI command: gh aw init (pkg/cli/init_command.go)
• 1 validation file: pkg/workflow/engine_validation.go

Overall Quality: Largely professional. The codebase has strong documentation habits, good error message patterns with actionable guidance and "did you mean" suggestions. Two targeted improvements were identified.

Key Finding: The gh aw init command's help text opens with an incomplete description that omits its most impactful default behavior (MCP configuration), creating a first-impression accuracy gap for enterprise users evaluating the command.

---

Quality Highlights ✅

1. Engine Validation Error Messages

• File: pkg/workflow/engine_validation.go
• What works well: Error messages are exemplary – they state the problem precisely, include inline YAML examples, point to documentation URLs, and offer "did you mean?" suggestions. This is enterprise-grade UX.
• Reference: "inline engine definition references unknown runtime.id: %s. Known runtime IDs are: %s.\n\nDid you mean: %s?\n\nExample:\nengine:\n runtime:\n id: codex\n\nSee: %s"

2. Engines Reference Documentation

• File: docs/src/content/docs/reference/engines.md
• What works well: Clear comparison tables, per-engine YAML examples, security model section with plain-language explanations. Well-organized with appropriate use of > [!WARNING] and > [!NOTE] callouts.

---

Improvement Opportunities 💡

High Priority

Opportunity 1: Incomplete Opening Description in init Help Text – Single File Improvement

• File: pkg/cli/init_command.go
• Current State (line 18): "Initialize the repository for agentic workflows by configuring .gitattributes and creating the dispatcher agent file."
• Issue: The summary line describes only 2 of 5+ default operations. By default, init also creates copilot-setup-steps.yml and .github/mcp.json (arguably the most important outputs for Copilot users). A user running gh aw init --help gets a misleading picture of what the command actually does.
• User Impact: Enterprise users doing due diligence before running a command expect the Short description to give an accurate, complete summary. Finding that the command did "more than described" undermines trust.
• Suggested Change: Update the Short field to cover the primary outcomes.
• Design Principle: Trust and Reliability – accurate information

Before (line 17):

Short: "Initialize the repository for agentic workflows",
Long: `Initialize the repository for agentic workflows by configuring .gitattributes and creating the dispatcher agent file.

After:

Short: "Initialize the repository for agentic workflows",
Long: `Initialize the repository for agentic workflows. Configures .gitattributes, creates the dispatcher agent file, and sets up GitHub Copilot MCP integration by default.

---

Medium Priority

Opportunity 2: Broken Absolute URL in Quick Start – Single File Improvement

• File: docs/src/content/docs/setup/quick-start.mdx
• Current State (line 123): [Peli's Agent Factory](https://github.github.com/gh-aw/blog/2026-01-12-welcome-to-pelis-agent-factory/)
• Issue: All other navigation links in the file use relative paths (e.g., /gh-aw/setup/creating-workflows/). The github.github.com domain may not resolve correctly outside the deployment context, and the pattern is inconsistent with the rest of the file. The "What's Next?" section is the last thing a first-time user sees — a dead or inconsistent link here creates a poor finishing impression.
• User Impact: First-time users who complete the quick-start and click the "What's next?" link may hit a broken page, ending a positive onboarding moment on a frustrating note.
• Suggested Change: Replace the absolute URL with the relative path format used throughout the file.
• Design Principle: Consistency and Trust

Before (line 123):

[Peli's Agent Factory](https://github.github.com/gh-aw/blog/2026-01-12-welcome-to-pelis-agent-factory/)

After:

[Peli's Agent Factory](/gh-aw/blog/2026-01-12-welcome-to-pelis-agent-factory/)

---

Files Reviewed

Documentation

• docs/src/content/docs/setup/quick-start.mdx – Rating: ⚠️ Needs Minor Work (broken link in "What's Next?")
• docs/src/content/docs/reference/engines.md – Rating: ✅ Professional

CLI Commands

• gh aw init (pkg/cli/init_command.go) – Rating: ⚠️ Needs Minor Work (incomplete opening summary)

Validation Code

• pkg/workflow/engine_validation.go – Rating: ✅ Professional

---

Metrics

• Files Analyzed: 4
• Quality Distribution:
  • ✅ Professional: 2
  • ⚠️ Needs Minor Work: 2
  • ❌ Needs Significant Work: 0

---

🎯 Actionable Tasks

Here are 2 targeted improvement tasks, each affecting a single file:

Task 1: Improve init Help Text Opening Description

File to Modify: pkg/cli/init_command.go

Current Experience

Line 18 opens the Long description with:

"Initialize the repository for agentic workflows by configuring .gitattributes and creating the dispatcher agent file."

This describes only 2 of the 5+ default operations. The MCP integration (.github/mcp.json, copilot-setup-steps.yml) — the most impactful output for most users — is buried several paragraphs down.

Quality Issue

Design Principle: Trust and Reliability – accurate information at first glance

Enterprise users read the opening line of help text to understand scope before running a command. An incomplete description of default behavior violates the accuracy contract.

Proposed Improvement

Change line 18 from:

Initialize the repository for agentic workflows by configuring .gitattributes and creating the dispatcher agent file.

To:

Initialize the repository for agentic workflows. Configures .gitattributes, creates the dispatcher agent file, and by default sets up GitHub Copilot MCP integration (copilot-setup-steps.yml and .github/mcp.json).

Why This Matters

• User Impact: Users get an accurate overview of what init will do before running it
• Quality Factor: Accuracy / Trust
• Frequency: Every user running gh aw init --help for the first time

Success Criteria

• Changes made to pkg/cli/init_command.go only
• Opening Long description accurately reflects all default operations
• Quality rating improves from ⚠️ to ✅

Scope Constraint

• Single file only: pkg/cli/init_command.go

---

Task 2: Fix Inconsistent URL Format in Quick Start

File to Modify: docs/src/content/docs/setup/quick-start.mdx

Current Experience

Line 123 contains:

[Peli's Agent Factory](https://github.github.com/gh-aw/blog/2026-01-12-welcome-to-pelis-agent-factory/)

All other cross-references in this file use relative paths (e.g., /gh-aw/setup/creating-workflows/, /gh-aw/introduction/how-they-work/). This absolute URL uses a different domain pattern and is the final link in the onboarding guide.

Quality Issue

Design Principle: Consistency – uniform navigation patterns reduce cognitive friction and avoid broken links

Proposed Improvement

[Peli's Agent Factory](/gh-aw/blog/2026-01-12-welcome-to-pelis-agent-factory/)

Why This Matters

• User Impact: The "What's Next?" section is the last thing a first-time user sees. A broken or inconsistent link here ends onboarding on a frustrating note.
• Quality Factor: Consistency, Reliability
• Frequency: Every user completing the Quick Start guide

Success Criteria

• Changes made to docs/src/content/docs/setup/quick-start.mdx only
• Link uses relative path format consistent with the rest of the file
• Quality rating improves from ⚠️ to ✅

Scope Constraint

• Single file only: docs/src/content/docs/setup/quick-start.mdx

---

References: §24994595719

📊 User experience analysis by Delight · ● 1M · ◷

• expires on Apr 30, 2026, 12:24 PM UTC

[github-actions[bot]]

This discussion has been marked as outdated by Delight.

A newer discussion is available at Discussion #29305.