[delight] UX Analysis: 2026-06-30 — repository-package-manifest.md warning visibility + smoke-agent-all-none failure message

[github-actions[bot]]

User Experience Analysis Report — 2026-06-30

Executive Summary

Today's analysis focused on:

• 2 documentation files
• 1 CLI command (run)
• 2 workflow message configurations
• 1 validation file

Overall Quality: Professional — the codebase maintains a high standard across most areas.

Key Finding: A critical package requirement (README.md must exist) is buried in prose with no visual emphasis in repository-package-manifest.md, increasing the risk of silent validation failures for package authors.

---

Quality Highlights ✅

Highlight 1: Exemplary Error Messages in Expression Validation

• File: pkg/workflow/expression_syntax_validation.go
• What works well: Every validation error includes a clear problem statement, precise position context (at position N), and a concrete remedy sentence. This is a model for the rest of the codebase.
• Reference: "Ensure all '}}' have a corresponding opening '${{'. Check for typos or missing opening braces."`

Highlight 2: Practical Prompt Examples in Skills Documentation

• File: docs/src/content/docs/reference/custom-agent-for-aw.mdx
• What works well: Each task section (create, update, upgrade, import, debug) includes a verbatim copyable prompt reflecting actual enterprise use cases. The file avoids abstract descriptions and shows exact input a user would type.

---

Improvement Opportunities 💡

High Priority

Opportunity 1: Critical Constraint Buried Without Visual Emphasis

• File: docs/src/content/docs/reference/repository-package-manifest.md
• Current State (Documentation section, last sentence): "Missing README.md causes package validation to fail."
• Issue: This is a hard validation requirement — omitting README.md breaks gh aw add — but it reads as a casual observation at the end of a paragraph. Users skimming the page will miss it.
• User Impact: Package authors spend time debugging opaque validation failures that could be prevented by a clearly visible constraint.
• Suggested Change: Promote the constraint to a GitHub-flavored Markdown warning admonition.
• Design Principle: Clarity and Precision — explicit requirements must be visually distinct from explanatory prose.

Medium Priority

Opportunity 2: Generic Failure Message in Smoke Test Workflow

• File: .github/workflows/smoke-agent-all-none.md
• Current State: run-failure: "❌ [{workflow_name}]({run_url}) {status}. Check the logs for details."
• Issue: "Check the logs for details." is a dead-end instruction. This pattern has been flagged previously across smoke-pi.md, smoke-gemini.md, and impeccable-skills-reviewer.md.
• User Impact: Engineers investigating test failures must take an extra navigation step instead of clicking directly to the run.
• Suggested Change: Replace the static instruction with a hyperlinked call to action using {run_url}.
• Design Principle: Efficiency and Productivity — provide direct paths to outcomes.

---

Files Reviewed

Documentation

• docs/src/content/docs/reference/custom-agent-for-aw.mdx — Rating: ✅ Professional
• docs/src/content/docs/reference/repository-package-manifest.md — Rating: ⚠️ Needs Minor Work

CLI Commands

• gh aw run (source: cmd/gh-aw/main.go) — Rating: ✅ Professional

Workflow Messages

• .github/workflows/shared/ollama-threat-scan.md — Rating: ✅ Professional
• .github/workflows/smoke-agent-all-none.md — Rating: ⚠️ Needs Minor Work

Validation Code

• pkg/workflow/expression_syntax_validation.go — Rating: ✅ Professional

Metrics

• Files Analyzed: 5
• Quality Distribution: ✅ Professional: 4 · ⚠️ Needs Minor Work: 2 · ❌ Needs Significant Work: 0

---

🎯 Actionable Tasks

Here are 2 targeted improvement tasks, each affecting a single file:

Task 1: Surface README.md Requirement as a Warning — repository-package-manifest.md

File to Modify: docs/src/content/docs/reference/repository-package-manifest.md

Current Experience

The Documentation section ends with a buried validation constraint:

The manifest does not support a `docs` field.
Missing `README.md` causes package validation to fail.

The critical failure condition is the last sentence in a prose block — invisible to users scanning the page.

Quality Issue

Design Principle: Clarity and Precision — explicit requirements must be visually distinct from explanatory prose.

Enterprise users reading reference documentation scan headings, tables, and callouts. A critical validation constraint presented as a trailing prose sentence violates the reader's expectation that such requirements are highlighted.

Proposed Improvement

Before:

The manifest does not support a `docs` field.
Missing `README.md` causes package validation to fail.

After:

The manifest does not support a `docs` field.

> [!WARNING]
> `README.md` is required. `gh aw add` fails validation when the package root does not contain a `README.md` file.

Why This Matters

• User Impact: Package authors see the constraint before creating a package, preventing a frustrating silent validation failure during gh aw add.
• Quality Factor: Visual hierarchy — warning callouts are processed before prose in scanning behavior.
• Frequency: Every user authoring a new installable package reads this reference page.

Success Criteria

• Changes made to docs/src/content/docs/reference/repository-package-manifest.md only
• > [!WARNING] admonition renders correctly in GitHub-flavored Markdown
• Buried trailing sentence removed (no duplicate information)
• Quality rating improves from ⚠️ to ✅

Scope Constraint: Single file only — docs/src/content/docs/reference/repository-package-manifest.md

---

Task 2: Add Hyperlinked Log Action to run-failure — smoke-agent-all-none.md

File to Modify: .github/workflows/smoke-agent-all-none.md

Current Experience

run-failure: "❌ [{workflow_name}]({run_url}) {status}. Check the logs for details."

"Check the logs for details." is a static instruction with no link, requiring a manual navigation step.

Quality Issue

Design Principle: Efficiency and Productivity — provide direct paths to outcomes.

This is the fourth smoke workflow flagged with this pattern (after smoke-pi.md, smoke-gemini.md, impeccable-skills-reviewer.md).

Proposed Improvement

Before:

run-failure: "❌ [{workflow_name}]({run_url}) {status}. Check the logs for details."

After:

run-failure: "❌ [{workflow_name}]({run_url}) {status}. [View run logs]({run_url}) for details."

Why This Matters

• User Impact: Engineers investigating guard policy test failures get a direct clickable link to the run.
• Quality Factor: Efficiency — reduces friction in the failure investigation path.
• Frequency: Every failing smoke test run generates this message for reviewers.

Success Criteria

• Changes made to .github/workflows/smoke-agent-all-none.md only
• {run_url} token used correctly
• "Check the logs for details." replaced with no duplicate text
• Quality rating improves from ⚠️ to ✅

Scope Constraint: Single file only — .github/workflows/smoke-agent-all-none.md

📊 User experience analysis by Delight · 45.1 AIC · ⌖ 10.4 AIC · ⊞ 1.9K · ◷

• expires on Jul 3, 2026, 7:38 AM UTC-08:00