[delight] User Experience Analysis Report - 2026-08-22

[github-actions[bot]]

Executive Summary

Today's analysis focused on:

• 2 documentation files (blog posts)
• 0 CLI commands (build unavailable in sandbox — Go toolchain download blocked by network policy; see Missing Tool note below)
• 2 workflow message configurations (smoke-test workflows)
• 1 validation file

Overall Quality: Professional overall, with one concrete opportunity to make a security-critical compiler error message more actionable.

Key Finding: The persist-credentials checkout validation error in pkg/workflow/imported_steps_validation.go is already well-written but omits a copy-pasteable YAML example, forcing users to guess exact syntax when fixing a strict-mode compile failure.

Quality Highlights ✅

Example 1: Well-structured blog narrative

• File: docs/src/content/docs/blog/2026-01-13-meet-the-workflows-continuous-refactoring.md
• What works well: Clear structure (problem → mechanism → concrete metrics → call-to-action), backed by real PR/issue links and merge-rate statistics ("112 merged PRs out of 142 proposed (79% merge rate)"). Copy-paste-ready gh aw add-wizard commands make the "Using These Workflows" section immediately actionable.
• Quote/Reference: Lines 45, 71-73 — concrete evidence plus runnable commands, exactly the pattern enterprise readers need to evaluate and adopt a workflow quickly.

Example 2: Data-driven storytelling

• File: docs/src/content/docs/blog/2026-06-02-agent-of-the-day.md
• What works well: Presents a table of token-usage trends with clear before/after framing and links every claim to a run ID or issue number, giving readers verifiable provenance instead of vague claims.

Improvement Opportunities 💡

High Priority

Opportunity 1: Add a fix example to the persist-credentials compile error - Single File Improvement

• File: pkg/workflow/imported_steps_validation.go
• Current State: Lines 95-101 build the error message users see when strict: true compilation fails due to a checkout step missing persist-credentials: false:

  msg := fmt.Sprintf(
      "actions/checkout step(s) without 'persist-credentials: false' detected in the agent job: %s. "+
          "Without this setting the git token is stored in .git/config and leaked to the agent. "+
          "Add 'persist-credentials: false' to the 'with:' block of each checkout step. "+
          "See: https://github.github.com/gh-aw/reference/steps/",
      strings.Join(offendingStepNames, ", "),
  )

• Issue: The message correctly explains what is wrong and why it matters, but tells the user to "add ... to the 'with:' block" without showing what that block should look like. Users unfamiliar with the checkout action's with: syntax must open external docs mid-compile-failure to resolve a security block.
• User Impact: This is a strict-mode compiler error that stops a build. Every second spent looking up correct YAML syntax is friction during an already-interrupting failure. An inline example lets users copy-paste the fix and recompile immediately.
• Suggested Change: Append a minimal YAML snippet to the message, e.g.:

  msg := fmt.Sprintf(
      "actions/checkout step(s) without 'persist-credentials: false' detected in the agent job: %s. "+
          "Without this setting the git token is stored in .git/config and leaked to the agent. "+
          "Add 'persist-credentials: false' to the 'with:' block, for example:\n"+
          "  - uses: actions/checkout@v4\n"+
          "    with:\n"+
          "      persist-credentials: false\n"+
          "See: https://github.github.com/gh-aw/reference/steps/",
      strings.Join(offendingStepNames, ", "),
  )

• Design Principle: Trust and Reliability — "Clear error messages with actionable solutions."

Medium Priority

Opportunity 2: Vague failure message in smoke-aider workflow - Single File Improvement

• File: .github/workflows/smoke-aider.md
• Current State: Line 52: run-failure: "⚠️ [{workflow_name}]({run_url}) {status}. Aider encountered unexpected challenges..."
• Issue: "encountered unexpected challenges" is vague and doesn't direct the reader to logs or next steps, unlike the sibling smoke-agent-all-merged.md workflow, whose failure message ("Check the logs for details.") is more actionable.
• User Impact: Low-medium — this is a private smoke-test workflow, but the pattern is copied into other workflow templates; inconsistent phrasing across smoke workflows undermines a predictable, professional voice.
• Suggested Change: run-failure: "⚠️ [{workflow_name}]({run_url}) {status}. Check the logs for details." for consistency with other smoke workflows.
• Design Principle: Professional Communication — "Consistent voice."

Files Reviewed

Documentation

• docs/src/content/docs/blog/2026-01-13-meet-the-workflows-continuous-refactoring.md - Rating: ✅
• docs/src/content/docs/blog/2026-06-02-agent-of-the-day.md - Rating: ✅

CLI Commands

• Not analyzed this run (build environment blocked outbound Go toolchain download)

Workflow Messages

• .github/workflows/smoke-agent-all-merged.md - Rating: ✅
• .github/workflows/smoke-aider.md - Rating: ⚠️

Validation Code

• pkg/workflow/imported_steps_validation.go - Rating: ⚠️

Metrics

• Files Analyzed: 5
• Quality Distribution:
  • ✅ Professional: 3
  • ⚠️ Needs Minor Work: 2
  • ❌ Needs Significant Work: 0

🎯 Actionable Tasks

Task 1: Add inline YAML fix example to persist-credentials compile error

File to Modify: pkg/workflow/imported_steps_validation.go

Current Experience

The strict-mode compile error (lines 95-101) tells users to add persist-credentials: false to a checkout step's with: block but does not show the YAML syntax inline, requiring a trip to external docs during a build failure.

Quality Issue

Design Principle: Trust and Reliability

Users hitting this strict-mode failure must interrupt their workflow to look up exact YAML syntax for a one-line fix, adding avoidable friction to a security-motivated compile gate.

Proposed Improvement

Append a minimal, copy-pasteable YAML example directly in the error message.

Before:

msg := fmt.Sprintf(
    "actions/checkout step(s) without 'persist-credentials: false' detected in the agent job: %s. "+
        "Without this setting the git token is stored in .git/config and leaked to the agent. "+
        "Add 'persist-credentials: false' to the 'with:' block of each checkout step. "+
        "See: https://github.github.com/gh-aw/reference/steps/",
    strings.Join(offendingStepNames, ", "),
)

After:

msg := fmt.Sprintf(
    "actions/checkout step(s) without 'persist-credentials: false' detected in the agent job: %s. "+
        "Without this setting the git token is stored in .git/config and leaked to the agent. "+
        "Add 'persist-credentials: false' to the 'with:' block, for example:\n"+
        "  - uses: actions/checkout@v4\n"+
        "    with:\n"+
        "      persist-credentials: false\n"+
        "See: https://github.github.com/gh-aw/reference/steps/",
    strings.Join(offendingStepNames, ", "),
)

Why This Matters

• User Impact: Removes a context-switch during a build failure by giving the exact fix inline.
• Quality Factor: Actionable error messages, reduced cognitive load.
• Frequency: Triggered on every strict-mode compile of a workflow with an unguarded checkout step — a common onboarding mistake.

Success Criteria

• Change made to pkg/workflow/imported_steps_validation.go only
• Error message includes inline YAML example
• Existing unit tests in pkg/workflow/imported_steps_validation_test.go that assert on error message substrings (e.g. errorMsg: "strict mode: actions/checkout step(s) without 'persist-credentials: false'") still pass, since the added text is appended rather than replacing the existing substrings
• Quality rating improves from ⚠️ to ✅

Scope Constraint

• Single file only: pkg/workflow/imported_steps_validation.go
• No changes to other files required
• Can be completed independently

---

Task 2: Align smoke-aider failure message with sibling smoke workflows

File to Modify: .github/workflows/smoke-aider.md

Current Experience

Line 52 reads: run-failure: "⚠️ [{workflow_name}]({run_url}) {status}. Aider encountered unexpected challenges..." — vague and gives no next step.

Quality Issue

Design Principle: Professional Communication (Consistent voice)

Sibling smoke-test workflows (e.g. smoke-agent-all-merged.md) use the clearer, actionable pattern "Check the logs for details." The inconsistency undermines a predictable voice across the smoke-test suite.

Proposed Improvement

Before:

run-failure: "⚠️ [{workflow_name}]({run_url}) {status}. Aider encountered unexpected challenges..."

After:

run-failure: "⚠️ [{workflow_name}]({run_url}) {status}. Check the logs for details."

Why This Matters

• User Impact: Gives a concrete next step (check logs) instead of a vague, flavor-text phrase.
• Quality Factor: Consistency and actionability of status messages.
• Frequency: Shown on every failed run of this smoke-test workflow.

Success Criteria

• Change made to .github/workflows/smoke-aider.md only
• run-failure message matches the "Check the logs for details." pattern used elsewhere
• Quality rating improves from ⚠️ to ✅

Scope Constraint

• Single file only: .github/workflows/smoke-aider.md
• No changes to other files required
• Can be completed independently

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• storage.googleapis.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "storage.googleapis.com"

See Network Configuration for more information.

📊 User experience analysis by Delight · auto · 32.8 AIC · ⌖ 5.72 AIC · ⊞ 12.3K · ◷

• expires on Aug 25, 2026, 6:51 AM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Delight.

A newer discussion is available at Discussion #55104.