[delight] User Experience Analysis Report - 2026-08-02

[github-actions[bot]]

Executive Summary

Today's analysis focused on:

• 2 documentation file(s)
• 0 CLI command(s) (deferred — build tooling unavailable in sandbox)
• 2 workflow message configuration(s)
• 1 validation file(s)

Overall Quality: Professional overall, with one high-impact documentation defect found.

Key Finding: schedule-syntax.md — the reference doc that teaches users the schedule syntax — repeatedly shows YAML code blocks with duplicate schedule: keys under a single on: map. This is invalid YAML; only the last key survives on parse, silently discarding all preceding examples if copy-pasted as shown.

Quality Highlights ✅

Example 1: Clear typo-suggestion error messages

• File: pkg/workflow/event_validation.go
• What works well: The ValidateEventTypes function gives precise, actionable "Did you mean" suggestions using Levenshtein distance, and thoughtfully avoids false positives for new/unknown GitHub event names by only flagging close matches. Well documented with a clear file-level comment explaining the design rationale.
• Quote/Reference: "unknown event type %q in 'on:' section.\n\nDid you mean: %s?\n\nValid event types include: %s\n\nSee: https://docs.github.com/..." (line 168-173)

Example 2: Quick Reference table in schedule-syntax.md

• File: docs/src/content/docs/reference/schedule-syntax.md
• What works well: The "Quick Reference" table (lines 120-141) is an excellent scannable summary of fuzzy vs. fixed schedule patterns before diving into detail sections — respects the reader's time.

Improvement Opportunities 💡

High Priority

Opportunity 1: Invalid YAML duplicate keys in schedule-syntax.md

• File: docs/src/content/docs/reference/schedule-syntax.md
• Current State: Multiple fenced YAML code blocks (e.g. lines 54-56, 66-74, 82-84, 94-98, 106-109, 117-119, 125-127, 157-161) show several schedule: ... lines stacked directly under one on: map, like:

  on:
    schedule: daily
    schedule: daily on weekdays  # Monday-Friday only

• Issue: In real YAML, a mapping cannot have two identical keys — only the last schedule: line takes effect (or a strict parser rejects the document outright). The doc presents these as if all variants apply simultaneously, but they are meant to be mutually exclusive, alternative single-line examples.
• User Impact: An enterprise user copy-pasting one of these blocks into a workflow file gets either a parser error or silently loses all but the last schedule — with no indication anything went wrong, undermining trust in the documentation's accuracy.
• Suggested Change: Split each stacked example into separate one-line on: blocks (or comment out all but one active line with clear "// Option A / Option B" style annotations), consistent with how the "Fixed Schedules" and "Multiple Schedules" sections already correctly present single valid blocks.
• Design Principle: Trust and Reliability — documentation should be exactly copy-paste-safe.

Files Reviewed

Documentation

• docs/src/content/docs/blog/2026-01-13-meet-the-workflows-security-compliance.md - Rating: ✅
• docs/src/content/docs/reference/schedule-syntax.md - Rating: ⚠️ (excellent structure, but invalid YAML in examples)

Workflow Messages

• .github/workflows/daily-byok-ollama-test.md - Rating: ✅ (clear, concise status messages)
• .github/workflows/smoke-copilot-aoai-apikey.md - Rating: ⚠️ (playful "BREAKING NEWS" tone is fun but inconsistent with enterprise messaging; previously flagged in memory)

Validation Code

• pkg/workflow/event_validation.go (tested via event_validation_test.go) - Rating: ✅

Metrics

• Files Analyzed: 5
• Quality Distribution:
  • ✅ Professional: 3
  • ⚠️ Needs Minor Work: 2
  • ❌ Needs Significant Work: 0

🎯 Actionable Tasks

Task 1: Fix Invalid YAML Examples in Schedule Syntax Reference

File to Modify: docs/src/content/docs/reference/schedule-syntax.md

Current Experience

Several code blocks stack multiple schedule: lines under one on: key to show variants, e.g. (lines 54-56):

on:
  schedule: daily
  schedule: daily on weekdays  # Monday-Friday only

This pattern repeats in the "Daily with Time Constraints" (66-74), "Hourly Schedules" (82-84), "Interval Schedules" (94-98), "Weekly Schedules" (106-109), "Weekly with Time Constraints" (117-119), "Bi-weekly and Tri-weekly Schedules" (125-127), and "UTC Offset Support" (157-161) sections.

Quality Issue

Design Principle: Trust and Reliability — documentation must be accurate and safe to copy-paste.

YAML mappings cannot contain duplicate keys. A user who copies one of these blocks verbatim either triggers a parse error in strict YAML parsers or silently loses all schedule variants but the last one defined. Nothing in the surrounding text warns that only one line should be used at a time.

Proposed Improvement

Replace each stacked block with either (a) separate single-line on: blocks per variant, or (b) one block with only the first variant active and the rest shown as YAML comments demonstrating alternate syntax, e.g.:

Before:

on:
  schedule: daily
  schedule: daily on weekdays  # Monday-Friday only

After:

on:
  schedule: daily
# Alternative: restrict to weekdays only
# on:
#   schedule: daily on weekdays

Why This Matters

• User Impact: Prevents silent configuration loss and parser confusion for anyone copy-pasting from this widely-linked reference doc.
• Quality Factor: Technical accuracy and trust.
• Frequency: This is the canonical schedule-syntax reference, linked from /gh-aw/reference/triggers/ and used by every workflow author configuring schedules.

Success Criteria

• Changes made to docs/src/content/docs/reference/schedule-syntax.md only
• All 8 affected code blocks no longer contain duplicate schedule: keys under a single on: map
• Quality rating improves from ⚠️ to ✅

Scope Constraint

• Single file only: docs/src/content/docs/reference/schedule-syntax.md
• No changes to other files required
• Can be completed independently

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• storage.googleapis.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "storage.googleapis.com"

See Network Configuration for more information.

📊 User experience analysis by Delight · auto · 71.9 AIC · ⌖ 3.96 AIC · ⊞ 12.1K · ◷

• expires on Aug 5, 2026, 7:08 AM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Delight.

A newer discussion is available at Discussion #50018.