[delight] User Experience Analysis Report - 2026-09-05

[github-actions[bot]]

Executive Summary

Today's analysis focused on:

• 2 documentation file(s): docs/src/content/docs/blog/2026-01-13-meet-the-workflows-organization.md, docs/src/content/docs/reference/editors.mdx
• 0 CLI command(s) — the gh-aw binary could not be built in this sandbox (network access to the Go toolchain proxy was blocked), so this category was skipped for this run
• 2 workflow message configuration(s): .github/workflows/smoke-agent-scoped-approved.md, .github/workflows/smoke-project.md
• 1 validation file: pkg/workflow/mcp_scripts_dependencies_validation_wasm.go

Overall Quality: Generally professional. Documentation and workflow messages are clear and well-structured; the one gap found is a self-documentation gap in a WASM build-tag file, not a user-facing error message.

Key Finding: The editors.mdx reference page jumps straight into content without framing why a user would want an editor at all, and the WASM validation stub silently no-ops without explaining the behavioral difference from the native compiler — both are quick, single-file clarity fixes.

Quality Highlights ✅

Example 1: Clear, actionable smoke-test messages

• File: .github/workflows/smoke-agent-scoped-approved.md
• What works well: The messages block (lines 44-48) uses consistent, professional status wording with meaningful (not excessive) emoji, and the run-failure message explicitly tells the user to "Check the logs for details" — an actionable next step.
• Quote: "run-failure": "❌ [{workflow_name}]({run_url}) {status} testing guard policy: ... Check the logs for details."

Example 2: Well-structured error messages in the native validation file

• File: pkg/workflow/mcp_scripts_dependencies_validation.go
• What works well: newUnpinnedDependencyError (lines 160-169) explains the problem, names the expected format, and provides a concrete corrected example in one message — a strong template for actionable errors.
• Quote: "dependency %q for tool %q is not pinned to a release tag. Expected %s dependency format %q with an exact version. Example: dependencies: [%q]"

Improvement Opportunities 💡

High Priority

Opportunity 1: Add contextual intro to Workflow Editors reference page

• File: docs/src/content/docs/reference/editors.mdx
• Current State: The page opens directly with "The following editors can be used to author, compile, and preview agentic workflows" (line 91) without explaining who benefits from a dedicated editor versus using a plain text editor.
• Issue: Enterprise readers arriving from a search or nav link have no framing for why they'd choose one of these tools over just editing markdown/YAML directly.
• User Impact: Slightly increases cognitive load for first-time readers evaluating whether this page is relevant to them.
• Suggested Change: Add one or two sentences before the editor list explaining typical use cases (e.g., visual previews, in-browser compilation without local setup).
• Design Principle: Documentation Quality / Efficiency (reduce time to determine relevance).

Medium Priority

Opportunity 2: Document the WASM validation no-op behavior

• File: pkg/workflow/mcp_scripts_dependencies_validation_wasm.go
• Current State: The file unconditionally returns nil, silently skipping all MCP script dependency validation when compiled for js/wasm targets (e.g., the Compiler Playground referenced in editors.mdx).
• Issue: There's no comment explaining why this stub exists or that it causes a behavioral difference from the native compiler — a maintainer or playground user could be confused when dependency errors that fail locally pass silently in the browser.
• User Impact: Indirect — playground users may see workflows compile "successfully" where the CLI would reject them, undermining trust in the tool's consistency.
• Suggested Change: Add a doc comment explaining the rationale (e.g., dependency resolution requires OS/network access unavailable in WASM) and cross-reference the native implementation.
• Design Principle: Trust and Reliability (transparent about system behavior).

Files Reviewed

Documentation

• docs/src/content/docs/blog/2026-01-13-meet-the-workflows-organization.md - Rating: ✅
• docs/src/content/docs/reference/editors.mdx - Rating: ⚠️

CLI Commands

• (skipped this run — build blocked by sandboxed network restrictions)

Workflow Messages

• .github/workflows/smoke-agent-scoped-approved.md - Rating: ✅
• .github/workflows/smoke-project.md - Rating: ✅

Validation Code

• pkg/workflow/mcp_scripts_dependencies_validation_wasm.go - Rating: ⚠️

Metrics

• Files Analyzed: 5
• Quality Distribution:
  • ✅ Professional: 3
  • ⚠️ Needs Minor Work: 2
  • ❌ Needs Significant Work: 0

🎯 Actionable Tasks

Here are 2 targeted improvement tasks, each affecting a single file:

Task 1: Add contextual intro paragraph - docs/src/content/docs/reference/editors.mdx

File to Modify: docs/src/content/docs/reference/editors.mdx

Current Experience

The page begins at line 91 with a description of available editors but no framing of why a reader would want to use one instead of a plain text editor.

Quality Issue

Design Principle: Documentation Quality / Efficiency

Readers can't quickly assess relevance to their own workflow before scanning the full list of tools.

Proposed Improvement

Add a short intro sentence before the current opening line explaining the value proposition (visual preview, in-browser compilation, no local setup required).

Before:

The following editors can be used to author, compile, and preview agentic workflows. Some are built-in tools maintained alongside gh-aw; others are community-created projects.

After:

If you'd rather not hand-edit markdown and YAML frontmatter directly, the tools below provide visual previews, in-browser compilation, and richer editing support for agentic workflows — no local `gh aw` installation required for some options. The following editors can be used to author, compile, and preview agentic workflows. Some are built-in tools maintained alongside gh-aw; others are community-created projects.

Why This Matters

• User Impact: Readers decide in seconds whether this page solves their problem.
• Quality Factor: Documentation Quality (appropriate detail level, practical framing).
• Frequency: This is a reference page likely linked from onboarding docs, so many first-time users land here.

Success Criteria

• Changes made to docs/src/content/docs/reference/editors.mdx only
• Intro paragraph added above the existing opening sentence
• Quality rating improves from ⚠️ to ✅

Scope Constraint

• Single file only: docs/src/content/docs/reference/editors.mdx
• No changes to other files required
• Can be completed independently

---

Task 2: Document WASM no-op rationale - pkg/workflow/mcp_scripts_dependencies_validation_wasm.go

File to Modify: pkg/workflow/mcp_scripts_dependencies_validation_wasm.go

Current Experience

The stub silently returns nil for all js/wasm builds with no comment explaining why, diverging silently from the native validation logic in mcp_scripts_dependencies_validation.go.

Quality Issue

Design Principle: Trust and Reliability

Without an explanatory comment, a maintainer or a browser-playground user has no way to know that dependency validation is intentionally skipped in this build target, which could cause confusion when the same workflow behaves differently in the CLI versus the Compiler Playground.

Proposed Improvement

Add a doc comment explaining the rationale and pointing to the native implementation for context.

Before:

(go/redacted):build js || wasm

package workflow

func (c *Compiler) validateMCPScriptDependencies(workflowData *WorkflowData) error {
	return nil
}

After:

(go/redacted):build js || wasm

package workflow

// validateMCPScriptDependencies is a no-op on WASM/browser builds (e.g., the
// Compiler Playground) because dependency resolution requires filesystem and
// network access that is unavailable in that sandboxed environment. See the
// native implementation in mcp_scripts_dependencies_validation.go for the
// full validation logic used by the CLI.
func (c *Compiler) validateMCPScriptDependencies(workflowData *WorkflowData) error {
	return nil
}

Why This Matters

• User Impact: Prevents confusion for contributors and playground users about inconsistent validation behavior between build targets.
• Quality Factor: Trust and Reliability (transparency about system behavior).
• Frequency: Low direct frequency, but high confusion cost when encountered (silent divergence is hard to diagnose).

Success Criteria

• Changes made to pkg/workflow/mcp_scripts_dependencies_validation_wasm.go only
• Doc comment explaining the WASM no-op rationale added
• Quality rating improves from ⚠️ to ✅

Scope Constraint

• Single file only: pkg/workflow/mcp_scripts_dependencies_validation_wasm.go
• No changes to other files required
• Can be completed independently

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• storage.googleapis.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "storage.googleapis.com"

See Network Configuration for more information.

📊 User experience analysis by Delight · copilot · auto · 33.5 AIC · ⌖ 8.18 AIC · ⊞ 12.3K · ◷

• expires on Sep 8, 2026, 6:48 AM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Delight.

A newer discussion is available at Discussion #58998.