[delight] User Experience Analysis Report - 2026-09-10

[github-actions[bot]]

Executive Summary

Today's analysis focused on:

• 2 documentation files (mcp-gateway.md specification, 2026-09-09-agent-of-the-day.md blog post)
• 2 workflow message configurations (smoke-deepseek-harness.md, daily-credit-limit-test.md)
• 1 validation file (pkg/workflow/lock_validation.go)

CLI help-text sampling was skipped this run: the gh-aw binary could not be built in this sandbox (offline Go toolchain download was blocked), so no CLI commands could be exercised safely.

Overall Quality: Mostly professional, with one concrete accuracy defect found in a formal specification document.

Key Finding: The MCP Gateway Specification (docs/src/content/docs/reference/mcp-gateway.md) mixes JSON and TOML configuration syntaxes inconsistently within the same document, creating a real risk that operators copy-paste an example in the wrong format for their deployment method.

Quality Highlights ✅

Example 1: Blog Post — "Agent of the Day" (PureLock)

• File: docs/src/content/docs/blog/2026-09-09-agent-of-the-day.md
• What works well: Concrete, verifiable claims (PR numbers, coverage percentages, run timings) rather than vague marketing language. All three cited PRs ([purelock] Lock down selectHistoricalOperationalValueGrader, extractHostFromRemoteURL, extractOTLPAttributesFromObsMap with pure [Content t #56895, [purelock] Lock down lowercaseDiscussionTriggerTypesInLines, isSafeGitRefName with pure-function test suites #54539, Prevent PureLock Go cache restore collisions #57948) were verified as real, merged PRs with matching descriptions. The tone is professional yet engaging, and it ends with a clear call to action.
• Quote/Reference: "Every claim is backed by a gofmt, go vet, and go test -race pass recorded directly in the PR body — no unverified assertions, no rubber-stamped merges." This is exactly the kind of trust-building, precise communication enterprise documentation should aim for.

Example 2: Error Messages in Lock File Validation

• File: pkg/workflow/lock_validation.go
• What works well: Errors are actionable and include the exact remediation command.
• Quote/Reference: "lock file %s uses unsupported schema version '%s'.\n\nThis file was generated by a newer version of gh-aw...\n\nTo fix this:\n 1. Upgrade gh-aw: gh extension upgrade gh-aw\n 2. Or downgrade the lock file by editing the source .md file and recompiling:\n gh aw compile %s" — states the problem, explains why, and gives two concrete resolution paths.

Improvement Opportunities 💡

High Priority

Opportunity 1: Inconsistent Configuration Syntax in MCP Gateway Specification

• File: docs/src/content/docs/reference/mcp-gateway.md
• Current State: Section 4.1.1 defines the canonical configuration format as JSON delivered via stdin ("The gateway MUST accept configuration via stdin in JSON format", line ~163). All field names in the reference tables use camelCase (payloadDir, payloadPathPrefix, agentId). However, Sections 4.1.3.2 (line 330), 4.1.3.3 (line 363), and 4.1.3.9 (line 640) show configuration examples in TOML syntax with snake_case field names (payload_dir, payload_path_prefix, agent_id), introduced without explanation of when TOML applies vs. JSON.
• Issue: A reader implementing or configuring the gateway cannot tell from context whether TOML is an alternate supported format, a different tool (mcpg CLI, referenced only once at line 639), or an error. The spec's own Section 4.1 states JSON is the required configuration format, directly contradicting the TOML examples that follow.
• User Impact: Enterprise operators writing gateway configuration may copy the TOML snippet verbatim into a JSON-only config, producing a parse failure with no clear explanation of why. This creates avoidable friction and undermines trust in a document that positions itself as a formal spec ("following W3C conventions").
• Suggested Change: Add a brief clarifying note before the first TOML example (line ~328) explaining that these snippets represent the mcpg CLI/local config file format (a separate consumer), distinct from the JSON payload the gateway process itself reads via stdin — or convert the TOML examples to JSON if they are meant to represent the same configuration surface.
• Design Principle: Clarity and Precision; Trust and Reliability.

Files Reviewed

Documentation

• docs/src/content/docs/reference/mcp-gateway.md - Rating: ⚠️ Needs Minor Work
• docs/src/content/docs/blog/2026-09-09-agent-of-the-day.md - Rating: ✅ Professional

CLI Commands

• Skipped this run (build environment offline; no gh-aw binary available)

Workflow Messages

• .github/workflows/smoke-deepseek-harness.md - Rating: ✅ Professional
• .github/workflows/daily-credit-limit-test.md - Rating: ✅ Professional

Validation Code

• pkg/workflow/lock_validation.go - Rating: ✅ Professional

Metrics

• Files Analyzed: 5
• Quality Distribution:
  • ✅ Professional: 4
  • ⚠️ Needs Minor Work: 1
  • ❌ Needs Significant Work: 0

🎯 Actionable Tasks

One targeted improvement task, affecting a single file:

Task 1: Clarify JSON vs. TOML Configuration Syntax in MCP Gateway Specification

File to Modify: docs/src/content/docs/reference/mcp-gateway.md

Current Experience

Section 4.1 ("Configuration Format", ~line 163) states unambiguously:

"The gateway MUST accept configuration via stdin in JSON format conforming to the MCP configuration file schema."

All field-reference tables that follow (Sections 4.1.2, 4.1.3) use camelCase JSON field names (payloadDir, payloadPathPrefix, agentId, agentIds). Yet three later subsections present configuration examples in TOML with snake_case names:

• Line ~330 (Section 4.1.3.2, "Payload Path Prefix for Agent Containers")
• Line ~363 (Section 4.1.3.3, payload size threshold)
• Line ~640 (Section 4.1.3.9, "Agent Identifier Configuration") — the only place that names the TOML format's actual consumer: **mcpg TOML spelling**:

The mcpg name only appears once (line 640), so a reader who reaches the earlier TOML block at line 330 has no indication that a different tool/config surface is being described.

Quality Issue

Design Principle: Clarity and Precision — a formal specification should never leave the reader guessing which configuration format applies where.

Introducing an unlabeled TOML dialect partway through a document that opens by mandating JSON creates ambiguity about which format is authoritative for the gateway's actual stdin input versus an unrelated CLI tool's local config file. This is a correctness/precision gap, not a style nit — it could lead to real misconfiguration.

Proposed Improvement

Label every TOML example with the same explanatory framing already used (but only once) at line 640, so each occurrence is unambiguous on its own.

Before (line ~328-336):

**Configuration Example**:

```toml
[gateway]
payload_dir = "/tmp/jq-payloads"
payload_path_prefix = "/workspace/payloads"
port = 8080
domain = "localhost"
agent_id = "secret"

**After**:

Configuration Example (mcpg TOML spelling — separate CLI config file, not the gateway's JSON stdin payload):

[gateway]
payload_dir = "/tmp/jq-payloads"
payload_path_prefix = "/workspace/payloads"
port = 8080
domain = "localhost"
agent_id = "secret"

Apply the same labeling convention to the TOML block at line ~363 (Section 4.1.3.3). This makes the distinction between the gateway's JSON stdin format and the `mcpg` CLI's TOML file format explicit and consistent everywhere it appears, rather than relying on the reader having already seen the one clarifying label at line 640.

**Why This Matters**
- **User Impact**: Prevents operators from copy-pasting a TOML snippet into a JSON-only gateway config and hitting an unexplained parse failure.
- **Quality Factor**: Clarity and Precision; Trust and Reliability.
- **Frequency**: This is a reference specification consulted by anyone integrating or deploying the MCP Gateway — a foundational, frequently-linked document.

**Success Criteria**
- [ ] Changes made to `docs/src/content/docs/reference/mcp-gateway.md` only
- [ ] Every TOML example block includes an inline label clarifying it represents the `mcpg` CLI config format, distinct from the gateway's JSON stdin input
- [ ] Quality rating improves from ⚠️ Needs Minor Work to ✅ Professional

**Scope Constraint**
- **Single file only**: `docs/src/content/docs/reference/mcp-gateway.md`
- No changes to other files required
- Can be completed independently


<!-- gh-aw-tracker-id: delight-daily -->

> [!WARNING]
> <details>
> <summary>Firewall blocked 1 domain</summary>
>
> The following domain was blocked by the firewall during workflow execution:
>
> - `storage.googleapis.com`
>
> To allow these domains, add them to the `network.allowed` list in your workflow frontmatter:
>
> ```yaml
> network:
>   allowed:
>     - defaults
>     - "storage.googleapis.com"
> ```
>
> See [Network Configuration](https://github.github.com/gh-aw/reference/network/) for more information.
>
> </details>

> 📊 *User experience analysis by [Delight](https://github.com/github/gh-aw/actions/runs/34491017483)* · copilot · auto · 71.9 AIC · ⌖ 7.88 AIC · ⊞ 12.5K · [◷](https://github.com/search?q=repo%3Agithub%2Fgh-aw+%22gh-aw-workflow-call-id%3A+github%2Fgh-aw%2Fdelight%22&type=discussions)
> - [x] expires <!-- gh-aw-expires: 2026-09-13T14:53:19.838Z --> on Sep 13, 2026, 6:53 AM UTC-08:00

<!-- gh-aw-agentic-workflow: Delight, gh-aw-tracker-id: delight-daily, engine: copilot, model: auto, id: 34491017483, workflow_id: delight, run: https://github.com/github/gh-aw/actions/runs/34491017483 -->

<!-- gh-aw-workflow-id: delight -->
<!-- gh-aw-workflow-call-id: github/gh-aw/delight -->

[github-actions[bot]]

This discussion has been marked as outdated by Delight.

A newer discussion is available at Discussion #60288.