[delight] User Experience Analysis Report - 2026-09-13

[github-actions[bot]]

Executive Summary

Today's analysis focused on:

• 2 documentation files (drive-memory.md, using-at-scale.mdx)
• 0 CLI commands (binary could not be built in this sandbox — network access to the Go toolchain proxy was blocked; this limitation is noted, not a workflow finding)
• 2 workflow message configurations (pdf-summary.md, daily-byok-ollama-test.md)
• 1 validation file (expression_secrets_serialization_validation.go)

Overall Quality: Professional overall, with one concrete, high-value clarity gap in a security-critical error message.

Key Finding: The toJSON(secrets) compile-time error/warning explains what is wrong and how to fix it in general terms, but never shows a corrected example, so users editing a workflow under time pressure have to infer the fix pattern themselves.

Quality Highlights ✅

Example 1: Clear preview scoping in drive-memory.md

• File: docs/src/content/docs/experimental/drive-memory.md
• What works well: Opens with an explicit [!CAUTION] admonition scoping the feature to enrolled preview repos, and the "Limitations" section lists constraints as a scannable bullet list rather than prose — exactly the efficient, low-cognitive-load format enterprise readers need before adopting an experimental feature.
• Quote/Reference: "Drive memory is an experimental, feature-gated integration ... Do not configure it unless GitHub has explicitly enrolled the repository in the private preview." (lines 8, 10-13)

Example 2: Layered guidance in using-at-scale.mdx

• File: docs/src/content/docs/guides/using-at-scale.mdx
• What works well: Consistently separates "developer-facing mechanics" from "organizational governance" (e.g. in the Reusing/Sharing section) and links every claim to a canonical reference page rather than duplicating content — reduces maintenance burden and keeps each doc authoritative on its own topic.

Improvement Opportunities 💡

High Priority

Opportunity 1: Secrets-serialization error lacks a corrected example - Single File Improvement

• File: pkg/workflow/expression_secrets_serialization_validation.go
• Current State: The error/warning message (lines 188-193) reads: "secrets serialization expression(s) detected that would expose all secrets to the agent. Found: %s. Use specific secret references (e.g. secrets.MY_SECRET) instead of passing the entire secrets context." It names the offending expression but gives only an abstract placeholder (secrets.MY_SECRET), not a rewritten version of what the user actually wrote.
• Issue: A user who hits this in strict mode (a hard compile failure) sees their own ${{ toJSON(secrets) }} expression echoed back, then a generic instruction — but no worked "before → after" showing how to replace a whole-context serialization with individual secret references in something like an env: block.
• User Impact: For an enterprise user under a failing CI build, extra round-trips (reading docs, guessing syntax) increase time-to-green. This is also a security-sensitive fix path — clearer guidance directly reduces the temptation to silently disable strict mode instead of fixing the real issue.
• Suggested Change: Extend the message to include one concrete before/after snippet, e.g. append: "For example, replace 'env: { ALL: ${{ toJSON(secrets) }} }' with 'env: { MY_SECRET: ${{ secrets.MY_SECRET }} }'." This keeps the change scoped to the message string in this one file.
• Design Principle: Trust and Reliability — "Clear error messages with actionable solutions."

Files Reviewed

Documentation

• docs/src/content/docs/experimental/drive-memory.md - Rating: ✅
• docs/src/content/docs/guides/using-at-scale.mdx - Rating: ✅

CLI Commands

• Not evaluated this run (binary build blocked by sandbox network restrictions)

Workflow Messages

• .github/workflows/pdf-summary.md - Rating: ✅ (themed emoji footer/messages are consistent and on-brand for the workflow's purpose; professional despite playful icons)
• .github/workflows/daily-byok-ollama-test.md - Rating: ✅ (status messages are terse, accurate, and include actionable status like "BYOK endpoint responded")

Validation Code

• pkg/workflow/expression_secrets_serialization_validation.go - Rating: ⚠️ Needs Minor Work

Metrics

• Files Analyzed: 5
• Quality Distribution:
  • ✅ Professional: 4
  • ⚠️ Needs Minor Work: 1
  • ❌ Needs Significant Work: 0

🎯 Actionable Tasks

Here is 1 targeted improvement task, affecting a single file:

Task 1: Add a worked example to the secrets-serialization error message

File to Modify: pkg/workflow/expression_secrets_serialization_validation.go

Current Experience

The validateSecretsSerializationExpressions function (around lines 188-193) builds this message when it detects ${{ toJSON(secrets) }} or similar whole-context serialization:

secrets serialization expression(s) detected that would expose all secrets to the agent.
Found: <expr>. Use specific secret references (e.g. secrets.MY_SECRET) instead of
passing the entire secrets context.

This appears as a hard compile error in strict mode, or a warning otherwise (see lines 195-199).

Quality Issue

Design Principle: Trust and Reliability — actionable, self-service error messages.

The message correctly identifies the problem and names the risky pattern, but stops short of showing the user what a corrected expression looks like in context (e.g., inside env:). Users must guess the right replacement syntax or search documentation/ADR-44922 for an example, adding friction to a security-relevant fix.

Proposed Improvement

Append a concrete before/after example to the msg string so the guidance is self-contained.

Before:

	msg := fmt.Sprintf(
		"secrets serialization expression(s) detected that would expose all secrets to the agent. "+
			"Found: %s. "+
			"Use specific secret references (e.g. secrets.MY_SECRET) instead of passing the entire secrets context.",
		strings.Join(allFound, ", "),
	)

After:

	msg := fmt.Sprintf(
		"secrets serialization expression(s) detected that would expose all secrets to the agent. "+
			"Found: %s. "+
			"Use specific secret references (e.g. secrets.MY_SECRET) instead of passing the entire secrets context. "+
			"For example, replace `env: { ALL_SECRETS: ${{ toJSON(secrets) }} }` with "+
			"`env: { MY_SECRET: ${{ secrets.MY_SECRET }} }`, listing only the secrets the step actually needs.",
		strings.Join(allFound, ", "),
	)

Why This Matters

• User Impact: Cuts the fix loop from "read error → search docs/ADR → guess syntax" down to "read error → copy pattern → done."
• Quality Factor: Trust and Reliability (actionable error messages)
• Frequency: Triggered any time a workflow author writes toJSON(secrets), including by newcomers copying patterns from unrelated GitHub Actions tutorials that are common outside gh-aw.

Success Criteria

• Changes made to pkg/workflow/expression_secrets_serialization_validation.go only
• Compile-time error/warning text includes a concrete before/after example
• Existing unit tests for this validator (e.g. expression_secrets_serialization_validation_test.go) updated to match the new message substring, if they assert on exact message text

Scope Constraint

• Single file only: pkg/workflow/expression_secrets_serialization_validation.go
• No changes to other files required (test file may need a message-substring update if it asserts exact text, but the core change is contained here)
• Can be completed independently

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• storage.googleapis.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "storage.googleapis.com"

See Network Configuration for more information.

📊 User experience analysis by Delight · copilot · auto · 52.1 AIC · ⌖ 14.5 AIC · ⊞ 12.5K · ◷

• expires on Sep 16, 2026, 6:50 AM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Delight.

A newer discussion is available at Discussion #60864.