[docs-noob-tester] 📚 Documentation Noob Test Report - 2026-05-17

[github-actions[bot]]

Summary

• Date: 2026-05-17
• Pages visited: Home (/gh-aw/), Quick Start (/gh-aw/setup/quick-start/), CLI Commands (/gh-aw/setup/cli/)
• Overall impression: The home page is welcoming and clear about the tool's purpose, but the Quick Start guide has some areas that could confuse absolute beginners — particularly around secrets setup and the concept of "frontmatter".

---

🟢 What Worked Well

• Home page headline is clear: "Repository automation, running the coding agents you know and love, with strong guardrails in GitHub Actions." — immediately tells me what this does.
• Two prominent CTAs: "Quick Start with CLI" and "Creating Workflows" are front and center — easy to find.
• Prerequisites section in Quick Start: Nicely structured checklist (AI Account, GitHub Repo, GitHub Actions enabled, GitHub CLI, logged in). This is helpful for a beginner.
• Estimated time (10 minutes): Setting expectations is great UX.
• CLI Commands page: The "Most Common Commands" table at the top is excellent — a beginner can immediately see the core workflow at a glance.
• Navigation is clean: The top nav has "Quick Start", "Create", "Examples", "Docs", "FAQ" — logical structure.

---

🔴 Critical Issues (Block Getting Started)

1. COPILOT_GITHUB_TOKEN setup is buried and unclear for beginners

The note about setting up COPILOT_GITHUB_TOKEN is a critical step — without it, the workflow won't run — but it's shown as a collapsible "Note" callout mid-page. A new user who misses this will be stuck. The instructions say:

Create a fine-grained PAT under your user account. Under Permissions → Account permissions, set Copilot Requests to Read...

For a beginner who has never created a PAT, this is dense. There's no link to GitHub's PAT creation UI, and "Copilot Requests" permission is not intuitive to find.

Recommendation: Add a dedicated step (Step 1.5 or a pre-step) for secrets setup with a direct link to https://github.com/settings/tokens?type=beta and a screenshot/GIF of where to find the "Copilot Requests" permission.

2. gh aw add-wizard githubnext/agentics/daily-repo-status — unclear what this format means

The command uses <owner>/<repo>/<workflow-name> format but a beginner won't know what githubnext/agentics is or whether they need to change it. The explanation comes after the command, not before.

Recommendation: Move the format explanation before the command, or add an inline comment in the code block.

3. "Frontmatter" jargon used without definition on first mention

In Step 4, the guide references "frontmatter" without defining it for someone who doesn't know YAML:

the frontmatter (the configuration block between the --- markers at the top)

This is actually a good inline definition! But it only appears in Step 4. The term appears earlier in the page (and across the site) without this explanation.

Recommendation: Add a brief tooltip, glossary link, or inline definition at the first occurrence of "frontmatter" on the page.

---

🟡 Confusing Areas

4. gh aw add vs gh aw add-wizard vs gh aw new vs gh aw init — too many similar commands

On the CLI Commands page, beginners see four commands that all seem to do "add a workflow":

• init — Set up repository
• add-wizard — Add with interactive setup
• add — Add from other repos (non-interactive)
• new — Create a new workflow from scratch

There's no clear "use this one first" guidance. The Quick Start uses add-wizard, but a beginner reading the CLI page may not know which to start with.

Recommendation: Add a brief "Start here" note or a decision tree: "New to gh-aw? Use add-wizard. Creating your own? Use new."

5. The video on Quick Start doesn't have captions/transcript

The page includes a video ("Your browser doesn't support HTML5 video") which apparently shows installing and triggering a run. This is great! But there's no transcript or caption, and if the video fails to load, users get just a fallback message.

Recommendation: Add alt-text or a brief step list below the video as a fallback.

6. Step 2 command gh aw add shown without context

After gh aw add-wizard githubnext/agentics/daily-repo-status, the page says:

"This will take you through an interactive process to..."

But a beginner doesn't know if they should run this in any directory or specifically their repo root. There's no "Make sure you're in your repository directory" reminder.

Recommendation: Add cd /path/to/your-repo as a prerequisite command before the wizard.

7. CLI page is comprehensive but overwhelming

The CLI Commands page is very thorough (~30+ commands listed), which is great for reference but overwhelming for a beginner. The "Most Common Commands" table helps, but it's below a lot of installation instructions.

Recommendation: Consider a "beginner path" callout at the top: "First time? You only need: init, add-wizard, run, logs."

---

📸 Screenshots

📎 home.png — Home page overview

📎 quick-start.png — Quick Start guide

📎 cli.png — CLI Commands reference page

---

🎯 Recommendations (Prioritized)

Priority	Change	Impact
🔴 High	Add dedicated secrets-setup step with direct PAT link	Unblocks most new users
🔴 High	Add cd your-repo reminder before wizard command	Prevents silent failures
🟡 Medium	Define "frontmatter" on first occurrence	Reduces jargon confusion
🟡 Medium	Add "start here" guidance on CLI page	Reduces overwhelm
🟢 Low	Add video transcript/fallback	Improves accessibility
🟢 Low	Decision tree for add/add-wizard/new/init	Clarifies command choices

---

References:

• §25981870631

Warning

Firewall blocked 5 domains

The following domains were blocked by the firewall during workflow execution:

• accounts.google.com
• android.clients.google.com
• clients2.google.com
• safebrowsingohttpgateway.googleapis.com
• www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

Generated by 📝 Documentation Noob Tester · ● 17.8M · ◷

• expires on May 18, 2026, 5:08 AM UTC