[docs-noob-tester] 📚 Documentation Noob Test Report - 2026-09-17

[github-actions[bot]]

Summary

• Date of test: 2026-09-17
• Pages visited:
  • `(localhost/redacted) (Home)
  • `(localhost/redacted) (Quick Start Guide)
  • `(localhost/redacted) (CLI Commands)
• Overall impression: As a brand-new user, I was pleasantly surprised — the home page immediately makes the value proposition clear ("Run the coding agents you know and love... in GitHub Actions") and the "Get Started with CLI" call-to-action is impossible to miss. The Quick Start guide is genuinely a 10-minute, copy-paste-able path to a working workflow.

🔴 Critical Issues Found

None encountered. All links I checked (gh CLI install, GitHub Actions settings docs, engine auth docs) resolved to real targets, and every code block in the Quick Start guide (gh extension install github/gh-aw, gh aw add-wizard githubnext/agentics/repo-status, gh aw compile) was copy-paste-ready with no placeholder text left in.

🟡 Confusing Areas

1. "Frontmatter" jargon on first mention — The Quick Start guide's very first paragraph uses "YAML frontmatter" before explaining what it is; a parenthetical definition follows immediately ("the configuration block between the --- markers"), which helps, but a total beginner unfamiliar with Markdown/Jekyll-style frontmatter conventions may still be briefly confused on first read. Minor, since the definition is right there.
2. Four AI engine auth paths shown at once — Step 2's "Configuring authentication" section shows tabs/sections for Copilot, Claude, Codex, Gemini, and Pi all at once. A first-time user has to scan all five to find the one they care about; a single interactive picker (like the CLI wizard itself provides) would reduce cognitive load compared to the static page listing all five paths.
3. CLI Commands page is dense — The page opens with a full command reference; a brand-new user landing there directly (rather than via Quick Start) could feel overwhelmed. The "Day-one commands" table plus the tip box saying "Start with the day-one commands... can be skipped for most users" is a great mitigation, but it only helps once the user notices it — it's a bit far down the page relative to how early users typically bail.

🟢 What Worked Well

• Home page clarity: The tagline and three prominent buttons ("Get Started with CLI", "Creating Workflows", "Open Workflow Wizard") make the next step obvious within seconds.
• Quick Start structure: Clearly numbered steps (Install → Add workflow → Wait → Customize), an "Estimated time: 10 minutes" badge, and an explicit Prerequisites checklist (AI account, repo, Actions enabled, gh CLI version, auth) set expectations up front.
• CLI page's "Day-one commands" callout: A tip box explicitly tells newcomers which commands matter and says the rest "can be skipped for most users" — exactly the kind of guardrail a noob needs to avoid getting lost in an exhaustive reference.
• No dead links or console errors found on any of the three pages during this pass.
• Copy-paste commands throughout Quick Start are complete and don't require the reader to substitute unclear placeholders.

Recommendations

Quick wins:

• Add a one-line inline definition of "frontmatter" the very first time it's used (e.g., "the YAML configuration block — like a header — between the --- markers"), even before the parenthetical.
• Move the "Day-one commands" tip box (or an equivalent) to the very top of the CLI Commands page so it's the first thing a new visitor sees, not something they must scroll to find.

Longer-term:

• Consider collapsing the four/five engine authentication sections into an interactive selector or tabbed UI on the Quick Start page, so a user only sees the one path relevant to their chosen engine instead of scanning all five.
• Consider a "New here? Start with Quick Start →" banner at the top of the CLI Commands page for users who land there directly (e.g., via search) without going through Quick Start first.

Screenshots

📎 [home.png] — Home page hero and navigation — asset URL: https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/52c02ae17f60e87ac27fcbf9bcd137a0895420174235987cb43fd80d62c1610d.png?raw=true

📎 [quick-start-top.png] — Quick Start guide, full page — asset URL: https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/27db4afc6af5d0930fa9925ecbd7af8e334a0d40ac66e8e0a5805f34186c5cb7.png?raw=true

📎 [cli-commands.png] — CLI Commands reference page, full page — asset URL: https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/a7fc9590bda8b0560ce95259b032009bcd6e49fc683a5b3f6dfe6f98866798ed.png?raw=true

Warning

Firewall blocked 6 domains

The following domains were blocked by the firewall during workflow execution:

• accounts.google.com
• android.clients.google.com
• clients2.google.com
• update.googleapis.com
• www.google.com
• www.gstatic.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "update.googleapis.com"
    - "www.google.com"
    - "www.gstatic.com"

See Network Configuration for more information.

Generated by 📝 Documentation Noob Tester · copilot · auto · 68.2 AIC · ⌖ 10 AIC · ⊞ 11.1K · ◷

• expires on Sep 17, 2026, 7:55 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Documentation Noob Tester.

A newer discussion is available at Discussion #61713.