[docs-noob-tester] 📚 Documentation Noob Test Report - 2026-08-13

[github-actions[bot]]

Summary

• Date of test: 2026-08-13
• Pages visited:
  • (github.github.io/redacted) (home)
  • (github.github.io/redacted)
  • (github.github.io/redacted)
• Overall impression: As a brand-new user, the site is clean, well-organized, and the Quick Start guide is genuinely usable end-to-end in ~10 minutes — but the sheer depth of the sidebar and repeated engine/auth branching can feel overwhelming before you've even run your first command.

🔴 Critical Issues Found

None encountered that would fully block getting started. The Quick Start's core path (install extension → add-wizard → wait for run) is complete and each command is copy-pasteable.

🟡 Confusing Areas

1. "gh-aw" jargon used before it's defined. The home page and nav both say "gh-aw" repeatedly before spelling out that it's "the GitHub CLI extension." A first-time visitor briefly wonders if gh-aw is a product name, a package, or a GitHub Action.
2. Two similar CTAs on the home page: "Quick Start with CLI" and the nav's "Quick Start" both point to the same /setup/quick-start/ page, plus a separate "Creating Workflows" CTA. It's not obvious as a beginner which one to click first — is "Creating Workflows" a different, more advanced start?
3. Sidebar is huge on first load. The Quick Start page's sidebar has 6+ expandable sections (Introduction, Setup, AI Engines, Guides, Design Patterns, ...). For a noob just trying to follow one linear tutorial, this signals "there's a LOT more to learn" and can feel intimidating before finishing step 1.
4. Authentication branching inside Step 2 ("Configuring authentication? Select the engine you chose above: Copilot / Claude / Codex / Gemini / Pi") is presented as tabs but reads as a wall of options for a user who just wants the default (Copilot) path. It would help to visually de-emphasize the non-default engines for first-time readers.
5. CLI Commands page mixes beginner and advanced content on one page — "Most Common Commands" table is immediately followed by "Pinning to a Specific Version," "Alternative: Standalone Installer," and "Advanced: GitHub Enterprise Server Support" all inline, rather than collapsed/deferred. The page does have a nice "Tip: New to gh-aw? You can stop here..." callout, which helps, but it appears after several advanced install variants.

🟢 What Worked Well

• Prerequisites list is excellent — concrete, checkable items (gh --version, gh auth status) instead of vague requirements.
• Quick Start's time estimate ("about 10 minutes") sets clear expectations up front.
• Step-by-step structure with numbered steps and a "Tip" callout for auth troubleshooting (fine-grained PAT vs. copilot-requests: write) is genuinely helpful and anticipates a common failure point.
• .lock.yml explanation ("this is the compiled GitHub Actions workflow... regenerate with gh aw compile") preempts a likely point of confusion (why is there a generated file, can I edit it?).
• CLI Commands page's "Most Common Commands" table is a great single-glance reference with a "When to use" column — very beginner-friendly design once you scroll past the install section.

Recommendations

Quick wins:

• Add one sentence near the top of the home page explicitly defining gh-aw as "the gh aw GitHub CLI extension" before using the shorthand repeatedly.
• Differentiate the two home-page CTAs, e.g., "Quick Start with CLI (10 min, guided)" vs. "Creating Workflows (write your own from scratch)" so beginners can self-select.
• Move the "New to gh-aw? You can stop here" tip to before the version-pinning/standalone-installer/enterprise sections on the CLI Commands page, or collapse those sections by default.

Longer-term:

• Consider a collapsed/minimal sidebar variant specifically on the Quick Start page (or a "linear tutorial mode") so first-time users aren't shown the entire doc tree while completing their first task.
• For the engine-authentication step, consider defaulting to a single visible panel (Copilot) with the other engines behind a "different engine?" toggle, rather than showing all four/five inline.

Screenshots

📎 [home.png] — asset URL: https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/c2ee669671d427a966bc9784c5256236380563a6db8d47714f9e46967d308561.png?raw=true

📎 [quickstart.png] — asset URL: https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/f585571d9367db9e9ccfba86ba99677970236243d761c9483bf381d241c85310.png?raw=true

📎 [cli.png] — asset URL: https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/3a8641957ccc0a69290b0c71e1f25865efff8d6778977dc20a75ad2ad3096754.png?raw=true

Warning

Firewall blocked 5 domains

The following domains were blocked by the firewall during workflow execution:

• accounts.google.com
• android.clients.google.com
• clients2.google.com
• www.google.com
• www.gstatic.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "www.google.com"
    - "www.gstatic.com"

See Network Configuration for more information.

Generated by 📝 Documentation Noob Tester · auto · 34.9 AIC · ⌖ 3.57 AIC · ⊞ 9.7K · ◷

• expires on Aug 13, 2026, 8:29 PM UTC-08:00