[docs-noob-tester] 📚 Documentation Noob Test Report - 2026-09-06

[github-actions[bot]]

Summary

• Date of test: 2026-09-06
• Pages visited:
  • Home — `(localhost/redacted)
  • Quick Start Guide — `(localhost/redacted)
  • CLI Commands — `(localhost/redacted)
• Overall impression: As a brand-new user, the docs are clear and well organized — the home page immediately explains what the tool does and points to "Get Started with CLI," and the Quick Start guide walks through a full working example (install → add sample workflow → wait for result → customize) with realistic time estimates and a visual example of the output.

Critical Issues Found

None found that block getting started. All three pages loaded correctly, no 404s, and the documented CLI commands (gh extension install github/gh-aw, gh aw add-wizard, gh aw compile, gh aw run) are shown with clear syntax.

One near-miss worth flagging: the sample-output image on the Quick Start page ("The report will look something like this:") initially reported naturalWidth/Height = 0 via script inspection before it scrolled into view — this turned out to be lazy-loading behavior, not an actual broken image (confirmed 200 response and full load after scroll), but a first-time reader skimming quickly could briefly wonder if the image is broken.

Confusing Areas

• Jargon introduced without inline definition: the intro paragraph mentions "recurring agentic workflow" and "the stable-engine path" without explicitly defining "engine" (AI engine/runtime) before this point — a newcomer may not yet know that "engine" = Copilot/Claude/Codex/Gemini/Pi choice.
• Five parallel setup guides collapsed into tabs: the "Configuring authentication" section has 5 tabs (Copilot/Claude/Codex/Gemini/Pi) but only the Copilot tab content was inspected by default — a first-time user has to know to click through tabs to find instructions for their chosen engine; it's not obvious from a glance which tab is "selected" by default relative to what was chosen in Step 2.
• gh aw add-wizard walks through 6 sub-steps (check prerequisites, select engine, configure auth, add workflow, add skills, trigger run) listed in prose — a numbered list or short animated GIF of the actual interactive prompts would make expectations clearer before running the command for real.

What Worked Well

• Home page nav is excellent: "Quick Start," "Create," "Gallery," "Docs," "FAQ" are all one click away, and the two CTA buttons ("Get Started with CLI" and "Creating Workflows") make the very next action obvious.
• Quick Start prerequisites are listed as a clean checklist (AI account, repo, Actions enabled, gh CLI v2.0.0+, login, OS) — no assumed context.
• Time estimates ("~10 minutes," "2–3 minutes for the run") set good expectations up front.
• The CLI Commands page leads with a "Day-one commands" table that includes a "When to use" column — this is much more useful for a beginner than an alphabetical command reference, and a tip box explicitly says "New to gh aw? Start with the day-one commands... can be skipped for most users," which is reassuring.
• Fallback install path (curl ... install-gh-aw.sh | bash) is provided for users hitting gh extension install auth issues — good anticipation of a common failure mode.
• Version pinning instructions (gh extension install github/gh-aw@v0.1.0) are clearly explained for teams wanting reproducibility.

Recommendations

Quick wins:

1. Add a one-line inline definition or tooltip for "engine" the first time it's used on the Quick Start page (e.g., "engine — the AI provider/runtime that executes the workflow, e.g., GitHub Copilot, Claude, Codex, Gemini, or Pi").
2. Indicate visually which authentication tab (Copilot/Claude/Codex/Gemini/Pi) corresponds to the engine chosen in the wizard, or add a note like "select the tab matching what you chose in Step 2."

Longer-term:

1. Consider a short GIF/video of gh aw add-wizard running end-to-end so new users know what to expect from the interactive prompts before running it themselves.
2. For images below the fold (like the sample report screenshot), consider loading="eager" or a fixed placeholder height so first-time readers don't momentarily wonder if content is broken while scrolling.

Screenshots

📎 home.png — Home page, nav and CTAs — asset URL: https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/2d9d7391d69db7d355855c8a464dd0690de5e0df777c9e1d19a86e11d121d4aa.png?raw=true

📎 quick-start-top.png — Quick Start guide, prerequisites and Step 1 — asset URL: https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/138d6349b8dd745cb578d4203f3957c7163135df10e7716fdcac847b369c839f.png?raw=true

📎 quick-start-report-image.png — Quick Start guide, sample "Daily Repo Status Report" output image — asset URL: https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/df8367bcc7d2838e5cef6b52ef49997cab09b5977910d84bcd086ca4cdbab923.png?raw=true

📎 cli-page.png — CLI Commands page, "Day-one commands" table — asset URL: https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/341d1c446c2b370839c896e928271d2cea5a6ca84f26a9cb53452f4d75d24251.png?raw=true

Warning

Firewall blocked 5 domains

The following domains were blocked by the firewall during workflow execution:

• accounts.google.com
• android.clients.google.com
• clients2.google.com
• www.google.com
• www.gstatic.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "www.google.com"
    - "www.gstatic.com"

See Network Configuration for more information.

Generated by 📝 Documentation Noob Tester · copilot · auto · 41.4 AIC · ⌖ 11.5 AIC · ⊞ 10.8K · ◷

• expires on Sep 6, 2026, 7:52 PM UTC-08:00