[docs-noob-tester] 📚 Documentation Noob Test Report - 2026-08-28

[github-actions[bot]]

Summary

• Date of test: 2026-08-28
• Pages visited: /gh-aw/ (home), /gh-aw/setup/quick-start/, /gh-aw/setup/cli/
• Overall impression (as a brand-new user): The landing page sells the concept well and the Quick Start is genuinely one of the more beginner-friendly setup guides I've read — it's scoped to a single realistic outcome (a working scheduled workflow) instead of a wall of abstract concepts. A few small gaps would still trip up someone with zero GitHub Actions/CLI background.

⚠️ Testing note: I was unable to capture browser screenshots in this sandbox — the Playwright browser binary could not launch (libnspr4.so/libnss3.so missing, no root/apt access to install them). This is a sandbox/tooling limitation, not a documentation problem. I instead reviewed the fully rendered HTML (via curl, HTTP 200 for all 3 pages) and the underlying MDX/Markdown source to evaluate content, structure, and navigation.

---

🔴 Critical Issues Found

None that would fully block a new user. All 3 pages loaded successfully (HTTP 200), the primary CLI commands (gh extension install, gh aw add-wizard, gh aw compile, gh aw run) are consistent between the Quick Start and CLI reference pages, and the linked resources (Copilot, Anthropic, OpenAI, Gemini docs) are all plausible/real destinations.

🟡 Confusing Areas

1. "AI Account" prerequisite is a bit ambiguous for a total beginner. The prerequisites list says "AI Account – GitHub Copilot, Anthropic Claude, OpenAI Codex, or Google Gemini... If you already have GitHub Copilot, start there — it requires no extra account setup." A first-timer may not know whether they have Copilot access, or that Copilot avoids a whole extra sign-up/API-key step. A one-line "not sure? use Copilot, it needs zero extra setup" callout at the very top would help people avoid over-thinking this decision before they've even started.

2. add-wizard argument format explained after the command is shown. Step 2 shows gh aw add-wizard githubnext/agentics/daily-repo-status and only explains the <owner>/<repo>/<workflow-name> pattern in the sentence right after. A brand-new reader has to run the command (or trust it blindly) before understanding why the argument looks like a path. Flipping the order (explain the pattern, then show the command) would remove a beat of confusion.

3. Frontmatter jargon appears without a definition on first use on the Quick Start page ("If you changed the frontmatter..."), even though a definition is provided (linked and described later, "the YAML configuration block between the --- markers"). Since Quick Start is likely many users' first page, it would help to inline that same short definition the first time frontmatter is mentioned, rather than only in the customize-workflow step.

4. The CLI Commands page is reference-shaped, not tutorial-shaped, which is appropriate for its purpose, but there's no visible signpost on it (or from Quick Start) reminding a beginner "you don't need to read all of this — the Quick Start guide already used the only 2 commands you need." The page opens with a big table of 11 commands and an "Advanced: Enterprise" collapsible, which could feel overwhelming to someone who just wants to know "what do I run next." (There is a helpful tip pointing to #commands, but it comes after the full command table rather than before it.)

5. Left sidebar navigation is very long (Introduction → Setup → AI Engines → Guides → Gallery → Design Patterns → Practices → Reference → Specs, dozens of sub-items). For a first-time visitor this is a lot of surface area to scan before finding "Quick Start," even though it is the second item in the list — the sheer density of the nav could make the page feel more complex than the actual onboarding flow is.

🟢 What Worked Well

• Focused, single-outcome tutorial. The Quick Start doesn't try to teach every concept — it drives toward one specific, verifiable result (a "Daily Repo Report" issue appearing after 2–3 minutes), which is a great way to build confidence before diving into concepts like frontmatter or safe outputs.
• Clear prerequisites list with direct links to install/verify each dependency (gh --version, gh auth status, GitHub Actions settings).
• Fallback install path provided immediately — the standalone curl | bash installer is given as a same-page fallback right under the gh extension install command, which anticipates a very common real-world failure mode (private extension registries, auth issues).
• Multi-engine tabs (Copilot/Claude/Codex/Gemini/Pi) for authentication setup are a nice touch — a reader only needs to expand the tab for the engine they picked, keeping the page from being cluttered with irrelevant setup steps.
• "What's next?" and "Related guides" sections at the end give clear, concrete next steps instead of just ending abruptly after the tutorial.
• CLI Commands page's summary table (Command / Description / When to use) at the very top is genuinely useful as a fast-scan reference once a user is past the beginner stage.

Recommendations

Quick wins:

1. Move the "explain the <owner>/<repo>/<workflow-name> pattern" sentence to before the add-wizard command block.
2. Add one bolded sentence near the top of Prerequisites: "New here? Start with GitHub Copilot — no extra account or API key required."
3. Move the [!TIP] New to gh aw? ... jump to Commands callout on the CLI page to the very top, before the full command table.

Longer-term:

1. Inline a one-sentence definition of "frontmatter" the first time it's used on Quick Start, matching the phrasing already used later in the doc.
2. Consider a lighter "first 5 minutes" collapsed/pinned nav view for new visitors, separate from the full reference tree, so Setup/Quick Start is visually distinct from the much larger Reference section.

Screenshots

No screenshots were captured this run — the sandboxed Playwright browser could not launch due to missing shared libraries (libnspr4.so, libnss3.so) and no privileged package-install access was available to resolve it. Content review was instead performed via rendered HTML and page source.

Generated by 📝 Documentation Noob Tester · copilot · auto · 58.3 AIC · ⌖ 5.19 AIC · ⊞ 10.2K · ◷

• expires on Aug 28, 2026, 10:25 PM UTC-08:00