[docs-noob-tester] 📚 Documentation Noob Test Report - 2026-06-12

[github-actions[bot]]

Summary

• Date of test: 2026-06-12
• Pages visited:
  • https://github.github.com/gh-aw/ (Home)
  • https://github.github.com/gh-aw/setup/quick-start/ (Quick Start)
  • https://github.github.com/gh-aw/setup/cli/ (CLI Commands)
• Overall impression: Well-structured and gets you to a working state quickly, but a complete beginner will struggle with the AI account prerequisite choices and the COPILOT_GITHUB_TOKEN setup — these are the main blockers before running a first workflow.
• Workflow run: §27396194767

---

🔴 Critical Issues Found

1. AI Account prerequisite is ambiguous for new users

The prerequisites list four AI options (GitHub Copilot, Anthropic Claude, OpenAI Codex, Google Gemini) with no guidance on which is easiest or cheapest to get started with. A true beginner doesn't know:

• Which options are free vs paid?
• Which is the recommended starting point?
• Do you need an account at both GitHub AND an AI provider?

This is the very first prerequisite — if users can't satisfy it, they stop here.

📎 [confusing-prerequisites-ai-account.png] — The prerequisites section without guidance on which AI engine to pick:

2. COPILOT_GITHUB_TOKEN setup note is dense and intimidating

Step 2's inline note uses unfamiliar terms without explanation:

• "fine-grained PAT" — not explained inline
• "Account permissions → Copilot Requests → Read" — the navigation path is specific and may not match what users see
• gh secret set ... < /path/to/token.txt — implies a file-based workflow that beginners won't know how to perform

This is a critical blocker: if users can't configure the secret, the workflow silently fails.

📎 [confusing-copilot-token-setup.png] — Dense token setup note:

---

🟡 Confusing Areas

3. The add-wizard command format isn't self-explanatory

gh aw add-wizard githubnext/agentics/daily-repo-status

The <owner>/<repo>/<workflow-name> format is explained inline, but a beginner wonders: What is githubnext/agentics? Is it official? How do I find other workflows? A single sentence — "This is an official example repository maintained by GitHub" — would help.

4. .lock.yml and "frontmatter" are unexplained upfront

Step 4 says "if you changed the frontmatter" and only describes it as "the configuration block between the --- markers." For a beginner seeing YAML for the first time, this is opaque. Similarly, .lock.yml is mentioned with a link but no plain-English reassurance that you never edit it.

5. "From your repository root" assumes context

Step 2 says "from your repository root" without telling users how to navigate there (cd path/to/your-repo). Small thing, but a real trip-up for new CLI users.

6. CLI page lists 4 installation methods — overwhelming

The CLI page shows: basic install → pinning → standalone installer → GitHub Actions setup action → GHES support. A beginner only needs the first one; the rest create decision fatigue.

📎 [cli-commands.png] — Multiple installation methods on the CLI page:

---

🟢 What Worked Well

• ✅ Clear CTAs on the home page — "Quick Start with CLI" button is immediately visible and goes exactly where needed.
• ✅ ⏱️ Estimated time: 10 minutes — Sets realistic expectations.
• ✅ Numbered steps (1–4) — Linear flow is easy to follow.
• ✅ Code examples for every command — No ambiguity about what to type.
• ✅ Auth troubleshooting tip in Step 1 — Perfectly placed inline hint for auth issues at install time.
• ✅ Prerequisites list is explicit — Uses checkable commands (gh auth status, gh --version).
• ✅ "Most Common Commands" table on CLI page — Great orientation summary at the top.

📸 View full page screenshots

Home page:

Quick Start guide:

---

💡 Recommendations

Quick wins

1. Add a "Recommended for new users" callout in Prerequisites — e.g., "New to this? Start with GitHub Copilot — it integrates directly with your existing GitHub account."
2. Simplify the COPILOT_GITHUB_TOKEN note — Link to a dedicated Auth page with a step-by-step screenshot walkthrough, or link to existing GitHub docs on fine-grained PATs.
3. Explain githubnext/agentics — One sentence: "This is the official GitHub example workflows repository."

Medium-term

4. Collapse advanced installation options — Wrap pinning, standalone installer, and Actions setup action into <details> blocks on the CLI page.
5. Add a "Did it work?" verification step — After Step 3, show how to find the created issue and what a successful output looks like.
6. Define "frontmatter" and "lock file" inline — One plain-English sentence before each linked reference.

Longer-term

7. "Choose your AI engine" decision table — Compare Copilot, Claude, Gemini, Codex on free-tier availability, sign-up complexity, and recommended use cases.
8. UI-first getting started path — Not all users are comfortable with CLI; a browser-only quick start would broaden accessibility.

---

References:

• §27396194767

Warning

Firewall blocked 5 domains

The following domains were blocked by the firewall during workflow execution:

• accounts.google.com
• android.clients.google.com
• clients2.google.com
• safebrowsingohttpgateway.googleapis.com
• www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

Generated by 📝 Documentation Noob Tester · 767.8 AIC · ⌖ 14.6 AIC · ⊞ 19.1K · ◷

• expires on Jun 12, 2026, 9:38 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Documentation Noob Tester.

A newer discussion is available at Discussion #39021.