[docs-noob-tester] 📚 Documentation Noob Test Report - 2026-06-17

[github-actions[bot]]

Summary

• Date: 2026-06-17
• Pages visited:
  • /gh-aw/ (Home)
  • /gh-aw/setup/quick-start/ (Quick Start)
  • /gh-aw/setup/cli/ (CLI Commands)
• Overall impression: As a first-time user, I found the docs have a clean structure and a well-organized Quick Start. However, several jargon terms and a subtle formatting issue in the hero copy created friction right from the start.

---

🔴 Critical Issues Found

1. Home page intro sentence has a missing separator

The opening paragraph reads:

"Wake up to ready-to-review repository improvements-automated triage, CI insights, docs updates, and test enhancements from simple markdown workflows."

The word improvements-automated is likely meant to be improvements — automated (em dash) or improvements: automated. As written, it reads as a compound adjective which is confusing and grammatically awkward. This is the very first full sentence a new visitor reads after the hero tagline.

📎 [home-full.png] — https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/29ef24e5e6cfa32e76e74991e3cfc80a4cfd2381c51c6d939d990e76b251f0fd.png?raw=true

---

🟡 Confusing Areas

2. "PAT" is used without expansion in the Quick Start

Step 2's NOTE box says "Create a fine-grained PAT" without ever spelling out "Personal Access Token". For beginners who haven't worked with GitHub API auth before, this abbreviation is a blocker.

Recommendation: Change to "Create a fine-grained Personal Access Token (PAT)".

📎 [quick-start-token-setup.png] — https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/524a0dcfb528d410db118b74e77c137698443faa4defc6d350e29dc5c282f37b.png?raw=true

3. Two tokens to understand before anything runs

The Quick Start introduces COPILOT_GITHUB_TOKEN as "distinct from the default GITHUB_TOKEN" — but beginners likely don't know what GITHUB_TOKEN is either.

Recommendation: Add a one-liner: "GitHub Actions provides an automatic GITHUB_TOKEN for the repo, but Copilot needs its own separate token with AI access."

📎 [quick-start-prereqs.png] — https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/a092c3dba0760a3c6c6482c792eea3504e69025f70b4e200365261a711a2ef56.png?raw=true

4. add-wizard command format is not obvious

The command gh aw add-wizard githubnext/agentics/daily-repo-status uses a <owner>/<repo>/<workflow-name> format explained only after the command. A comment above the code block would make it self-documenting.

5. "frontmatter" introduced without enough inline context

Step 4 mentions frontmatter with a brief parenthetical. As a noob I'd wonder: did I change frontmatter? The example shows engine config but doesn't clearly connect it back to the file structure.

6. CLI Commands page is very long (49KB)

Excellent content but overwhelming for beginners. The "Most Common Commands" table at the top is great — a brief "start here" note for beginners vs. the rest for advanced users would help.

📎 [cli-commands.png] — https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/64d7c8d060c3eee729b4f252f138dd8b4bfffaf8ae8b9b5cd8ebaf3b52e8a8b3.png?raw=true

---

🟢 What Worked Well

• ✅ "Quick Start with CLI" CTA is immediately visible and prominent on the home page hero.
• ✅ 4-step numbered structure in the Quick Start gives a clear sense of progress.
• ✅ Prerequisites section is thorough with specific versions (gh v2.0.0+).
• ✅ Callout boxes for token setup (NOTE/TIP) contain actionable steps.
• ✅ Most Common Commands table at the top of the CLI page is excellent.
• ✅ All key linked pages return 200 — no broken links found.
• ✅ Navigation bar is clean and logical.
• ✅ "What's next?" section gives good forward momentum.

---

Recommendations

Quick wins (low effort, high impact):

1. 🔧 Fix improvements-automated → improvements — automated in home page intro
2. 🔧 Expand "PAT" to "Personal Access Token (PAT)" on first use
3. 🔧 Add a comment showing add-wizard format: # <owner>/<repo>/<workflow-name>
4. 🔧 Add a brief note clarifying GITHUB_TOKEN vs COPILOT_GITHUB_TOKEN

Medium effort:
5. 📝 Add a "Which token do I need?" decision card for Copilot/Claude/Codex/Gemini users
6. 📝 Add a "Start here" intro to the CLI page for beginners

Longer term:
7. 🎯 Interactive AI engine selection wizard on the Quick Start page

---

Screenshots

📎 [home-full.png] — https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/29ef24e5e6cfa32e76e74991e3cfc80a4cfd2381c51c6d939d990e76b251f0fd.png?raw=true
📎 [quick-start-prereqs.png] — https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/a092c3dba0760a3c6c6482c792eea3504e69025f70b4e200365261a711a2ef56.png?raw=true
📎 [quick-start-token-setup.png] — https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/524a0dcfb528d410db118b74e77c137698443faa4defc6d350e29dc5c282f37b.png?raw=true
📎 [cli-commands.png] — https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/64d7c8d060c3eee729b4f252f138dd8b4bfffaf8ae8b9b5cd8ebaf3b52e8a8b3.png?raw=true

---

References: §27667631098

Warning

Firewall blocked 5 domains

The following domains were blocked by the firewall during workflow execution:

• accounts.google.com
• android.clients.google.com
• clients2.google.com
• safebrowsingohttpgateway.googleapis.com
• www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

Generated by 📝 Documentation Noob Tester · ◷

• expires on Jun 17, 2026, 9:32 PM UTC-08:00