[docs-noob-tester] 📚 Documentation Noob Test Report - 2026-06-08

[github-actions[bot]]

Summary

• Date of test: 2026-06-08
• Pages visited: Home, Quick Start, CLI Commands
• Overall impression: The site looks polished and the concept is compelling, but several steps in the Quick Start assume significant prior knowledge that could block a true beginner.

---

🔴 Critical Issues Found

1. COPILOT_GITHUB_TOKEN setup is confusing

The Quick Start says "Create a fine-grained PAT... set Copilot Requests to Read" — too much jargon. Missing: the full navigation path in GitHub Settings, what distinguishes this from GITHUB_TOKEN, and why a separate token is needed.

📎 [quick-start-auth.png] — https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/4652634437bdc2080f077b0b5c21c1e21fa59b2556c79bac043bf0bc50add0cd.png?raw=true

2. No guidance on which AI engine to choose

Four providers are listed (Copilot, Claude, Codex, Gemini) with no recommendation for beginners. A new user doesn't know which is free/paid or easiest to set up. A simple "If you have GitHub Copilot, start here" would help enormously.

📎 [quick-start.png] — https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/e2ce3254095e585d13f428c0626e545a0f3a15cf4b737d585400dc42cbbea0d9.png?raw=true

3. Typo in sidebar: "Organziations" (x2)

Both "Safe Rollout in Organziations" and "Sharing Workflows in Organziations" are misspelled. Typos in navigation erode trust with new users.

---

🟡 Confusing Areas

4. Unexplained jargon: frontmatter and .lock.yml

Step 4 uses the term frontmatter and warns not to edit .lock.yml, but never gives a simple mental model. Suggestion: add a brief callout early in Quick Start — "Workflows are Markdown files with a YAML config block called frontmatter. A .lock.yml is auto-generated — never edit it directly."

5. add-wizard workflow format is not intuitive

gh aw add-wizard githubnext/agentics/daily-repo-status uses <owner>/<repo>/<workflow-name> format. A beginner wonders: "Where does this workflow come from? Can I use any GitHub repo?" A link to a workflow gallery would help.

6. CLI page lists 50+ commands — overwhelming

The page depth is daunting for a first-timer. The "Most Common Commands" table at the top is great, but it needs more visual prominence (e.g., a callout box) to help beginners skip the rest.

📎 [cli-common-cmds.png] — https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/0fcbe9930493464c5bc46d23abf51b71baa8bea43ae0f3c2689ea1edafcc9940.png?raw=true

---

🟢 What Worked Well

• Clear hero message: "Repository automation, running the coding agents you know and love" immediately explains the value.
• Estimated time (10 min): Sets realistic expectations up front.
• Step-by-step structure: Install → Add → Wait → Customize is logical.
• Most Common Commands table: Top 10 commands at the top of CLI page is great.
• Safety-first messaging: The guardrails section builds confidence that AI won't go rogue.

📎 [home.png] — https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/151bbb699b521164856514f731b5308794f8285268f23f95755c766a2b633284.png?raw=true
📎 [cli.png] — https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/f984532bc22c8b780a343e4080869d8406cb55026c8bc94a16ebcfeb7e965398.png?raw=true

---

Recommendations

Quick wins:

1. 🔧 Fix the "Organziations" typo in the sidebar (2 locations)
2. 🔧 Add full navigation path for creating COPILOT_GITHUB_TOKEN in GitHub Settings
3. 🔧 Add a recommendation: "New here? If you have GitHub Copilot, that's the easiest starting point."
4. 🔧 Add a short callout near the top of Quick Start defining frontmatter and .lock.yml

Longer-term:
5. 📖 Link to a workflow gallery/catalog from the add-wizard step
6. 📖 Make "Most Common Commands" visually distinct on the CLI page
7. 📖 Add a beginner learning path that surfaces only essential docs

---

References:

• §27117528125

Warning

Firewall blocked 5 domains

The following domains were blocked by the firewall during workflow execution:

• accounts.google.com
• android.clients.google.com
• clients2.google.com
• safebrowsingohttpgateway.googleapis.com
• www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

Generated by 📝 Documentation Noob Tester · ⌖ 14.8 AIC · ⊞ 18.9K · ◷

• expires on Jun 8, 2026, 9:35 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Documentation Noob Tester.

A newer discussion is available at Discussion #38034.