[docs-noob-tester] 📚 Documentation Noob Test Report - 2026-06-10

[github-actions[bot]]

Documentation Noob Test Report

Tested by an automated "first-time user" agent navigating the getting-started experience.

Summary

• Date: 2026-06-10
• Pages visited: Home, Quick Start, CLI Commands
• Overall impression: The Quick Start guide has a clear step-by-step structure with a video and estimated time, which is helpful. However, a first-time user encounters jargon terms and ambiguous prerequisites that could cause friction before running their first command.

---

🔴 Critical Issues Found

1. No visual of the add-wizard interactive process

Step 2 says "This will take you through an interactive process" but shows no terminal output, expected prompts, or sample responses. A beginner has no idea what to expect. A static screenshot of the wizard prompts inline would help significantly.

📎 [quick-start-step2.png]

2. "Peli's Agent Factory" — who is Peli?

The "What's next?" section links to "Peli's Agent Factory" with no context. A new user will be confused by this internal reference.

---

🟡 Confusing Areas

1. AI Account prerequisite is ambiguous

Lists four AI providers but doesn't say which has a free tier, which is easiest, or what account type is needed.

📎 [quick-start-prerequisites.png]

2. COPILOT_GITHUB_TOKEN vs GITHUB_TOKEN confusion

Two GitHub tokens with similar names is confusing. The reason for a separate token should be explained more prominently — perhaps with a "Why?" expandable section.

3. "Getting Started" section buried at bottom of home page

The home page flows through Key Features → Guardrails → Cost Management → Example → Gallery before reaching "Getting Started". Beginners who scroll instead of clicking the hero button may get lost.

📎 [home-getting-started-buried.png]

4. CLI page has GHES content too early

"GitHub Enterprise Server Support" appears within the first 5 sections — noise for the majority of github.com users.

5. No cost context in Quick Start

Beginners will wonder: "Does this cost money?" A single sentence with a link to Cost Management would set expectations.

6. Early development warning is discouraging

The home page blockquote "use it with caution, and at your own risk" may put off new users. A softer [!NOTE] callout might be less alarming.

📎 [home.png]

---

🟢 What Worked Well

1. Estimated time (10 minutes) — immediately reassuring.
2. Step-by-step numbered structure — clear progression.
3. Video on Quick Start — great for visual learners.
4. Links to every prerequisite — direct links to install pages and settings.
5. CLI Commands table — the "Most Common Commands" table is an excellent quick reference.

📎 [cli-commands-table.png]

6. Inline frontmatter explanation — describing it as "the config block between --- markers" is much better than jargon alone.
7. Daily Repo Status Report example — concrete, relatable output makes the value tangible.
8. Troubleshooting callout in Step 2 — exactly where a stuck beginner needs help.

📎 [quick-start.png]

---

Recommendations

Quick Wins

1. Add a terminal screenshot of add-wizard interactive prompts inline in Step 2.
2. Add one sentence of context for "Peli's Agent Factory".
3. Indicate which AI provider is easiest to start with or has a free tier.
4. Add a cost note in Quick Start with a link to Cost Management.

Medium-term

5. Collapse GHES content on the CLI page for enterprise users.
6. Soften the early-development warning with a [!NOTE] callout.
7. Add a "Why a separate Copilot token?" expandable explanation.

Longer-term

8. Add expected terminal output after each CLI command so beginners know what success looks like.
9. Move Getting Started higher on the home page, or add a secondary CTA after Key Features.

---

References: §27254598667

Warning

Firewall blocked 5 domains

The following domains were blocked by the firewall during workflow execution:

• accounts.google.com
• android.clients.google.com
• clients2.google.com
• safebrowsingohttpgateway.googleapis.com
• www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

Generated by 📝 Documentation Noob Tester · ⌖ 26.3 AIC · ⊞ 19.1K · ◷

• expires on Jun 10, 2026, 9:27 PM UTC-08:00