📚 Documentation Noob Test Report - 2026-04-01

[github-actions[bot]]

Test conducted by: Copilot coding agent acting as a brand-new user
Date: 2026-04-01
Workflow run: §23850471341
Note: Playwright visual screenshots were unavailable due to network isolation in this environment. All analysis performed via curl + HTML text extraction.

Pages Visited

#	Page	URL	Status
1	Home	/gh-aw/	✅ 200
2	Quick Start	/gh-aw/setup/quick-start/	✅ 200
3	CLI Commands	/gh-aw/setup/cli/	✅ 200

Overall First Impression

As a complete beginner, the home page does a good job of conveying what GitHub Agentic Workflows does — "repository automation, running the coding agents you know and love." The hero section has a clear "Quick Start with CLI" CTA. The Quick Start guide is well-structured with numbered steps and clear commands. However, there are several beginner-blocking gaps detailed below.

---

🔴 Critical Issues (Block Getting Started)

1. "frontmatter" used without explanation in Step 4 of Quick Start

Step 4 says:

"If you have changed the frontmatter, regenerate the workflow YAML…"

This is the first mention of "frontmatter" in the guide without a tooltip, link, or inline definition. A brand-new user who has never seen YAML front matter will have no idea what this means or whether they've "changed" it. The glossary does define it, but there's no link at the point of use.

Suggestion: Add a parenthetical or tooltip: "(the --- YAML section at the top of the .md file)", and/or link to the glossary definition.

---

2. githubnext/agentics/daily-repo-status — what is githubnext/agentics?

Step 2 tells users to run:

gh aw add-wizard githubnext/agentics/daily-repo-status

But nowhere does it explain:

• What githubnext/agentics is (a separate GitHub repository of example workflows)
• Whether this is official
• Where to browse other available workflows

A new user may wonder: "Is this a local path? Is this the GitHub org githubnext? Can I find more workflows there?"

Suggestion: Add a one-sentence explanation: "githubnext/agentics is an official collection of pre-built workflow templates hosted at github.com/githubnext/agentics. Browse it to find more ready-made workflows."

---

3. Typo: "blacklog" → "backlog" in Step 4

Step 4, second bullet point:

"your issue blacklog"

This is a spellling error — should be backlog.

Fix: Change blacklog → backlog in docs/src/content/docs/setup/quick-start.mdx line ~101.

---

🟡 Confusing Areas (Slow Down Learning)

4. Prerequisites mention "AI Account" but don't explain what's needed for free vs. paid

The prerequisites list:

"AI Account - GitHub Copilot, Anthropic Claude or OpenAI Codex"

A new user doesn't know:

• Does GitHub Copilot Free work, or is a paid plan needed?
• Can I try this without a paid subscription?
• What's the minimum plan required?

Suggestion: Add a note clarifying which Copilot tiers work (e.g., "GitHub Copilot Individual or higher") and link to the engines page.

---

5. Step 2 mentions three secrets but doesn't tell you which to pick

"Set up the required secret — COPILOT_GITHUB_TOKEN, ANTHROPIC_API_KEY or OPENAI_API_KEY"

The secrets are listed as alternatives but there's no guidance on:

• Which is easiest to set up for a beginner?
• If you chose Copilot in the "Select an AI Engine" step, which secret applies?
• Where exactly do you add the secret (the interactive wizard presumably handles this, but it's unclear)?

Suggestion: Clarify that the wizard guides you through this, or add: "The wizard will prompt you — each engine has its own secret. If you chose Copilot, you'll need a COPILOT_GITHUB_TOKEN."

---

6. CLI page is overwhelming for newcomers — the "Most Common Commands" table is good, but it's buried under GHE/Enterprise content

The CLI page starts well with a "Most Common Commands" table, but quickly dives into:

• Pinning to specific versions
• Standalone installer
• GitHub Actions Setup Action
• GitHub Enterprise Server configuration (very long section)

A complete beginner following the Quick Start doesn't need GHE content. The advanced sections aren't collapsible or visually separated from beginner content.

Suggestion:

• Add a "📖 Getting Started?" callout at the top of the CLI page pointing to Quick Start
• Move GHE / Enterprise content to a dedicated sub-page or collapsible <details> section
• The "Most Common Commands" table is great — consider making it more prominent

---

7. Home page "Gallery" section links to blog posts but isn't immediately clear

The Gallery section lists workflow categories like "Issue & PR Management", "Continuous Documentation" etc., which are good. But clicking "Examples" in the nav takes you to #gallery which is a div on the index page — not a dedicated examples page. Newcomers may expect a proper examples catalog.

---

🟢 What Worked Well

✅ Home page headline is clear and compelling: "repository automation, running the coding agents you know and love, with strong guardrails in GitHub Actions."

✅ Early warning note about the experimental nature is responsible and builds trust:

"GitHub Agentic Workflows is in early development…Use it with caution, and at your own risk."

✅ Step-by-step Quick Start is well-organized with numbered sections. The estimated time (10 minutes) sets expectations nicely.

✅ Video in Quick Start — having a video demo at the top of the Quick Start is excellent for visual learners. It works correctly (video file exists and is properly linked).

✅ Screenshot of example output — showing what the "Daily Repo Report" looks like sets clear expectations for what users will get.

✅ Tip boxes for authentication issues and troubleshooting are well-placed.

✅ "What's next?" section at the end of Quick Start gives good direction for where to go after the first workflow.

✅ "Most Common Commands" table at the top of the CLI page is helpful for orientation.

✅ All key links work — no 404s found on any of the navigation links tested (9/9 checked).

✅ Glossary exists and covers key terms (frontmatter, compilation, etc.) — it just needs to be linked at point-of-use more often.

---

Recommendations (Prioritized)

🚀 Quick Wins (< 1 hour each)

1. Fix typo: blacklog → backlog in quick-start.mdx:101
2. Link "frontmatter" in Step 4 to the glossary or add an inline tooltip/definition
3. Explain githubnext/agentics with a one-sentence description and link in Step 2
4. Add Copilot tier guidance to the AI Account prerequisite

📋 Short-Term (1-2 days)

5. Clarify the secrets step in Quick Start — explain that the wizard handles it, or map engine → secret clearly
6. Add a callout on the CLI page linking back to Quick Start for beginners
7. Collapse GHE/Enterprise content on CLI page into <details> or a sub-page

🔭 Longer-Term

8. Create a dedicated Examples/Gallery page instead of the #gallery anchor-link approach
9. Add Copilot tier/pricing clarity to the prerequisites section with links

---

References:
§23850471341

AI generated by Documentation Noob Tester · history

• expires on Apr 2, 2026, 1:27 PM UTC

[github-actions[bot]]

This discussion has been marked as outdated by Documentation Noob Tester.

A newer discussion is available at Discussion #24114.