[docs-noob-tester] 📚 Documentation Noob Test Report - 2026-07-02

[github-actions[bot]]

Summary

• Date of test: 2026-07-02
• Pages visited:
  • `(localhost/redacted) (Home)
  • `(localhost/redacted) (Quick Start)
  • `(localhost/redacted) (CLI Commands)
• Overall impression: The homepage immediately communicates what the tool does, and the Quick Start guide is well-structured and gets you started in ~10 minutes. However, several beginner-unfriendly assumptions about prior knowledge create friction at key moments.

---

🔴 Critical Issues Found

1. githubnext/agentics is a mystery repository — Step 2 of Quick Start

In Step 2, the first thing a new user runs is:

gh aw add-wizard githubnext/agentics/daily-repo-status

The command refers to githubnext/agentics as if it's obvious — but a beginner will ask: "Is this my repository? A repository I need to create? Some required dependency?"

The explanation that follows ("references the daily-repo-status workflow hosted in the public githubnext/agentics examples repository") is one sentence and buried after the command. A noob might copy the command without reading ahead and be confused when prompted about permissions.

Suggestion: Add a single bolded sentence before the code block: "This uses a pre-built example workflow from GitHub's public examples repository — you don't need to create anything."

📎 [quick-start-step2.png] — https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/a4ee93b72aaa94e45044982e9a990a98314e76c92e8d26e57a0b5b7b269b9ba7.png?raw=true

2. OpenAI and Gemini users have no setup guidance in Quick Start

The prerequisites mention Codex and Gemini as options, and Step 2 says you can choose them. However, the detailed secret setup notes only cover COPILOT_GITHUB_TOKEN and ANTHROPIC_API_KEY. Users who select OpenAI or Gemini get only a link to the Authentication page with no inline callout.

Suggestion: Add equivalent [!NOTE] callouts for OPENAI_API_KEY and GEMINI_API_KEY, or add a single combined note covering all four.

---

🟡 Confusing Areas

3. "Frontmatter" used without beginner-friendly explanation

In Step 4 (Customize), the word "frontmatter" appears with only an inline link. A new user with no YAML background has no idea what this means. The link goes to the full Reference page, which is dense.

Suggestion: Add a one-line tooltip or parenthetical: "(the --- block at the top of the file that controls when and how the workflow runs)".

4. .lock.yml explanation is mid-sentence and easily skipped

After Step 2, there's a critical note about .lock.yml:

"The .lock.yml is the compiled GitHub Actions workflow generated from your markdown — it is what actually runs, and it is regenerated automatically, so you never edit it by hand."

This is important (it affects how users understand what they've just added to their repo) but it's in a plain paragraph. A beginner who skims will miss it and then be confused when they see an unfamiliar .lock.yml appear in their repo.

Suggestion: Put this in a [!NOTE] callout or a visually distinct aside.

5. CLI Commands page is overwhelming for beginners

The CLI Commands page has 40+ commands listed. While the "Most Common Commands" table at the top is excellent, there's no visual separation or "start here" guidance after that table. The GitHub Enterprise Server section appears before basic command usage examples, which is irrelevant to 95% of first-time users.

Suggestion: Add a collapsible <details> block for advanced/enterprise content, or use a "Beginner" vs "Advanced" toggle. The Most Common Commands table is perfect — just make it clearer that beginners can ignore everything else on first read.

📎 [cli-page-top.png] — https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/52dab7acc61a935a1d3f9daddaa6be4e6a64f8ecf89949f43cbdb05d65e297ce.png?raw=true

6. Step 3 shows a sample report image, but context around it is redundant

In Step 3, after describing what the Daily Repo Report contains, there's an image showing the result. But the same bullet points are listed above the image and the image comes after them — a beginner might wonder "is this image supposed to show me something new?"

Suggestion: Either make the image the primary deliverable demonstration (with less prose), or add a caption like "Here's what your new issue will look like" directly above the image.

📎 [quick-start-step3-result-image.png] — https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/8ffceef8ec25a952724771d0c9a60bdd75001823cd21f30b899c1535a7cf1465.png?raw=true

7. "Public Preview" notice is not visible on the home page

The home page includes:

i Note: GitHub Agentic Workflows is in Public Preview and may change significantly.

But this is visible only when the user scrolls past the hero section. The notice is important for setting expectations (things may break, APIs may change) and should ideally be visible without scrolling.

Suggestion: Consider placing a banner or badge near the hero title, or include it as a Starlight <Aside type="caution"> at the very top of the page content.

---

🟢 What Worked Well

• Hero tagline is clear: "Intelligent automation for GitHub. Run the coding agents you know and love, with strong guardrails and cost controls, in GitHub Actions." — this immediately communicates the value proposition.
• Quick Start CTAs are prominent: "Quick Start with CLI" is a primary button right in the hero. No hunting required.
• Time estimate builds confidence: "⏱️ Estimated time: 10 minutes" reduces fear for beginners.
• Prerequisites list is excellent: All 6 prerequisites are listed with links, version requirements, and verification commands.
• Video on Quick Start page: The embedded walkthrough video is a great addition for visual learners.
• Most Common Commands table on CLI page: This is exactly what a beginner needs — the 10 most important commands in a scannable table.
• Security architecture diagram: The mermaid diagram on the home page is a great visual for the layered security model.
• add-wizard interactive setup: The description of what the wizard does (check prereqs → select engine → set up secrets → add workflow → trigger run) is clear and reassuring.

---

Recommendations

Quick wins

1. Add one sentence before the add-wizard command explaining that githubnext/agentics is GitHub's public examples repo, not the user's own repo.
2. Add a [!NOTE] callout for OpenAI and Gemini secret setup, matching what exists for Copilot and Anthropic.
3. Define "frontmatter" inline with a brief parenthetical before linking to the reference.
4. Wrap the .lock.yml explanation in a [!NOTE] callout to make it stand out.

Medium-term improvements

5. Move the "Public Preview" notice above the fold — either as a top-of-page banner or near the hero title.
6. Add a visual separator or collapse the advanced/enterprise sections on the CLI Commands page so beginners aren't overwhelmed.

Longer-term documentation improvements

7. Create a "Learn in 5 minutes" video tour that covers all 4 Quick Start steps — the existing video is good but covers installation only.
8. Add inline "what does this mean?" expandable sections for jargon like "frontmatter", "lock file", and "safe outputs" on first use in the Quick Start guide.
9. Add a "What just happened?" section after Step 2 explaining the two new files in .github/workflows/ (the .md source and the .lock.yml compiled output).

---

Screenshots

📎 [home-hero.png] — https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/b8976cb549683cccdb4c2a962ab5e9ed7026b2835a796391c4ddf562b3865c6b.png?raw=true

📎 [quick-start.png] — https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/6d331de1b2f5d055d0f044e998503799046c70ac5ca28bb482a855afceb899cd.png?raw=true

📎 [quick-start-step2.png] — https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/a4ee93b72aaa94e45044982e9a990a98314e76c92e8d26e57a0b5b7b269b9ba7.png?raw=true

📎 [quick-start-step3-result-image.png] — https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/8ffceef8ec25a952724771d0c9a60bdd75001823cd21f30b899c1535a7cf1465.png?raw=true

📎 [cli-page-top.png] — https://github.com/github/gh-aw/blob/assets/Documentation-Noob-Tester/52dab7acc61a935a1d3f9daddaa6be4e6a64f8ecf89949f43cbdb05d65e297ce.png?raw=true

---

References:

• §28566699752

Warning

Firewall blocked 5 domains

The following domains were blocked by the firewall during workflow execution:

• accounts.google.com
• android.clients.google.com
• clients2.google.com
• safebrowsingohttpgateway.googleapis.com
• www.google.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "accounts.google.com"
    - "android.clients.google.com"
    - "clients2.google.com"
    - "safebrowsingohttpgateway.googleapis.com"
    - "www.google.com"

See Network Configuration for more information.

Generated by 📝 Documentation Noob Tester · 77.3 AIC · ⌖ 14.3 AIC · ⊞ 1.8K · ◷

• expires on Jul 2, 2026, 9:14 PM UTC-08:00