[claude-code-user-docs-review] 🔍 Claude Code User Documentation Review - 2026-05-22

[github-actions[bot]]

Executive Summary

A Claude Code user can adopt gh-aw, but the documentation continues to treat Copilot as the default and the path-of-least-resistance, with Claude/Codex/Gemini sitting as second-class citizens despite being prominently listed as supported engines. The core mechanics (auth, frontmatter, gh aw add-wizard engine selection) do work for Claude users, but feature reference pages and gh aw init itself still bake in Copilot assumptions without flagging the implication for non-Copilot users.

Key Finding: Adoption is viable but friction-heavy — 88% of in-doc engine examples (62 of 71) target engine: copilot, and major reference pages (Serena, audit, imports, threat-detection, web-search, mcp-scripts, qmd, sandbox, templating) show only Copilot examples. The same four major obstacles flagged yesterday remain unresolved, with only a small +1 movement in Claude example coverage (6 → 7).

Trend vs. prior run (2026-05-21): Overall score unchanged at 6/10. Blockers unchanged at 0 critical / 4 major / 6 minor. Claude auth URL inconsistency still present. Custom-engine examples still 0.

---

Persona Context

I reviewed this documentation as a developer who:

• ✅ Uses GitHub for version control
• ✅ Uses Claude Code as primary AI assistant
• ❌ Does NOT use GitHub Copilot
• ❌ Does NOT use Copilot CLI
• ❌ Does NOT have Copilot subscription

---

Question 1: Onboarding Experience

Can a Claude Code user understand and get started with gh-aw?

Yes, but with avoidable friction. The Quick Start (docs/src/content/docs/setup/quick-start.mdx) lists Anthropic Claude as a peer in Prerequisites (line 29) and gh aw add-wizard step 2 (line 70) prompts the user to "Select an AI Engine — Choose between Copilot, Claude, Codex, or Gemini." A dedicated Claude callout box at lines 82-86 explains how to set ANTHROPIC_API_KEY. That is genuinely helpful and represents real progress.

However, three specific issues persist for non-Copilot users:

Issue 1 — Copilot is the silent default everywhere outside Quick Start. engines.md:24: "Copilot CLI is the default — engine: can be omitted when using Copilot." how-they-work.mdx:26: "Workflows support GitHub Copilot (default)..." A Claude user must remember to set engine: claude in every workflow they touch; omitting the field silently picks Copilot.

Issue 2 — gh aw init installs Copilot-specific assets without warning. Per cli.md:134, init "creates the dispatcher agent file (.github/agents/agentic-workflows.agent.md)". Per custom-agent-for-aw.mdx:36, that file is explicitly "a Copilot agent file that registers the /agent agentic-workflows command in Copilot Chat." A Claude-only user runs gh aw init (the recommended first step) and ends up with a Copilot agent registered in their repo. No flag or doc warning explains that this file is unused for engine: claude workflows.

Issue 3 — Step 4's customization example uses Claude as a one-off. Quick Start line 127-131 shows engine: claude as a teaching example for changing the engine, but the linked downstream docs (frontmatter.md, creating-workflows.mdx) continue to show Copilot. This creates a discoverability cliff after Step 4.

Recommended Fixes:

• Add a one-paragraph "Using Claude" callout in how-they-work.mdx next to the engine list explaining that the default is Copilot and how to switch globally.
• Have gh aw init --no-mcp documentation (or a new --no-copilot-agent flag) note that the dispatcher file is Copilot-specific and skippable for other engines.
• Document a recommended engine: claude starter workflow in creating-workflows.mdx so the customization story doesn't end at Step 4.

---

Question 2: Inaccessible Features for Non-Copilot Users

What features or steps don't work without Copilot?

From the engine feature comparison table at engines.md:34-45:

Features That Require Copilot (4 features):

• engine.agent (custom agent file via .github/agents/) — Copilot-only
• engine.harness (custom harness script) — Copilot-only
• max-continuations (autopilot-style consecutive runs) — Copilot-only
• BYOK provider routing via COPILOT_PROVIDER_* env vars — Copilot-only

Features Only Available to Claude:

• max-turns (chat-iteration cap for long reasoning) — Claude-only

Features That Work Without Copilot (engine-agnostic):

• tools.edit, tools.github, tools.bash, tools.web-fetch, tools.playwright, tools.cache-memory, tools.repo-memory, tools.qmd, tools.agentic-workflows, tools.cli-proxy, mcp-servers
• All safe-outputs (create_issue, create_pull_request, add_comment, add_labels, etc.)
• Threat detection pipeline, content sanitization, secret redaction
• Strict mode, network firewall, MCP gateway
• All observability (gh aw logs, gh aw audit, gh aw status, gh aw health)
• engine.api-target (custom endpoint) and engine.bare work across Copilot/Claude/Codex/Gemini

Engine-Specific Tool Caveats:

• tools.web-search for Codex requires explicit web-search: opt-in (else -c web_search="disabled")
• Unrestricted bash (*, [":*"], null) on Claude silently flips it into bypassPermissions mode — important security detail buried in engines.md
• Tool allowlist not supported on Crush/OpenCode (per engines.md:45) — operationally significant for security-minded Claude users considering migration

Missing Documentation:

• No engine.harness equivalent explained for Claude users who want custom prompt-loading behavior — gap is acknowledged in the comparison table but no "workaround" or alternative pattern is suggested.
• No engine.agent analog for Claude — Claude Code itself supports subagents, but gh-aw doesn't surface this. A note explaining "Claude doesn't use custom agent files; use inline-sub-agents instead" would close the gap.
• The 41 Claude sample workflows in pkg/cli/workflows/ (down from a methodology-driven 109 last run) are not promoted in user-facing docs.

---

Question 3: Documentation Gaps and Assumptions

Where does the documentation assume Copilot usage?

Copilot-Centric Language Found In:

• docs/src/content/docs/introduction/how-they-work.mdx:26 — "Workflows support GitHub Copilot (default), Claude by Anthropic, Codex, and Gemini." The parenthetical "(default)" frames Copilot as the canonical engine.
• docs/src/content/docs/reference/engines.md:24 — "Copilot CLI is the default — engine: can be omitted when using Copilot." A symmetric "if using Claude, set engine: claude" is absent.
• docs/src/content/docs/reference/engines.md:28 — "Copilot supports the broadest gh-aw feature set, including custom agents and autopilot-style continuations" — accurate but reads as a recommendation toward Copilot.
• docs/src/content/docs/introduction/architecture.mdx:228 — example uses engine: copilot with no peer Claude example.
• docs/src/content/docs/setup/cli.md:134 — gh aw init description mentions the Copilot-specific dispatcher agent file as a standard outcome of init, without flagging engine-relevance.
• docs/src/content/docs/reference/custom-agent-for-aw.mdx:36 — Confirms the dispatcher is a "Copilot agent file" but this disclosure lives buried in a reference page, not in the init flow.
• 11 major reference pages contain ONLY engine: copilot examples and no Claude equivalents: serena.md, audit.md, imports.md, threat-detection.md, copilot-custom-agents.md, web-search.md, sandbox.md, templating.md, mcp-scripts.md, qmd.md, inline-sub-agents.md.

Missing Alternative Instructions:

• No engine: claude worked-example in threat-detection.md, mcp-scripts.md, inline-sub-agents.md, or templating.md — these are advanced features any Claude user would need to use.
• No "Claude API key best practices" section (rotation, budgeting, monitoring) parallel to the Copilot rate-limiting docs.
• Crush, OpenCode, and Pi listed in engines.md:20-22 but no per-engine auth section in auth.mdx.

---

Severity-Categorized Findings

🚫 Critical Blockers (Score: 0/10) — UNCHANGED

No critical blockers. A Claude user can:

• Install the extension (gh extension install github/gh-aw)
• Add a workflow (gh aw add-wizard prompts for engine selection)
• Configure ANTHROPIC_API_KEY (documented in Quick Start)
• Set engine: claude in frontmatter (documented)
• Run workflows and audit them

⚠️ Major Obstacles (Score: 4/10) — UNCHANGED

Obstacle 1: Default engine is Copilot, with omission penalty

Impact: Every workflow file that omits engine: silently picks Copilot. A Claude user copying any of the 11 reference-page examples ends up with a Copilot workflow unless they remember to override.

Current State: engines.md:24 codifies the default; how-they-work.mdx:26 reinforces it.

Why It's Problematic: This is a foot-gun for the persona. A Claude-Code user who installs the extension fresh, copies a threat-detection.md example, and runs gh aw compile will silently get a Copilot workflow that fails at runtime due to missing COPILOT_GITHUB_TOKEN.

Suggested Fix: Make engine: required in gh aw new templates (it already injects engine: when --engine flag is passed — make it the default). Alternatively, add a compile-time warning when engine: is omitted.

Affected Files: engines.md, how-they-work.mdx, all 11 reference pages listed in Question 3.

Obstacle 2: `gh aw init` installs Copilot-specific dispatcher silently

Impact: Running the recommended first command (gh aw init) commits a .github/agents/agentic-workflows.agent.md file that's explicitly a Copilot agent and has no effect on Claude/Codex/Gemini engines. A non-Copilot user gets a dead-weight file committed to their repo without consent.

Current State: cli.md:134 describes the dispatcher creation neutrally; only custom-agent-for-aw.mdx:36 discloses that it's a Copilot agent.

Suggested Fix: Add a --no-copilot-agent flag (or repurpose --no-mcp documentation) and a doc paragraph: "If you don't use Copilot, run gh aw init --no-copilot-agent. The dispatcher file registers a /agent agentic-workflows command in Copilot Chat and has no effect on other engines."

Affected Files: setup/cli.md, setup/creating-workflows.mdx:118, reference/custom-agent-for-aw.mdx.

Obstacle 3: Claude auth URL inconsistency across three docs pages

Impact: A Claude user gets three different URLs for "where to create an API key" depending on which page they read. Likely outcome: confusion about which Anthropic URL is canonical.

Current State (quoted):

• auth.mdx:109 — "Create an API key at (platform.claude.com/redacted) (a docs landing page, NOT the keys page)
• auth.mdx:199 — "Verify the key is active in the Anthropic Console"
• quick-start.mdx:84 — "Create an API key in Anthropic Console"

Suggested Fix: Pick one canonical URL — https://console.anthropic.com/settings/keys is the correct keys page — and use it consistently. Quick-start has it right; auth.mdx has it wrong.

Affected Files: reference/auth.mdx (lines 109 and 199).

Obstacle 4: Major reference pages show only Copilot examples

Impact: 11 reference pages (serena, audit, imports, threat-detection, copilot-custom-agents, web-search, sandbox, templating, mcp-scripts, qmd, inline-sub-agents) show only engine: copilot code blocks. Claude users have to mentally translate every example, with no certainty the translation is valid.

Current State: 62 copilot example references in fenced code blocks vs. 7 claude references across the entire docs tree.

Suggested Fix: Adopt a tabbed example pattern (e.g., the Starlight <Tabs> component) so each reference page can show the same example under copilot/claude/codex tabs. As an interim, add a single "works the same with engine: claude" footer line to each page where the feature is engine-agnostic.

Affected Files: serena.md, audit.md, imports.md, threat-detection.md, copilot-custom-agents.md, web-search.md, sandbox.md, templating.md, mcp-scripts.md, qmd.md, inline-sub-agents.md.

💡 Minor Confusion Points (Score: 6/10) — UNCHANGED

• Crush/OpenCode/Pi listed in engines.md but absent from auth.mdx — User table-scans the engine list, picks Crush, then has nowhere to look up its secret. File: reference/auth.mdx.
• Pi note "switches to provider-specific secret when model: uses provider/model format" doesn't name the secret — Vague, leaves the user guessing. File: reference/engines.md:22.
• engine: custom listed as supported but 0 examples in entire repo — Phantom feature. File: repository-wide.
• Architecture.mdx mermaid diagram (reference/architecture.mdx:279) lists only "Copilot, Claude, Codex" — omits Gemini despite Gemini being listed elsewhere as a peer.
• Default-implicit Copilot examples in code blocks that omit engine: reinforce Copilot dominance without showing it explicitly. File: quick-start.mdx, how-they-work.mdx.
• "GitHub Copilot, Claude (Anthropic), Codex (OpenAI), and Gemini (Google)" in README:40 — order consistently puts Copilot first across all docs; not wrong, but cumulative effect signals hierarchy.

---

Engine Comparison Analysis

Available Engines

Based on reference/engines.md, gh-aw supports seven engines:

• engine: copilot (default) — ⭐⭐⭐⭐⭐ docs / ⭐⭐⭐⭐⭐ examples / ⭐⭐⭐⭐⭐ auth
• engine: claude — ⭐⭐⭐⭐ docs / ⭐⭐ examples / ⭐⭐⭐⭐ auth (URL inconsistency)
• engine: codex — ⭐⭐⭐ docs / ⭐⭐ examples / ⭐⭐⭐⭐⭐ auth
• engine: gemini — ⭐⭐ docs / ⭐ examples / ⭐⭐⭐ auth
• engine: crush (experimental) — ⭐⭐ docs / ⭐ examples / ⭐⭐ auth
• engine: opencode (experimental) — ⭐⭐ docs / ⭐ examples / ⭐⭐ auth
• engine: pi (experimental) — ⭐⭐ docs / ⭐ examples / ⭐⭐ auth

Documentation Quality by Engine

Engine	Setup Docs	Examples	Auth Docs	Overall Score
Copilot	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐ (62 in docs)	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐
Claude	⭐⭐⭐⭐	⭐⭐ (7 in docs)	⭐⭐⭐⭐	⭐⭐⭐
Codex	⭐⭐⭐	⭐ (1 in docs)	⭐⭐⭐⭐⭐	⭐⭐⭐
Gemini	⭐⭐	⭐ (1 in docs)	⭐⭐⭐	⭐⭐
Custom (crush/opencode/pi)	⭐⭐	⭐ (0 in docs)	⭐⭐	⭐⭐

---

Tool Availability Analysis

Tools Review

From reference/tools.md and reference/engines.md:

Engine-Agnostic Tools (work identically for Claude users):

• edit, github, playwright, cache-memory, repo-memory, qmd, agentic-workflows, cli-proxy, mcp-servers, web-fetch

Engine-Specific Tool Behavior:

• web-search — Codex has native support; Claude/Copilot/Gemini/Crush/OpenCode/Pi all go through a third-party MCP server (e.g., Tavily). Documented in tools.md:65-67 and engines.md:52.
• bash — Documented as engine-agnostic, but engines.md warns that unrestricted bash on Claude flips it into bypassPermissions mode, silently bypassing allowlist enforcement. Security-significant for Claude users.
• Tool timeouts — Claude default 60s, Codex default 120s (per tools.md:155).

Unclear/Undocumented:

• qmd: example uses engine: copilot only — no statement that it works on Claude, though no engine restriction is mentioned either.

No top-level frontmatter tool block is named after an engine (no copilot: or claude: tool keys exist).

---

Authentication Requirements

Current Documentation

Quick Start guide covers authentication for:

• ✅ Copilot — detailed, pre-filled PAT URL, explicit fine-grained PAT requirement
• ✅ Claude — callout box with gh secret set ANTHROPIC_API_KEY command
• ❓ Codex — link-only treatment in Step 2; no inline callout box
• ❓ Gemini — link-only treatment in Step 2; no inline callout box
• ❌ Crush/OpenCode/Pi — not covered

Missing for Claude Users

• Canonical Anthropic URL not consistent across three doc pages (see Major Obstacle 3).
• No guidance on Anthropic API key budgeting, rotation, or monitoring beyond "verify the key is active."
• No worked example of engine.api-target for routing to Claude via a custom endpoint (e.g., a corporate proxy).

Secret Names (Documented)

• Copilot: COPILOT_GITHUB_TOKEN (must be fine-grained PAT; GitHub Apps/OAuth tokens explicitly unsupported)
• Claude: ANTHROPIC_API_KEY (only; CLAUDE_CODE_OAUTH_TOKEN explicitly ignored — good explicit guidance)
• Codex: OPENAI_API_KEY (primary) or CODEX_API_KEY (alternative; runtime tries CODEX_API_KEY first)
• Gemini: GEMINI_API_KEY
• Crush/OpenCode/Pi: COPILOT_GITHUB_TOKEN by default (Pi can switch to a "provider-specific secret" but the name is unspecified)

---

Example Workflow Analysis

Workflow Count by Engine

In docs/src/content/docs/ (user-facing examples):
  engine: copilot - 62 occurrences
  engine: claude  -  7 occurrences
  engine: codex   -  1 occurrence
  engine: gemini  -  1 occurrence
  engine: custom  -  0 occurrences

In pkg/cli/workflows/ (test/sample fixtures):
  engine: copilot - 117 workflows
  engine: claude  -  41 workflows
  engine: codex   -  13 workflows
  engine: gemini  -   0 workflows
  no engine field -  13 workflows (rely on default copilot)

Quality of Examples

Copilot Examples: Comprehensive across all features. Every reference page has at least one Copilot example.

Claude Examples: Concentrated in setup/quick-start, setup/cli, reference/faq, reference/network, reference/frontmatter-hash-specification, and the research-plan-assign-ops blog post. Conspicuously absent from threat-detection, mcp-scripts, inline-sub-agents, templating, sandbox, qmd, web-search, audit, serena, imports, and copilot-custom-agents reference pages.

Codex/Gemini Examples: Effectively non-existent in user-facing docs (1 each).

---

Recommended Actions

Priority 1: Critical Documentation Fixes

1. Reconcile Claude auth URLs to https://console.anthropic.com/settings/keys — Pick one canonical URL across reference/auth.mdx (lines 109 and 199) and setup/quick-start.mdx (line 84). Cosmetically tiny, but it's been flagged three runs in a row and is the single most likely point of new-user confusion.
2. Document the Copilot-specific dispatcher file in gh aw init help text — Add a one-line note to setup/cli.md:134 and setup/creating-workflows.mdx:118 explaining that the dispatcher agent file is Copilot-specific and irrelevant for other engines.
3. Add engine: claude examples to the 4 highest-traffic reference pages — threat-detection.md, mcp-scripts.md, inline-sub-agents.md, templating.md are advanced features Claude users will hit early.

Priority 2: Major Improvements

1. Add Crush, OpenCode, and Pi sections to reference/auth.mdx — They're in the engines table but absent from auth docs.
2. Adopt a tabbed code-example component for engine-agnostic features so each reference page can show copilot/claude/codex side-by-side.
3. Add a "Choosing your engine" decision guide at the top of reference/engines.md — A short table of "use copilot if X, use claude if Y, use codex if Z" would help users self-select instead of defaulting.

Priority 3: Nice-to-Have Enhancements

1. Promote the 41 Claude sample workflows from pkg/cli/workflows/ to a user-facing examples gallery — They exist but are invisible.
2. Document engine: custom properly or remove it from the engines list — Zero examples is worse than not advertising it.
3. Fix reference/architecture.mdx:279 mermaid diagram to include Gemini.

---

Positive Findings

What Works Well

• ✅ Quick Start has an explicit ANTHROPIC_API_KEY callout box (line 82-86) and an engine: claude customization example (line 127-131) — this is real progress over prior runs.
• ✅ add-wizard step 2 explicitly prompts the user to choose between Copilot, Claude, Codex, or Gemini (line 70) — interactive flow respects the persona.
• ✅ gh aw secrets bootstrap auto-detects engines in use and prompts only for missing secrets — engine-aware tooling (setup/cli.md:241).
• ✅ reference/engines.md has a clear feature-comparison table showing exactly which features work on which engines.
• ✅ Claude OAuth/subscription-token rejection is explicitly documented at auth.mdx:121-123 — saves users from a common failure mode.
• ✅ engine.api-target (custom endpoint) is documented as working across Copilot/Claude/Codex/Gemini — feature parity for advanced routing.
• ✅ Security architecture (introduction/architecture.mdx) is engine-agnostic and works equally well for Claude users — SafeOutputs, threat detection, AWF, secret redaction all apply.
• ✅ gh aw new --engine claude injects the engine into the generated template (setup/cli.md:202-216) — good CLI ergonomics for Claude-first users.

---

Conclusion

Can Claude Code Users Successfully Adopt gh-aw?

Answer: Yes, with significant effort.

Reasoning: Every gh-aw capability a Claude user needs is technically available and documented somewhere — the auth flow works, the engine selector works, the security model is engine-agnostic, and 41 Claude sample workflows exist in the repo. But the documentation continues to treat Claude as a second-class citizen relative to Copilot, with Copilot as the silent default, Copilot-only sample code on 11 major reference pages, and a Copilot-specific dispatcher file silently committed on gh aw init. A motivated Claude user will succeed, but only after translating Copilot examples in their head and noticing that several files in their repo are irrelevant to their setup.

The four major obstacles identified in prior runs (2026-05-20 and 2026-05-21) remain unaddressed in this review window. The only positive trend is one additional Claude example in the docs (6 → 7).

Overall Assessment Score: 6/10 (unchanged from prior run)

Breakdown:

• Clarity for non-Copilot users: 6/10
• Claude engine documentation: 7/10
• Alternative approaches provided: 6/10
• Engine parity: 5/10

Next Steps

The Priority 1 actions are all low-effort, low-risk doc edits with high return-on-keystroke. Reconciling the three Anthropic URLs alone would close the most-recurring finding across the last three review runs. After that, adding engine: claude examples to four high-traffic reference pages (threat-detection, mcp-scripts, inline-sub-agents, templating) would meaningfully narrow the example gap.

If the maintainers want a structural fix rather than incremental edits, adopting a tabbed engine-example component (Starlight <Tabs>) across reference pages would solve the example-parity problem permanently and scale to future engines.

---

Appendix: Files Reviewed

Complete List of Documentation Files Analyzed

• README.md
• docs/src/content/docs/setup/quick-start.mdx
• docs/src/content/docs/setup/cli.md
• docs/src/content/docs/setup/creating-workflows.mdx
• docs/src/content/docs/introduction/how-they-work.mdx
• docs/src/content/docs/introduction/architecture.mdx
• docs/src/content/docs/reference/tools.md
• docs/src/content/docs/reference/engines.md
• docs/src/content/docs/reference/auth.mdx
• docs/src/content/docs/reference/auth-projects.mdx
• docs/src/content/docs/reference/web-search.md
• docs/src/content/docs/reference/github-tools.md
• docs/src/content/docs/reference/playwright.md
• docs/src/content/docs/reference/qmd.md
• docs/src/content/docs/reference/cache-memory.md
• docs/src/content/docs/reference/repo-memory.md
• docs/src/content/docs/reference/custom-agent-for-aw.mdx
• docs/src/content/docs/guides/upgrading.md

---

Report Generated: §26289895195
Workflow: claude-code-user-docs-review
Engine Used: claude (eating our own dog food 🐕)
Trend: Score unchanged vs. 2026-05-21 (6/10); Claude doc examples +1 (6 → 7); Claude auth URL inconsistency still present.

References:

• §26289895195 — current run
• §26229012080 — prior run (2026-05-21)
• §26165163800 — two runs ago (2026-05-20)

Generated by 📝 Claude Code User Documentation Review · ● 16M · ◷

• expires on May 23, 2026, 1:22 PM UTC

[github-actions[bot]]

This discussion has been marked as outdated by Claude Code User Documentation Review.

A newer discussion is available at Discussion #34207.