[claude-code-user-docs-review] 🔍 Claude Code User Documentation Review - 2026-08-07

[github-actions[bot]]

Executive Summary

A Claude Code user (no Copilot, no Copilot CLI) can complete basic gh-aw setup — engines are genuinely pluggable, and ANTHROPIC_API_KEY + engine: claude is documented in README.md and quick-start.mdx. But Copilot remains the implicit default throughout (silent fallback when engine: is omitted), tooling depth (wizard, doctor) is asymmetric, and the example corpus has grown more Copilot-skewed since our last review (131 Copilot vs 59 Claude vs 15 Codex, up from 35/37/9 on 2026-08-06). Key finding: the single biggest risk for a Claude-only user isn't missing docs — it's a silent engine switch when engine: is left out of frontmatter.

Severity Findings

Critical Blockers

• engine: silently defaults to Copilot when omitted from workflow frontmatter. A Claude Code user who copies an example without an explicit engine: field gets switched to Copilot — an engine they likely have no credentials for — with no compile-time warning tied to this default. (docs/src/content/docs/setup/quick-start.mdx:116)

Major Obstacles

• gh aw doctor diagnostics are described purely in terms of gh CLI/repo-host auth (GH_HOST, --repo, owner-type checks). It's never stated whether doctor also validates ANTHROPIC_API_KEY, leaving Claude users without a documented way to self-check their engine auth. (docs/src/content/docs/setup/cli.md:273)
• add-wizard's Copilot auth flow is documented in deep detail (PAT scopes, auto-opened prefilled token-creation page, validation heuristics). No equivalent paragraph exists for what happens "when the Claude engine is selected." (docs/src/content/docs/setup/cli.md:180)
• tools.md singles out Codex as needing web-search: disabled handling, but never confirms whether Claude works out of the box or needs a third-party MCP server for web search — the "some engines require third-party MCP servers" line is unattributed. (docs/src/content/docs/reference/tools.md:65, :67)
• Example parity gap has widened: 131 Copilot workflows vs 59 Claude vs 15 Codex (up from 35/37/9 last run). A Copilot-only copilot-sdk: true + custom driver-script pattern spans 69 workflows with no Claude equivalent anywhere, and four shared/copilot-*.md partials (agent/session/PR analysis) have no shared/claude-*.md counterpart. (repo-wide scan of .github/workflows/*.md)
• No documented bridge for reusing a local claude login session. CLAUDE_CODE_OAUTH_TOKEN is explicitly rejected ("ignored if set"), forcing users to mint a separate ANTHROPIC_API_KEY with no discussion of cost/billing implications vs. an existing Claude subscription. (docs/src/content/docs/setup/quick-start.mdx:147)

Minor Confusion

• gh aw init --engine claude is said to "skip Copilot-specific artifacts" but never states what (if anything) is created for Claude instead. (docs/src/content/docs/setup/cli.md:151)
• tools.timeout defaults are only given for Claude (60s) and Codex (120s), omitting Copilot and Gemini defaults entirely. (docs/src/content/docs/reference/tools.md:157)
• A pi engine value is listed as a valid --engine choice with zero documentation anywhere in the reviewed docs. (docs/src/content/docs/setup/cli.md:269)
• The architecture diagram enumerates engines as "Copilot, Claude, Codex," omitting Gemini, and lists Copilot first throughout. (docs/src/content/docs/introduction/architecture.mdx:299)

Engine & Tool Matrix

	Copilot	Claude	Codex	Custom
Setup docs	Full (wizard flow, PAT scopes, auto-open browser) cli.md:180	Partial (secret + frontmatter, thin wizard depth) quick-start.mdx:141-147	Partial (secret + CODEX_API_KEY precedence) quick-start.mdx:150-155	N/A (env: block for arbitrary secrets) tools.md:194
Examples (repo-wide)	131	59	15	0
Auth secret	COPILOT_GITHUB_TOKEN or copilot-requests: write	ANTHROPIC_API_KEY	OPENAI_API_KEY/CODEX_API_KEY	user-defined
Default behavior	Implicit default when engine: omitted quick-start.mdx:116	Must be explicit	Must be explicit	Must be explicit
Score (setup+examples+auth clarity)	9/10	6/10	5/10	3/10 (undocumented as first-class)

Engine-example-counter's parity_observations confirm the gap is structural, not incidental: Claude examples cover advanced knobs well (model-provider, custom agent personas, strict mode, network allowlists) but are absent from two specific niches — pluggable custom-driver integration (copilot-sdk: true + driver: .github/drivers/*.cjs) and self-service session/usage analysis tooling (shared/copilot-*-analysis*.md). Ironically, several "claude" engine workflows exist only to have Claude analyze Copilot's telemetry — there's no reverse example of a Claude user analyzing their own agent's sessions.

Auth Gaps

• No stated engine-secret validation in doctor (cli.md:273) — biggest actionable gap.
• add-wizard auth guidance depth is Copilot-only (cli.md:180).
• Local Claude Code CLI credentials (CLAUDE_CODE_OAUTH_TOKEN) cannot be reused; only a fresh ANTHROPIC_API_KEY works, and this is stated as a rejection with no workaround (quick-start.mdx:147).
• web-search: engine-dependency is documented for Codex only; Claude's requirement is unstated (tools.md:65,67).

Recommended Actions

Priority 1

1. Emit a compile-time warning (or require explicit engine:) rather than silently defaulting to Copilot — this is the one true blocker for adoption by mistake.
2. Document whether gh aw doctor validates ANTHROPIC_API_KEY/OPENAI_API_KEY/GEMINI_API_KEY, and add that check if missing.

Priority 2
3. Add a Claude-equivalent auth-flow paragraph to add-wizard docs (cli.md:180) matching Copilot's depth.
4. State explicitly whether Claude needs a third-party MCP server for web-search: (tools.md:65-67).
5. Add at least one Claude example each for the driver-script pattern and session/usage self-analysis, to close the structural example gap.

Priority 3
6. Fill in Copilot/Gemini tools.timeout defaults; document the pi engine or remove it from --engine option lists; add Gemini to the architecture diagram.

Trend note: since 2026-08-06, critical_blockers held steady at 1 but major_obstacles grew from 4→5 and the Copilot:Claude example ratio worsened (0.94→2.2). Findings appended to review-history.jsonl in cache-memory for tracking.

Generated by 📝 Claude Code User Documentation Review · agent · 286 AIC · ⊞ 5.4K · ◷

• expires on Aug 8, 2026, 4:58 AM UTC-08:00