[claude-code-user-docs-review] 🔍 Claude Code User Documentation Review - 2026-08-17

[github-actions[bot]]

Executive Summary

A Claude Code user can onboard to gh-aw without ever touching GitHub Copilot — ANTHROPIC_API_KEY setup and --engine claude are documented — but the docs consistently treat Copilot as the "real" default and Claude as the exception path. The sharpest gap: Claude Code users are explicitly told their existing subscription auth (CLAUDE_CODE_OAUTH_TOKEN from claude login) is not supported, forcing a separate pay-per-token setup many won't expect.

Severity Findings (click to expand)

Critical Blockers

• Subscription auth explicitly rejected. CLAUDE_CODE_OAUTH_TOKEN, including a token from claude login, "is not supported and is ignored if set" — a Claude Code user's existing daily-driver auth cannot be reused; they must provision ANTHROPIC_API_KEY or Anthropic WIF instead. (README.md:41, docs/src/content/docs/setup/quick-start.mdx:148-150, docs/src/content/docs/setup/cli.md:259)
• Silent engine fallback to Copilot. If the engine: field is removed from frontmatter, "the runtime defaults to Copilot" — a Claude user copying/trimming a workflow can silently end up needing Copilot secrets instead of Anthropic ones, with no warning at compile time. (docs/src/content/docs/setup/quick-start.mdx:116)

Major Obstacles

• gh aw init only describes what Claude users don't get. The Copilot path gets a custom agent file + default MCP integration; --engine claude is documented only as "Skip Copilot-specific artifacts," with no description of any Claude-equivalent scaffolding. (docs/src/content/docs/setup/cli.md:151-158)
• Example imbalance. Across .github/workflows/*.md, engine: copilot appears in 133 workflows vs. 60 for engine: claude (Codex: 15, custom: 0) — Claude users have roughly half as many worked examples to learn patterns from.
• web-search: support for Claude is unstated. Docs say "some engines require third-party MCP servers for web search" without naming which ones — a Claude user can't tell if it works natively or needs extra setup. (docs/src/content/docs/reference/tools.md:62-65,67)
• No Claude-specific self-hosted runner guide. The security architecture doc links a dedicated Copilot + ARC/DinD deployment guide; no equivalent is referenced for Claude. (docs/src/content/docs/introduction/architecture.mdx:197)

Minor Confusion

• Framing bias in prerequisites. "If you already have GitHub Copilot, start there — it requires no extra account setup" implicitly casts Claude as the higher-friction second choice, even though the actual Claude setup is only 2 steps. (docs/src/content/docs/setup/quick-start.mdx:71)
• cli-proxy: classification inconsistency. Documented as a generic/engine-agnostic tool in the tools reference, but called out as a hard requirement specifically for Pi elsewhere — unclear if any Claude workflow ever needs it. (docs/src/content/docs/reference/tools.md:129-149 vs. docs/src/content/docs/setup/quick-start.mdx:169)
• copilot-requests actionlint carve-out has no Claude counterpart mentioned, leaving it unclear whether Claude-specific permission scopes need similar linter handling. (docs/src/content/docs/setup/cli.md:394)

Engine & Tool Matrix

Engine	Setup	Examples	Auth	Score
Copilot	Multi-step wizard, PAT guidance, default engine (how-they-work.mdx:26)	133 workflows — dominant reference set	copilot-requests perm or COPILOT_GITHUB_TOKEN PAT, richly documented (cli.md:180)	5/5
Claude	2 steps (quick-start.mdx:144-146), but subscription token blocked	60 workflows (~45% of Copilot's)	ANTHROPIC_API_KEY/WIF only; CLAUDE_CODE_OAUTH_TOKEN unsupported	3.5/5
Codex	2 steps, CODEX_API_KEY or OPENAI_API_KEY	15 workflows, narrow task variety (mostly smoke tests)	Documented, precedence rule stated (quick-start.mdx:153-157)	3/5
Custom	Not covered in these 6 docs	0 dogfooded example workflows (only appears inside a shared/include snippet)	N/A	1/5

Tool classification (from reference/tools.md): 15 engine-agnostic, 3 Copilot-specific, 1 Claude-specific, 1 Codex-specific, 1 Pi-specific, 1 ambiguous (web-search:). Tool-operation timeout defaults are stated only for Claude (60s) and Codex (120s) — Copilot/Gemini/Pi defaults are unstated in the same table (tools.md:157).

Auth Gaps

• Claude: ANTHROPIC_API_KEY or Anthropic WIF — clearly documented, but the explicit rejection of claude login OAuth tokens is the single biggest surprise for this persona. (quick-start.mdx:148-150)
• Copilot: Best-documented path — wizard flow, PAT scope guidance (Public repositories + Copilot Requests: Read), and a note that GitHub never exposes stored secret values so the wizard must re-prompt. (cli.md:180)
• Codex: CODEX_API_KEY takes precedence over OPENAI_API_KEY if both set — documented, no gaps found.
• Pi: Auth described only at a high level ("depending on model: prefix") with no inline setup steps, deferring entirely to an external guide — weakest auth documentation of the five engines. (quick-start.mdx:166-169)
• Cross-cutting: Default GITHUB_TOKEN cannot create GitHub Projects (needs a separate PAT), and MCP gateway API keys mounted into agent containers are explicitly not a strong security boundary against a compromised agent — both apply to all engines but are easy to miss. (cli.md:944-949, architecture.mdx:291)

Recommended Actions

Priority 1

1. Add a prominent callout near the ANTHROPIC_API_KEY setup step clarifying why claude login tokens don't work and what the cost/setup implications are, so Claude Code users aren't surprised mid-setup.
2. Make the Copilot fallback-on-missing-engine: behavior a compile-time warning, or at least surface it more prominently than a single note in quick-start.

Priority 2
3. Document what gh aw init --engine claude does produce, not just what it skips (cli.md:151-158).
4. Clarify per-engine web-search: support explicitly (which engines need an MCP server, which don't) (tools.md:62-67).
5. Add a Claude-equivalent (or engine-agnostic) self-hosted runner / ARC-DinD guide reference (architecture.mdx:197).

Priority 3
6. Add at least one dogfooded engine: custom example workflow.
7. Rebalance example-workflow coverage so Claude/Codex aren't ~2-9x thinner than Copilot's.
8. Reframe the prerequisites section to present engine choice neutrally rather than Copilot-first.

References:

• §32030135004

Generated by 📝 Claude Code User Documentation Review · agent · 162.7 AIC · ⌖ 6.06 AIC · ⊞ 5.7K · ◷

• expires on Aug 18, 2026, 4:41 AM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Claude Code User Documentation Review.

A newer discussion is available at Discussion #53691.