[claude-code-user-docs-review] 🔍 Claude Code User Documentation Review - 2026-09-27

[github-actions[bot]]

Executive Summary

gh-aw's docs remain usable for Claude Code users — no critical blockers found today — but Copilot is still the clear "first-class" path: quick-start gives Copilot a full auth decision tree while Claude gets two bullets and an external link, gh aw init only auto-scaffolds a custom agent file + MCP wiring for Copilot, and Claude's example/smoke-test coverage sits at ~52% of Copilot's. Key finding: the CLAUDE_CODE_OAUTH_TOKEN pitfall (a Claude Code user's most natural first auth attempt) is well-documented deep in reference pages but still has no inline callout in quick-start.mdx's Claude tab — the 20th consecutive daily run flagging this.

Severity Findings

Critical / Major / Minor findings (click to expand)

Critical Blockers: None found this run.

Major Obstacles:

1. CLAUDE_CODE_OAUTH_TOKEN silently ignored, no inline warning in quick-start — docs/src/content/docs/setup/quick-start.mdx:145-153; explained only downstream at cli.md:240, engines/claude.md:22, reference/auth.mdx:218-220. The failure produces "an authentication error from the Claude CLI that never mentions the token" (cli.md:240).
2. gh aw init scaffolding parity gap — Copilot auto-gets .github/agents/agentic-workflows.md + .github/mcp.json + copilot-setup-steps.yml; Claude/Codex/Gemini/Pi users must hand-author an equivalent with no worked example (cli.md:135-140,142).
3. Example/smoke-test coverage gap — Claude: 58 engine declarations / 9 smoke variants vs Copilot: 112 / 13, including provider-specific Copilot-only variants (aoai-apikey, aoai-entra, arm, sub-agents) with no Claude equivalent.
4. Claude auth depth imbalance — Copilot's quick-start explains org-billing vs PAT tradeoffs in detail (cli.md:161); Claude's WIF option is one line with no cost/complexity guidance (quick-start.mdx:151).

Minor Confusion:

1. Prerequisites bullet links Claude to generic anthropic.com, not console.anthropic.com where the API key is actually created (quick-start.mdx:72 vs :148).
2. reference/engines.md:56 calls Claude web-search "native" with no opt-in qualifier, while reference/tools.md:128 says it's opt-in for Claude too — inconsistent phrasing between the two docs.
3. cli.md:142 ("remaining steps are the same for every engine") understates the MCP/agent-file asymmetry described two paragraphs earlier in the same doc.
4. No mention in the core docs of how ANTHROPIC_API_KEY billing relates to an existing Claude Pro/Max/Team subscription — a plausible confusion point for subscribers assuming their plan covers Actions usage.

Engine & Tool Matrix

Engine	Setup	Examples (declarations / smoke variants)	Auth	Score
Copilot	Full decision tree; auto-scaffolded agent+MCP (cli.md:113-144,161)	112 / 13	org billing or PAT	5/5
Claude	2-step + external link (quick-start.mdx:145-153)	58 / 9	ANTHROPIC_API_KEY or WIF; OAuth token unsupported	3/5
Codex	2-step, CODEX_API_KEY precedence noted	75 / 5	OPENAI_API_KEY / CODEX_API_KEY	3.5/5
Gemini	2-step + WIF	1 / 0	GEMINI_API_KEY or WIF	2/5
Pi / Custom	Link-only, no scaffolding	Pi 31/4; custom 1/0	Delegates to provider prefix	2/5

Copilot-only features: engine.agent, engine.harness, max-continuations, init scaffolding (reference/engines.md:50-63, cli.md:135-140). Claude-only: deprecated engine.max-turns alias. Codex's declarations are 93% nested-YAML-form (generated later) vs Claude's 34% nested — suggesting template drift across engine docs/examples over time.

Auth Gaps

• Claude: ANTHROPIC_API_KEY or Anthropic WIF; CLAUDE_CODE_OAUTH_TOKEN explicitly unsupported and silently ignored (reference/auth.mdx:218-220, reference/faq.md:486-488) — warning absent from quick-start's Claude tab itself.
• Codex: OPENAI_API_KEY; CODEX_API_KEY takes precedence.
• Gemini: GEMINI_API_KEY or Google WIF.
• Copilot: org billing (copilot-requests: write) or fine-grained PAT (COPILOT_GITHUB_TOKEN) — most thoroughly documented path, including default-token detection logic (cli.md:161).
• Pi: no dedicated key; delegates via model: prefix to Copilot/Anthropic/OpenAI auth, plus requires tools.github.mode: gh-proxy and tools.cli-proxy: true.

Recommended Actions

Priority 1: Add an inline callout (matching the TIP/WARNING admonition style used elsewhere, e.g. architecture.mdx:196,292) to quick-start.mdx's Claude auth subsection stating CLAUDE_CODE_OAUTH_TOKEN/claude login tokens are unsupported, where a Claude Code user would look first — not three links away.

Priority 2: Give gh aw init --engine claude a worked example of an equivalent agent file / MCP registration, or explicitly state none is provided and link a copy-pasteable template.

Priority 3: Fix the anthropic.com → console.anthropic.com prerequisite link and align the web-search opt-in wording between reference/engines.md and reference/tools.md.

---

Tooling note: .claude/agents/doc-reader.md and engine-example-counter.md still lack the required name: frontmatter field and fail to register as agents (13th consecutive day of this issue); Edit permission to fix them was denied again this run. Analysis fell back to two general-purpose agents carrying the same instructions inline — finding quality is unaffected, but this is filed separately via missing_tool.

Generated by 📝 Claude Code User Documentation Review · claude · agent · 258 AIC · ⌖ 9.01 AIC · ⊞ 6.4K · ◷

• expires on Sep 28, 2026, 4:48 AM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Claude Code User Documentation Review.

A newer discussion is available at Discussion #64007.