[claude-code-user-docs-review] 🔍 Claude Code User Documentation Review - 2026-08-25

[github-actions[bot]]

Executive Summary

Reviewed as a Claude Code user who does not use GitHub Copilot. Setup for Claude works end-to-end, but the docs consistently treat Copilot as the default path and Claude as the "also supported" path: onboarding steers to Copilot first, gh aw init only scaffolds custom-agent/MCP artifacts for Copilot, and a known auth footgun (CLAUDE_CODE_OAUTH_TOKEN being silently ignored) remains undocumented in the two docs a new user reads first. This is the 4th consecutive daily run flagging the same unresolved auth gap.

Severity Findings (click to expand)

Critical Blockers

• CLAUDE_CODE_OAUTH_TOKEN silently ignored, misleading error, unresolved 4+ days. A Claude Code user who runs claude login locally and reuses that token in Actions gets a generic Claude CLI auth failure that never mentions the token is the problem. The warning exists only in quick-start.mdx:210-211 (inside a collapsible tab) and cli.md:284 (buried under the secrets set command). It is absent from how-they-work.mdx and architecture.mdx, the two docs most likely read first. (Flagged in review-history.jsonl on 2026-08-22, -23, -24, -25 — still not surfaced in a primary doc.)

Major Obstacles

• gh aw init doesn't scaffold anything for Claude. The custom agent file (.github/agents/agentic-workflows.md) and MCP wiring (.github/mcp.json, copilot-setup-steps.yml) are Copilot-only; Claude's replacement is "author an agent file in your own agent's format (Claude Code subagents...)" with no template, path convention, or example (cli.md:183-184).
• No local CLI/version prerequisite stated for Claude. quick-start.mdx:67-76 gives GitHub CLI an explicit v2.0.0+ requirement but says nothing about whether a local claude CLI or Claude Code subscription is needed at all — a Claude Code user may reasonably expect their existing login to just work (it doesn't; see the token issue above).
• Copilot-first bias throughout onboarding. quick-start.mdx:71: "If you already have GitHub Copilot, start there — it requires no extra account setup." architecture.mdx:249's configuration example and the architecture mermaid diagram (architecture.mdx:201-205, node COPILOT[Agent CLI]) both use Copilot as the default illustration rather than a neutral placeholder.
• Example-workflow imbalance. Of 292 workflow files in .github/workflows/, engine declarations split 106 copilot / 61 claude / 46 codex / 0 custom — a Claude user browsing for real-world patterns hits Copilot examples ~1.7x as often, and finds zero worked custom engine examples despite it being a documented option.

Minor Confusion

• cli.md:188 ("the engine chosen at init does not restrict workflows") reads as mildly contradicted by the same file's table showing Copilot-only artifacts are never retrofitted for other engines.
• tools.md states tool behavior is engine-agnostic only implicitly — parity is asserted by omission except for two explicit callouts (Codex web-search default, Claude/Codex tools.timeout defaults), leaving readers to infer "universal unless stated otherwise."
• Pi's auth-resolution logic (model: prefix decides Copilot/Anthropic/OpenAI creds) is one sentence with no example string (quick-start.mdx:231-235).

Engine & Tool Matrix

Engine	Setup	Examples	Auth	Score
Copilot	gh aw init fully scaffolds agent + MCP wiring; org-billing no-PAT path preferred (cli.md:159-188)	106 workflows	copilot-requests:write or COPILOT_GITHUB_TOKEN, two documented paths (cli.md:205)	9/10
Claude	No scaffolding; manual agent file, no template (cli.md:183-184)	61 workflows	ANTHROPIC_API_KEY/WIF, but CLAUDE_CODE_OAUTH_TOKEN trap undocumented in primary docs	5/10
Codex	No scaffolding, same gap as Claude	46 workflows	CODEX_API_KEY/OPENAI_API_KEY, straightforward, no known footgun	6/10
Custom	Docs mention it exists; no init support	0 worked examples in .github/workflows/*.md	N/A	2/10

Tool classification (tools.md) is largely universal across engines — edit, github, bash, playwright, cache-memory, repo-memory, mcp-servers all apply the same way. Two engine-specific carve-outs: Codex disables web-search by default (tools.md:67), and tools.timeout defaults differ per engine (Claude 60s / Codex 120s, tools.md:178). drive-memory is feature-gated/private-preview for all engines (tools.md:93-96).

Auth Gaps

• Required secrets (how-they-work.mdx:30-36): Copilot → copilot-requests:write or COPILOT_GITHUB_TOKEN; Claude → ANTHROPIC_API_KEY or WIF; Codex → CODEX_API_KEY/OPENAI_API_KEY; Gemini → GEMINI_API_KEY/WIF; Pi → derived from model: prefix.
• Setup depth is uneven: Copilot gets a fuller two-option walkthrough (cli.md:205); Claude's equivalent is a shorter tab in quick-start.mdx:209-216 with no add-wizard transcript.
• The core gap, again: nothing in how-they-work.mdx or architecture.mdx warns that a claude login OAuth token won't work in Actions — the only two mentions are downstream docs a user may not reach before hitting the confusing error.

Recommended Actions

Priority 1

• Add the CLAUDE_CODE_OAUTH_TOKEN warning to how-they-work.mdx and/or architecture.mdx (not just quick-start.mdx/cli.md), since those are read first and this has now recurred for 4 consecutive daily reviews.

Priority 2

• Provide a worked Claude Code subagent template/example alongside the Copilot custom-agent scaffold in cli.md:183, and a sample MCP host config for gh aw mcp-server under Claude.
• State explicit local-CLI/version prerequisites (or explicitly say none are needed) for Claude/Codex/Gemini, matching the gh CLI v2.0.0+ treatment.

Priority 3

• Rebalance onboarding framing so Copilot isn't the implied default (neutral placeholder in architecture.mdx:249 diagram/example); add at least one worked engine: custom example workflow.

Generated by 📝 Claude Code User Documentation Review · claude · agent · 238.2 AIC · ⌖ 6.27 AIC · ⊞ 6.3K · ◷

• expires on Aug 26, 2026, 4:44 AM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Claude Code User Documentation Review.

A newer discussion is available at Discussion #56036.