[claude-code-user-docs-review] 🔍 Claude Code User Documentation Review - 2026-09-25

[github-actions[bot]]

Executive Summary

A Claude Code user (no GitHub Copilot) can complete onboarding, but the docs consistently privilege Copilot: inline, two-path auth instructions for Copilot vs. a truncated Claude section that defers to an external page, and a long-standing gotcha where Claude's own claude login credential is silently rejected. No hard blockers found — but friction is real and one finding has now recurred unresolved for 34 consecutive daily reviews since 2026-08-22.

Severity Findings (Critical → Major → Minor)

Critical Blockers: none this run.

Major Obstacles

1. CLAUDE_CODE_OAUTH_TOKEN silently ignored — a Claude Code user's natural first move (reuse their local claude login token) fails with a generic, unrelated CLI auth error that never names the token as the cause. Documented only in prose, no callout box. (docs/src/content/docs/setup/cli.md:240) — unresolved across 34 consecutive daily runs since 2026-08-22.
2. gh aw init artifact parity gap — Copilot gets an auto-scaffolded custom agent file + MCP wiring by default; Claude/Codex/Gemini/Pi do not. The replacement path ("author an agent file in your own agent's format... Claude Code subagents") is only sketched, not walked through. (docs/src/content/docs/setup/cli.md:135-140)
3. Example/smoke-test parity gap — Claude has 55 engine declarations vs. Copilot's 107 (~51%) across .github/workflows/*.md; Copilot also has a dedicated ~14-variant smoke matrix (arm, aoai-apikey, aoai-entra, sdk, sub-agents, etc.) with no Claude equivalent.
4. Claude quickstart section is incomplete standalone — explicitly defers to /gh-aw/engines/claude/ for "complete setup," unlike Copilot's fully inline org-billing-or-PAT instructions. (docs/src/content/docs/setup/quick-start.mdx:145-153 vs. :133-143)

Minor Confusion

• custom engine id has zero standalone example workflows — only reachable as a nested snippet inside shared/genaiscript.md.
• Web-search MCP server needed for Claude is not named inline; readers are pointed to a separate reference page. (docs/src/content/docs/reference/tools.md:126-128)
• Pi's auth row omits the model: provider-prefix mechanism that actually determines which credential Pi uses — explained only in quick-start, not in the engine table. (docs/src/content/docs/introduction/how-they-work.mdx:36 vs. docs/src/content/docs/setup/quick-start.mdx:171-175)
• ~17 of 298 workflow files omit engine: entirely (implicit default), which isn't reflected in any documented count.

Engine & Tool Matrix

Engine	Setup	Examples	Auth	Score
Copilot	Fully inline, 2 paths (org billing / PAT)	107 declarations, ~14 smoke variants	copilot-requests: write or COPILOT_GITHUB_TOKEN PAT	Best documented
Claude	Inline basics, defers to external page for "complete setup"	55 declarations (~51% of Copilot), 8 smoke variants (permission-scope, not engine-variant)	ANTHROPIC_API_KEY or WIF; CLAUDE_CODE_OAUTH_TOKEN silently ignored (cli.md:240)	Usable, weakest auth-error UX
Codex	Inline, single path	74 declarations, 5 smoke variants	CODEX_API_KEY (precedence) or OPENAI_API_KEY	Clear
Custom	No dedicated section	0 standalone examples (only nested in shared/genaiscript.md)	N/A	Underdocumented

Most documented tools (edit, github, linear, jira, bash, playwright, cache-memory, repo-memory, mcp-servers) are engine-agnostic across all 5 engines (docs/src/content/docs/reference/tools.md). Two artifacts are Copilot-exclusive by design: the auto-scaffolded custom agent file and MCP wiring from gh aw init (cli.md:135-140). web-search: default-disabled behavior is spelled out only for Codex/Claude/Copilot — Gemini and Pi are unmentioned (tools.md:126-128), an ambiguous entry.

Parity observation: Copilot's dominance (107 vs. 55 vs. 74) plus its exclusive smoke-variant matrix means a Claude Code user has fewer worked examples to copy from when adapting a non-trivial workflow.

Auth Gaps

• Claude: ANTHROPIC_API_KEY (Console-issued) or Anthropic WIF; the natural claude login / CLAUDE_CODE_OAUTH_TOKEN path is unsupported and fails silently rather than with a named error (cli.md:240, quick-start.mdx:147-153).
• Copilot: org Copilot billing (adds copilot-requests: write) or a fine-grained PAT with Copilot Requests: Read stored as COPILOT_GITHUB_TOKEN — both paths fully inline (quick-start.mdx:135-143).
• Codex: OPENAI_API_KEY, or reuse existing CODEX_API_KEY (quick-start.mdx:157-161).
• Gemini: GEMINI_API_KEY or Google WIF (quick-start.mdx:165-169).
• Pi: dynamic — determined by the model: provider prefix (copilot/..., anthropic/..., openai/...); also requires tools.github.mode: gh-proxy + tools.cli-proxy: true. This mechanism is documented in quick-start but absent from the engine summary table in how-they-work.mdx:36.
• General gap: no dedicated "Troubleshooting Claude auth" callout exists anywhere in the reviewed docs; the OAuth-token gotcha lives as a single sentence in the CLI reference, easy to miss before hitting the error.

Recommended Actions

Priority 1

• Add an explicit warning callout (not just prose) at quick-start.mdx:145-153 and cli.md:240 stating that claude login/CLAUDE_CODE_OAUTH_TOKEN does not work with gh-aw, before the user hits the misleading CLI error.

Priority 2

• Inline a minimal worked example for authoring a Claude Code subagent/agent file for dispatch (referenced but not shown at cli.md:139), matching the detail level Copilot gets for its init artifacts.
• Backfill Claude-engine smoke-test variants analogous to Copilot's arm/sdk/sub-agents matrix to close the example-parity gap.

Priority 3

• Add a standalone custom engine example under .github/workflows/ (currently only nested in shared/genaiscript.md).
• Fold Pi's model: prefix auth mechanism into the how-they-work.mdx engine table so it's visible without cross-referencing quick-start.

_{Run: §36136215936}

Generated by 📝 Claude Code User Documentation Review · claude · agent · 200.9 AIC · ⌖ 5.25 AIC · ⊞ 6.5K · ◷

• expires on Sep 26, 2026, 4:48 AM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Claude Code User Documentation Review.

A newer discussion is available at Discussion #63624.