[claude-code-user-docs-review] 🔍 Claude Code User Documentation Review - 2026-10-07

[github-actions[bot]]

Executive Summary

A Claude Code user (no Copilot) can get a gh-aw workflow running end-to-end today — ANTHROPIC_API_KEY is documented inline in 2 numbered steps (quick-start.mdx:143-153). No critical blockers were found this run. The main friction is polish asymmetry: Copilot gets auto-scaffolded init artifacts, fully inline numbered auth steps (including keyless WIF), and 11 smoke-test auth variants; Claude gets a bare link for WIF, an unsupported-credential trap for the exact token most Claude Code users already hold, and 4 smoke variants. Key finding: CLAUDE_CODE_OAUTH_TOKEN from a normal claude login session — the credential a Claude Code user is most likely to already have sitting around — is silently ignored by gh-aw, with a generic error that never names the token (quick-start.mdx:151, cli.md:232).

Severity Findings (Critical → Major → Minor)

Critical Blockers

None found this run.

Major Obstacles

1. gh aw init artifact-parity gap — custom agent file and MCP wiring (.github/mcp.json, copilot-setup-steps.yml) are auto-scaffolded only for --engine copilot. For Claude/Codex, cli.md:131 says only: "author an agent file in your own agent's format (Claude Code subagents, Codex prompts) from the same instructions" and "Register gh aw mcp-server in your own MCP host configuration" — no template, no example config. Evidence: cli.md:107,125-132.
2. CLAUDE_CODE_OAUTH_TOKEN trap — the OAuth token from a standard claude login (what most individual Claude Code users already have) is explicitly unsupported and silently ignored; the run fails with a generic Claude CLI auth error that never mentions the token. Evidence: quick-start.mdx:151, cli.md:232.
3. Anthropic WIF is link-only — keyless setup is one sentence + a link (quick-start.mdx:149), versus Copilot's two full inline paths with numbered steps (quick-start.mdx:131-141). Google WIF has the identical gap (quick-start.mdx:169).
4. Example/smoke-test parity gap — Claude: 60 workflow examples / 4 smoke-claude-*.md auth variants vs Copilot: 114 examples (~1.9x) / 11 smoke-copilot-*.md variants covering SDK/auth/model permutations (arm, sdk, small, sub-agents, aoai-apikey, aoai-entra, auto, dynamic-workflow, mai, plain). A Claude user has far fewer worked patterns to copy from.

Minor Confusion

1. Pi engine's auth model is ambiguous — described as "inherited... selected by the provider prefix in model:" (how-they-work.mdx:36), unclear if it's a 6th independent path or a router over the other four.
2. Copilot BYOK web-tool restriction interaction with engine.copilot-sdk: true is stated but its parity with Claude/Codex's native web tools isn't fully spelled out (tools.md:130).
3. MCP wiring replacement for Claude/Codex is a bare cross-link with zero example YAML (cli.md:131).

Engine & Tool Matrix

Engine	Setup	Examples	Auth	Score
Copilot	Inline, 2 paths (org billing / PAT), 3 numbered steps (quick-start.mdx:131-141)	114 workflows, 11 smoke-auth variants	copilot-requests: write or COPILOT_GITHUB_TOKEN (how-they-work.mdx:32)	9/10
Claude	Inline API key (2 steps), WIF link-only, OAuth-token trap (quick-start.mdx:143-153)	60 workflows, 4 smoke-auth variants	ANTHROPIC_API_KEY or WIF; CLAUDE_CODE_OAUTH_TOKEN not supported	6/10
Codex	Inline API key (2 steps) (quick-start.mdx:155-161)	77 workflows, 3 smoke-auth variants	CODEX_API_KEY (precedence) or OPENAI_API_KEY	7/10
Custom/long-tail (goose, aider, opencode, pydantic-ai, kiro, gemini, deepseek-harness, cursor, crush)	No gh aw init scaffolding; each has exactly one engine-conformance-<engine>.md + smoke pair	2-4 workflows each	Engine-specific, mostly undocumented in the 6 core files	4/10

init artifact table confirms Copilot-only scaffolding (cli.md:127-132): .gitattributes and dispatcher skill ship for all engines; custom agent file + MCP wiring ship only for Copilot. Total workflows scanned: 324 top-level .github/workflows/*.md; 306 declare an engine (84 inline scalar + 222 nested id: form), 18 have none (orchestration-only). copilot=114 vs claude=60 (1.9x) matches the historical band; codex=77 is ~1.28x claude.

Auth Gaps

• Anthropic WIF: one sentence + link, no inline steps, no example YAML (quick-start.mdx:149).
• Google WIF: identical gap (quick-start.mdx:169).
• CLAUDE_CODE_OAUTH_TOKEN: explicitly unsupported, silently ignored, failure message doesn't name it (quick-start.mdx:151, cli.md:232) — repeated for gh aw secrets set.
• Claude/Codex MCP wiring: link-only pointer to "GH-AW as an MCP Server" doc, no concrete config shown in this doc set (cli.md:131).
• Claude/Codex custom agent file: described abstractly only — "author an agent file in your own agent's format... from the same instructions" — no command or template referenced (cli.md:131).

Recommended Actions

Priority 1

• Add an explicit, loud warning (not just prose) when CLAUDE_CODE_OAUTH_TOKEN is detected but unsupported, naming the token in the error message itself.
• Inline the Anthropic/Google WIF steps in quick-start.mdx the same way Copilot's PAT path is inlined, rather than a single sentence + link.

Priority 2

• Generate a minimal Claude/Codex agent-file template and MCP config snippet from gh aw init --engine claude|codex, even if less automated than Copilot's path.
• Add 4-6 more smoke-claude-*.md auth/SDK variants to close the smoke-coverage gap with Copilot's 11.

Priority 3

• Clarify Pi's auth model (independent vs router) in how-they-work.mdx:36.
• Add an example MCP host config for Claude/Codex next to the link in cli.md:131.

Generated by 📝 Claude Code User Documentation Review · claude · agent · 334.1 AIC · ⌖ 5.01 AIC · ⊞ 5.2K · ◷

• expires on Oct 8, 2026, 4:55 AM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Claude Code User Documentation Review.

A newer discussion is available at Discussion #66879.