[claude-code-user-docs-review] 🔍 Claude Code User Documentation Review - 2026-10-03

[github-actions[bot]]

Executive Summary

gh-aw now supports Claude Code as a first-class engine with clear API-key setup, and the long-standing CLAUDE_CODE_OAUTH_TOKEN silent-failure trap is documented in two places (quick-start.mdx:153, cli.md:240) — the critical blocker flagged in this review's first 16 runs remains resolved. The main friction left for non-Copilot adopters is structural: gh aw init still auto-scaffolds a custom agent and MCP wiring only for Copilot (cli.md:139-140), and Claude has roughly half as many example/smoke-test workflows to learn from as Copilot (56 vs 110, ~1.96x). Key finding: a Claude Code user can fully set up and run gh-aw without Copilot, but must self-author the agent/MCP integration pieces that Copilot users get for free, with no concrete steps given for doing so.

Severity Findings (Critical / Major / Minor)

Critical Blockers: None open this run.

Major Obstacles

1. gh aw init artifact-parity gap — Copilot gets an auto-generated custom agent (.github/agents/agentic-workflows.md) and MCP wiring (.github/mcp.json, copilot-setup-steps.yml); Claude/Codex users are told only to "author an agent file in your own agent's format" / "register gh aw mcp-server in your own MCP host configuration" with zero concrete steps (cli.md:139-140).
2. Example/smoke-test parity — 56 Claude workflow examples vs 110 Copilot (~1.96x) across .github/workflows/*.md; Claude has far fewer reference patterns to copy from.
3. WIF setup is link-only for Claude — Anthropic Workload Identity Federation is a single bare link with no inline steps (quick-start.mdx:151), unlike the fully spelled-out API-key path right above it.
4. No stated coverage of whether gh aw doctor validates engine secrets like ANTHROPIC_API_KEY — its documented checks are limited to gh auth/repo state (cli.md:256-273), leaving the only auth-troubleshooting signal a downstream CLI error.

Minor Confusion

1. quick-start.mdx:72 tells new users "If you already have GitHub Copilot, start there — it requires no extra account setup," which can read as Copilot being the recommended/default path even though Claude's path (API key) is comparably short.
2. tools.timeout docs name only "Claude and Codex" defaults (60s), leaving Copilot/Gemini/Pi unaddressed (tools.md:237) — ambiguous whether the setting applies uniformly.
3. The custom engine has 0 standalone example workflows; its only occurrence is nested inside shared/genaiscript.md, an include-only fragment — hard to find a worked example.
4. Pi's auth model borrows credentials from Copilot/Anthropic/OpenAI based on a model: prefix (how-they-work.mdx:36) rather than having its own secret, which isn't obvious on first read.

Engine & Tool Matrix

Dimension	Copilot	Claude	Codex	Custom
Setup clarity	Org billing or PAT, fully inline (quick-start.mdx:135-143)	API key inline; WIF link-only (quick-start.mdx:147-153)	API key inline, CODEX_API_KEY precedence noted (quick-start.mdx:159-163)	Not a quick-start path; self-authored (how-they-work.mdx)
Example workflows	110	56 (~51% of Copilot)	76 (~69% of Copilot)	0 standalone (only nested in shared/genaiscript.md)
gh aw init artifacts	Custom agent + MCP wiring auto-created (cli.md:139-140)	Self-author, no steps given	Self-author, no steps given	N/A
Required secret(s)	COPILOT_GITHUB_TOKEN or copilot-requests: write	ANTHROPIC_API_KEY or WIF (CLAUDE_CODE_OAUTH_TOKEN unsupported, quick-start.mdx:153)	OPENAI_API_KEY / CODEX_API_KEY	Depends on wrapped engine
Overall UX score (1-5)	5	3	4	2

Cross-checked both YAML engine-declaration forms (engine: claude and engine:\n id: claude): copilot/claude ratio = 1.96x, codex/claude ratio = 1.36x, matching prior-run trends. 285 of 304 top-level workflow files declare an engine explicitly; 19 omit it (implicit default).

Tool-feature support is mostly engine-agnostic by omission rather than by design: of 18 tool/feature rows sampled from tools.md, only 4 state all-engine support explicitly, 2 are Copilot-only artifacts (cli.md:139-140), and 12 are "unclear" because the docs don't call out per-engine differences at all (e.g. cache-memory:, playwright:, mcp-servers: — tools.md:132-264).

Auth Gaps

• CLAUDE_CODE_OAUTH_TOKEN (from claude login) is unsupported and silently ignored; the resulting Claude CLI error "never mentions the token" (quick-start.mdx:153, cli.md:240) — documented, but only in these two spots, not cross-linked from a general troubleshooting page.
• Non-Copilot engines are told to "author an agent file in your own agent's format" or "register gh aw mcp-server in your own MCP host configuration" (cli.md:139-140) with no concrete steps — both point outside the reviewed doc set.
• gh aw doctor (cli.md:256-273) documents gh auth/repo-state checks only; no stated behavior for missing/invalid ANTHROPIC_API_KEY.
• Anthropic and Google WIF are each a single bare link (quick-start.mdx:151,171) with zero inline detail, unlike the concrete API-key step lists beside them.
• No statement of whether a Claude Code subscription session covers workflow runs (it doesn't — billing is via ANTHROPIC_API_KEY) beyond the OAuth-rejection note; a subscriber could reasonably assume otherwise.

Recommended Actions

Priority 1 — Give Claude/Codex users a concrete self-authoring recipe (or a gh aw init --engine claude scaffold) to close the init artifact-parity gap (cli.md:139-140).
Priority 2 — Replace the link-only WIF mentions with inline setup steps for Claude and Gemini (quick-start.mdx:151,171), matching the API-key path's level of detail.
Priority 3 — Add 3-5 more Claude example/smoke-test workflows to narrow the 1.96x example gap, and document whether gh aw doctor validates engine secrets.

---

Daily Claude Code adoption review. Prior-run history (41 runs since 2026-08-22) tracked in cache-memory review-history.jsonl.

Generated by 📝 Claude Code User Documentation Review · claude · agent · 254.5 AIC · ⌖ 4.92 AIC · ⊞ 5.2K · ◷

• expires on Oct 4, 2026, 6:09 AM UTC-08:00

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-10-04T14:09:26.149Z.

Closed by Workflow