[claude-code-user-docs-review] 🔍 Claude Code User Documentation Review - 2026-09-26

[github-actions[bot]]

Executive Summary

gh-aw's documentation is engine-neutral in structure (5 built-in engines, near-full tool parity) but keeps a Copilot-first tone that creates friction for Claude Code users: the highest-impact issue is that CLAUDE_CODE_OAUTH_TOKEN (from claude login) silently fails and is documented only in the CLI reference, not in the quick-start Claude setup steps. Key finding: this OAuth gap has now been flagged in 19 consecutive daily runs (35 since 2026-08-22) without being surfaced where a new user would actually look.

Severity Findings (Critical → Major → Minor)

Critical Blockers: None currently open.

Major Obstacles

1. CLAUDE_CODE_OAUTH_TOKEN silently ignored — cli.md:240 explains the token from claude login is not supported and fails with a misleading Claude-CLI error that never mentions the token. This warning does not appear in quick-start.mdx:145-153 (the Claude setup steps), so a Claude Code user following quick-start has no way to know their existing claude login session won't work.
2. gh aw init scaffolding parity gap — init auto-creates a custom agent file and MCP wiring only for Copilot (cli.md:114-116, cli.md:135-140); Claude/Codex users get no equivalent generated artifacts.
3. Example/smoke-test coverage gap — Claude has 58 example workflow declarations vs Copilot's 112 (~1.93x), and only 3 Claude smoke-test variants (smoke-claude.md, smoke-claude-on-copilot.md, smoke-github-claude.md) vs 9 for Copilot.
4. Anthropic WIF is link-only — quick-start.mdx:151 and how-they-work.mdx:33 reference Workload Identity Federation for Claude without a worked example, unlike the more concrete Copilot PAT/org-billing steps (quick-start.mdx:135-143).

Minor Confusion

1. Branding mismatch: user-facing copy says "Claude Code" (quick-start.mdx:28) but the frontmatter engine id is claude (how-they-work.mdx:33) — never explicitly reconciled.
2. web-search: is disabled by default for claude/codex/copilot (tools.md:128-129), but this isn't cross-referenced from any engine-specific auth section.
3. Pi's engine auth is polymorphic based on the model: prefix (copilot/*, anthropic/*, openai/*, codex/* — quick-start.mdx:171-175), requiring users to parse that field to know which secret applies.

Engine & Tool Matrix

Engine	Setup clarity	Example workflows	Auth	Notes/Score
Copilot	Clear, PAT or org billing (quick-start.mdx:135-143)	112	COPILOT_GITHUB_TOKEN / org copilot-requests	Strong — default engine, most scaffolding
Claude	Partial — OAuth pitfall undocumented inline, WIF link-only	58	ANTHROPIC_API_KEY or WIF (quick-start.mdx:145-153)	Fair — functional but under-signposted
Codex	Clear (quick-start.mdx:155-161)	75	OPENAI_API_KEY/CODEX_API_KEY	Good
Custom	No dedicated setup doc; only pattern is shared/genaiscript.md	1 (nested only)	N/A (user-defined)	Weak — no standalone example workflow

Tool support (from tools.md): 13 of 14 documented tool categories work identically across all engines (edit, github, linear, jira, bash, web-fetch, playwright, cache-memory, drive-memory, repo-memory, qmd, agentic-workflows, cli-proxy) — good parity here. The one exception, web-search (tools.md:128-129), is opt-in for claude/codex/copilot but the doc never states whether pi/gemini differ, leaving that ambiguous.

Example-count verification note: an automated counter this run initially miscounted engines as 38 (claude) vs 22 (copilot) by only matching the inline-scalar engine: claude form and missing the more common nested engine: / id: claude form. Manual verification combining both forms gives claude 58, copilot 112, codex 75, pi 31, custom 1 — consistent with prior runs' ~1.9x Copilot:Claude ratio.

Auth Gaps

Engine	Required secret(s)	Setup steps documented	Gap
Copilot	COPILOT_GITHUB_TOKEN or org copilot-requests permission	quick-start.mdx:135-143	None significant
Claude	ANTHROPIC_API_KEY or Anthropic WIF	quick-start.mdx:145-153	CLAUDE_CODE_OAUTH_TOKEN failure mode only in cli.md:240, not cross-linked from quick-start (high severity)
Codex	OPENAI_API_KEY/CODEX_API_KEY	quick-start.mdx:155-161	None significant
Gemini	GEMINI_API_KEY or Google WIF	quick-start.mdx:163-169	None significant
Pi	Depends on model: prefix	quick-start.mdx:171-175	Requires inferring secret from prefix (medium severity)

Recommended Actions

Priority 1 — Add a one-line inline warning in quick-start.mdx's Claude setup section (~line 145) cross-linking to the CLAUDE_CODE_OAUTH_TOKEN gap already documented at cli.md:240. This is the single most-repeated finding across 19 consecutive review runs.

Priority 2 — Extend gh aw init to scaffold an equivalent onboarding artifact for Claude/Codex (or explicitly document why Copilot alone gets one), and add a worked Anthropic WIF example alongside the existing link.

Priority 3 — Reconcile "Claude Code" vs engine: claude naming in quick-start; state explicitly which engines have web-search enabled by default; consider adding 2-3 more Claude smoke-test variants to close the coverage gap with Copilot.

Process note: this run also re-confirmed a 12-day-standing infra issue — .claude/agents/doc-reader.md and engine-example-counter.md lack required name: frontmatter and fail to register as invokable agents; write access to fix them is denied in this workflow's permission mode. Recommend a maintainer add name: doc-reader / name: engine-example-counter to those files directly.

Generated by 📝 Claude Code User Documentation Review · claude · agent · 193.3 AIC · ⌖ 8.51 AIC · ⊞ 6.4K · ◷

• expires on Sep 27, 2026, 4:44 AM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Claude Code User Documentation Review.

A newer discussion is available at Discussion #63838.