[claude-code-user-docs-review] Claude Code User Documentation Review - 2026-09-06

[github-actions[bot]]

Executive Summary

gh-aw documents five engines (Copilot, Claude Code, Codex, Gemini, Pi) with genuinely parallel setup instructions in quick-start.mdx, so this is not a Copilot-only tool on paper. The blocker is narrower but sharp: a Claude Code user who authenticates the way they normally would, via claude login, hits a silent failure. CLAUDE_CODE_OAUTH_TOKEN is accepted nowhere, is silently ignored, and the resulting error never mentions the token. This has now been confirmed present and unresolved across 15 consecutive daily reviews since 2026-08-22 (docs/src/content/docs/setup/cli.md:240).

Severity Findings

Critical Blockers (1), Major Obstacles (4), Minor Confusion (3)

Critical Blockers

1. CLAUDE_CODE_OAUTH_TOKEN silently ignored, misleading auth error - cli.md:240 states the token (including one from claude login) is not supported; it is silently ignored, so the run instead fails with an authentication error from the Claude CLI that never mentions the token. Only ANTHROPIC_API_KEY or Anthropic WIF work. A Claude Code user with an existing subscription session has no supported path and gets a generic, unrelated error instead of a message pointing at the real cause. Day 15 unresolved (first flagged 2026-08-22).

Major Obstacles

1. No warning where it matters - the Claude auth tab in quick-start.mdx:134-138 (the first place a new user configures Claude auth) says nothing about the OAuth-token pitfall; the only mention lives in cli.md:240 under the secrets set subcommand, which a quick-start-only reader never reaches.
2. Copilot-only scaffolding from gh aw init - cli.md:133-140's comparison table shows the custom agent file (.github/agents/agentic-workflows.md) and MCP wiring (.github/mcp.json, copilot-setup-steps.yml) are generated only for Copilot; Claude/Codex/Gemini/Pi all show an X, with no template or example given for authoring the Claude equivalent (cli.md:139: author an agent file in your own agent's format).
3. web-search tool support ambiguous per engine - tools.md:144 documents only that Codex disables it by default; Claude/Copilot/Gemini/Pi behavior is unstated, forcing guesswork.
4. Example coverage skew - engine-example-counter found 106 Copilot, 94 Codex, but only 56 Claude workflow examples across .github/workflows/*.md, roughly half of Copilot's count, though not a severe gap.

Minor Confusion

1. tools.timeout defaults given only for Claude (60s) and Codex (120s); Copilot/Gemini/Pi left unstated (tools.md:255).
2. drive-memory tool is private-preview/feature-gated with no engine-support detail (tools.md:171-176).
3. Quick-start prerequisites frame Copilot as needing no extra account setup (quick-start.mdx:73), subtly positioning Claude as the higher-friction option even though its actual setup (one API key plus one gh secret set) is comparably simple.

Engine and Tool Matrix

Engine	Setup	Examples	Auth	Score
Copilot	Default via gh aw init; auto-scaffolds custom agent + MCP wiring (cli.md:133-140)	106 files	copilot-requests: write perm or PAT (COPILOT_GITHUB_TOKEN)	5/5
Claude	--engine claude flag; no agent-file scaffold, manual authoring required (cli.md:139)	56 files	ANTHROPIC_API_KEY/WIF only - OAuth token silently ignored	3/5
Codex	--engine codex flag, clear docs	94 files (near parity with Copilot)	CODEX_API_KEY (precedence) or OPENAI_API_KEY	4/5
Custom (Gemini/Pi/other)	Manual engine id, per-engine extra config (e.g. Pi needs gh-proxy + cli-proxy)	27 files, thinly spread across ~10 experimental engines (mostly smoke-tests)	Derived from provider prefix or engine-specific key	2/5

Tool classification (tools.md): 13 of 16 reviewed tools/settings are engine-agnostic (shared); 3 are ambiguous (web-search, drive-memory, tools.timeout defaults) where per-engine behavior isn't fully spelled out.

Parity note (engine-example-counter): parsing used regex-based frontmatter matching (not full YAML) across 299 workflow files; 16 files had no detected engine key (likely relying on system default). Codex is essentially at parity with Copilot in raw example count - Claude is the engine with the most room to grow, not a documentation void.

Auth Gaps

• cli.md:240: CLAUDE_CODE_OAUTH_TOKEN unsupported and silently ignored - confirmed present, unresolved 15 runs.
• No cross-link from quick-start.mdx:134-138 (Claude tab) to the cli.md:240 warning - the two docs a new user actually reads don't connect.
• The auth reference page linked from quick-start.mdx:137 was out of scope for this review's file set - unconfirmed whether it repeats or resolves the gap.
• Required secrets are otherwise clearly enumerated per engine in how-they-work.mdx:32-36 and quick-start.mdx:123-157.

Recommended Actions

Priority 1

• Fix or clearly surface the CLAUDE_CODE_OAUTH_TOKEN gap: either support the token, or make the Claude CLI's auth error explicitly name it. Add a one-line warning directly in quick-start.mdx's Claude tab, not just cli.md:240.

Priority 2

• Provide a minimal Claude agent-file template/example alongside the existing Copilot scaffold generated by gh aw init --engine claude (currently cli.md:139 just says author your own).
• Document per-engine web-search support explicitly in tools.md instead of only covering Codex's default-off behavior.

Priority 3

• Fill in missing per-engine tools.timeout defaults (Copilot/Gemini/Pi) in tools.md:255.
• Add Claude/Codex example workflows to close the remaining gap vs. Copilot's 106.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• api.anthropic.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "api.anthropic.com"

See Network Configuration for more information.

Generated by 📝 Claude Code User Documentation Review · claude · agent · 273.3 AIC · ⌖ 8.81 AIC · ⊞ 6.3K · ◷

• expires on Sep 7, 2026, 4:49 AM UTC-08:00