[claude-code-user-docs-review] 🔍 Claude Code User Documentation Review - 2026-09-24

[github-actions[bot]]

Executive Summary

Reviewed gh-aw docs from a Claude Code user's perspective (no GitHub Copilot, no Copilot CLI). No critical blockers remain — Claude Code is a fully supported, first-class engine — but the OAuth-token trap flagged in prior runs is still unresolved after 17 consecutive runs, and Claude's onboarding remains structurally second-class to Copilot's (manual init scaffolding, ~2x fewer example workflows).

Severity Findings

Critical / Major / Minor findings (click to expand)

Critical Blockers: None found this run.

Major Obstacles:

1. CLAUDE_CODE_OAUTH_TOKEN (the token from claude login, likely already held by subscription-based Claude Code users) is silently ignored — cli.md:240. The warning lives only in the secrets set troubleshooting subsection and is not cross-referenced from quick-start.mdx:145-153 or how-they-work.mdx:33, where a new Claude user looks first. Failure surfaces as a generic Claude CLI auth error that never names the token.
2. gh aw init only auto-scaffolds the custom agent file and MCP wiring for Copilot (cli.md:139-140 table). Non-Copilot guidance says to "author an agent file in your own agent's format" and "register gh aw mcp-server in your own MCP host configuration" (cli.md:133-144) with no concrete Claude Code example — only a link to an out-of-scope reference page.
3. Example/smoke-test parity gap persists: Claude has 55 example workflows vs Copilot's 108 (~2x) in .github/workflows/*.md; Codex (74) is better served than Claude.

Minor Confusion:

1. Web-search opt-out behavior is documented for Codex/Claude/Copilot but not Gemini/Pi (tools.md:128).
2. tools.timeout defaults are given only for Claude (60s) and Codex (120s); Copilot/Gemini/Pi are unstated (tools.md:235).
3. Pi engine resolves auth via a model: provider-prefix string rather than the engine: field — an easy-to-miss distinct mechanism (quick-start.mdx:171-175).
4. quick-start.mdx:72 nudges readers toward Copilot first ("requires no extra account setup"); the Claude section is otherwise equally complete.
5. Custom engine has zero standalone example workflows — the only id: custom declaration is nested inside a shared/include fragment (shared/genaiscript.md), not independently discoverable.

Engine & Tool Matrix

Engine	Setup	Examples	Auth	Score
Copilot	Auto-scaffolded by gh aw init (custom agent file + MCP wiring), default engine (cli.md:115,135-143)	108	Org billing (copilot-requests: write) or COPILOT_GITHUB_TOKEN (how-they-work.mdx:32)	5/5
Claude	Manual — self-author agent file & MCP registration, no worked example (cli.md:133-144)	55	ANTHROPIC_API_KEY or WIF; claude login OAuth token silently ignored (cli.md:240)	3/5
Codex	Same manual init gap as Claude (cli.md:133-144)	74	CODEX_API_KEY/OPENAI_API_KEY (quick-start.mdx:155-161)	4/5
Custom	No init scaffolding; fully user-defined	0 standalone	Depends entirely on the user's own engine binary/config	2/5

Tool support in docs/reference/tools.md is engine-agnostic for the large majority of entries (edit, github, bash, web-fetch, playwright, cache-memory, mcp-servers, etc. — tools.md:20-302); only web-search names differing per-engine mechanics explicitly, and none of tools.md's entries are Copilot-exclusive — the only true Copilot-only artifacts live in cli.md's init command, not in the tool reference itself.

Auth Gaps

• Claude: CLAUDE_CODE_OAUTH_TOKEN is unsupported and silently dropped (cli.md:240) — the single biggest trap for subscription-only Claude Code users, since it's the credential they're most likely to already hold. No guidance exists for developers without a billed Anthropic Console account.
• Copilot: COPILOT_GITHUB_TOKEN requires a fine-grained PAT with elevated Copilot Requests scope beyond the default ephemeral GITHUB_TOKEN (cli.md:161) — a Copilot-specific nuance with no parallel called out for other engines.
• Pi: Credentials are resolved indirectly via the model: field's provider prefix rather than the engine: field (quick-start.mdx:171-175) — undocumented as a "gotcha" anywhere in scope.

Recommended Actions

Priority 1: Cross-reference the CLAUDE_CODE_OAUTH_TOKEN warning (cli.md:240) into quick-start.mdx's Claude setup steps (145-153) and how-they-work.mdx's engine table (33), so it's visible before a user attempts setup rather than after a cryptic failure.

Priority 2: Add a worked Claude Code example for the manual init steps (custom agent file authoring + MCP host registration) referenced at cli.md:133-144, instead of only linking out.

Priority 3: Backfill Claude-engine example workflows toward parity with Copilot (55 vs 108), and document tools.timeout/web-search defaults for Gemini/Pi engines (tools.md:128,235).

References:

• §36000515558

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• api.anthropic.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "api.anthropic.com"

See Network Configuration for more information.

Generated by 📝 Claude Code User Documentation Review · claude · agent · 286.9 AIC · ⌖ 8.37 AIC · ⊞ 6.5K · ◷

• expires on Sep 25, 2026, 4:53 AM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Claude Code User Documentation Review.

A newer discussion is available at Discussion #63440.