[claude-code-user-docs-review] Claude Code User Documentation Review - 2026-09-09

[github-actions[bot]]

Executive Summary

A Claude Code-only developer (no Copilot) can complete gh-aw setup end-to-end, but the docs remain structurally Copilot-centric: Copilot is the default engine, gets auto-generated integration artifacts, and has ~2x the example workflows Claude does. The long-standing CLAUDE_CODE_OAUTH_TOKEN trap (flagged across 17 straight daily runs since 2026-08-22) stays explicitly documented in prose (cli.md:240) — holding at Major for a 2nd consecutive run, down from Critical. No new Critical blockers found today.

Severity Findings

Critical Blockers (0) · Major Obstacles (4) · Minor Confusion (4)

Critical Blockers
None this run. (Runs from 2026-08-22–2026-09-07 rated the silent CLAUDE_CODE_OAUTH_TOKEN failure Critical; cli.md:240 now states explicitly that the token "is not supported... silently ignored, so the run instead fails with an authentication error... which is the signal to switch credentials.")

Major Obstacles

1. No CI bridge for local claude login sessions — only ANTHROPIC_API_KEY or Anthropic WIF work in Actions; an existing Claude Pro/Max login can't be reused (docs/src/content/docs/setup/cli.md:240, quick-start.mdx:135-137).
2. Copilot-only automation gap — gh aw init auto-generates .github/agents/agentic-workflows.md and .github/mcp.json/copilot-setup-steps.yml only for Copilot; Claude users get one sentence ("author an agent file in your own agent's format... Register gh aw mcp-server in your own MCP host configuration") with no template (cli.md:139-140).
3. gh aw doctor doesn't validate engine secrets — it checks gh auth/repo state but not ANTHROPIC_API_KEY, so a broken Claude auth setup isn't caught before a workflow run fails (cli.md:256-273).
4. Example imbalance — 56 Claude example workflows vs 107 Copilot (~2x) and 74 Codex; Claude is the 3rd-most-represented engine in .github/workflows/*.md despite first-class support.

Minor Confusion

1. Pi is a listed engine (how-they-work.mdx:26) but missing from the prerequisites AI-account list (quick-start.mdx:73) and from architecture.mdx:17,304.
2. tools.timeout defaults are given only for Claude (60s) and Codex (120s), not Copilot/Gemini/Pi, so it's unclear if Claude's default is atypical (tools.md:255).
3. The generic engine diagram node is literally IDed COPILOT["Agent CLI"] even though it represents any engine (architecture.mdx:206).
4. "If you already have GitHub Copilot, start there — it requires no extra account setup" (quick-start.mdx:73) frames Claude-only users as the exception path.

Engine & Tool Matrix

Engine	Setup	Examples	Auth	Score
Copilot	Auto: custom agent + MCP wiring generated by default (cli.md:115,139-140)	107 workflows (~38%)	copilot-requests:write perm OR COPILOT_GITHUB_TOKEN PAT; org-billing path documented in depth (quick-start.mdx:124-133)	9/10
Claude	Manual agent/MCP authoring, one-line pointer only (cli.md:139-140)	56 workflows (~20%)	ANTHROPIC_API_KEY or WIF; CLAUDE_CODE_OAUTH_TOKEN explicitly unsupported (cli.md:240)	6/10
Codex	Same manual-artifact gap as Claude	74 workflows (~26%)	CODEX_API_KEY (precedence) or OPENAI_API_KEY	6/10
Custom (Pi, etc.)	No generated artifacts; Pi needs github.mode: gh-proxy + cli-proxy: true	46 workflows (31 = Pi)	Credential chosen by model: provider prefix	5/10

tools.md is the one genuinely engine-neutral document (0 Copilot-specific mentions). Two CLI-generated artifacts remain Copilot-only: the custom agent file and MCP wiring (cli.md:139-140). Non-Claude example workflows (Copilot+Codex+custom = 227) outnumber Claude 4:1, so a Claude-only contributor has fewer reference patterns to copy.

Auth Gaps

• Claude: ANTHROPIC_API_KEY (console key) or WIF only; claude login OAuth token silently ignored — documented at cli.md:240 but still no dedicated troubleshooting callout, and WIF setup is link-only (unresolved since 2026-08-22, downgraded Critical→Major on 2026-09-08, held at Major again today).
• Copilot: two paths (org-billing permission, or PAT) both walked through step-by-step in quick-start.mdx:124-133 — no Claude equivalent depth exists.
• gh aw secrets bootstrap auto-detects engines and prompts for missing secrets (cli.md:242-254), but gh aw doctor does not validate ANTHROPIC_API_KEY presence/validity (cli.md:256-273) — the two commands' scopes aren't cross-referenced for users.

Recommended Actions

Priority 1

• Add a dedicated "Claude Code Authentication" troubleshooting section (not just inline prose) at cli.md:240, with a worked WIF example instead of a bare link.
• Extend gh aw doctor to validate ANTHROPIC_API_KEY/engine secrets, mirroring secrets bootstrap's engine detection.

Priority 2

• Generate a Claude Code subagent/skill template via gh aw init --engine claude, matching the Copilot custom-agent scaffold (cli.md:139).
• Add more Claude example workflows under .github/workflows/ to close the ~2x gap with Copilot.

Priority 3

• Add Pi to the prerequisites list and architecture diagrams for consistency.
• Rename the COPILOT mermaid node to a neutral ENGINE ID.
• Reframe quick-start.mdx:73's "start with Copilot" framing to present all five engines neutrally.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• api.anthropic.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "api.anthropic.com"

See Network Configuration for more information.

Generated by 📝 Claude Code User Documentation Review · claude · agent · 389.8 AIC · ⌖ 8.34 AIC · ⊞ 6.3K · ◷

• expires on Sep 10, 2026, 4:52 AM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Claude Code User Documentation Review.

A newer discussion is available at Discussion #59973.