[claude-code-user-docs-review] 🔍 Claude Code User Documentation Review - 2026-09-16

[github-actions[bot]]

Executive Summary

Day 25 of daily tracking: 0 critical blockers, 3 major obstacles, 2 minor confusions for a Claude-Code-only user (no Copilot access) trying to adopt gh-aw. Onboarding is still Copilot-free-capable overall, but the CLAUDE_CODE_OAUTH_TOKEN silent-ignore gap remains unresolved for a 9th consecutive run, and quick-start.mdx's Claude auth tab still has no inline OAuth/WIF mention at all — unlike the Gemini tab, which does. Separately, this run's own custom-agent tooling (doc-reader, engine-example-counter, reporting skill) failed to register due to missing name: frontmatter — filed via missing_tool; facts below were still gathered via fallback agents.

Severity Findings (click to expand)

Critical Blockers: None (holds at 0 since the 2026-09-08 downgrade).

Major Obstacles:

1. CLAUDE_CODE_OAUTH_TOKEN (incl. tokens from claude login) is silently ignored, documented only as plain prose at docs/src/content/docs/setup/cli.md:240 — no admonition/callout, unlike other warnings in the same doc.
2. docs/src/content/docs/setup/quick-start.mdx:133-137 (Claude tab) never mentions the OAuth-token trap or the WIF alternative at all; the Gemini tab (:149-153) explicitly surfaces its own WIF alternative — asymmetric treatment.
3. Example/smoke-test parity: repo-wide engine: claude = 54 vs engine: copilot = 106 (~51%, up slightly from yesterday's ~48%, but both totals dropped so the absolute gap is essentially flat). Sharpest gap: Copilot has 7 auth/config-variant smoke tests (smoke-copilot-aoai-apikey.md, -aoai-entra.md, -arm.md, -auto.md, -mai.md, -sdk.md, -small.md) vs 0 Claude equivalents.

Minor Confusion:

1. docs/src/content/docs/introduction/architecture.mdx:17 again lists only 4 engines (omits Pi) — reported resolved on 2026-09-14; today's read shows it flipped back. Needs re-verification, possibly a flaky read.
2. gh aw init only auto-scaffolds a custom agent file + MCP config for Copilot, not Claude/Codex/Gemini/Pi (docs/src/content/docs/setup/cli.md:135-140) — carried from yesterday, downgraded from co-major since it's a documented asymmetry rather than a blocker.

Engine & Tool Matrix

Engine	Setup clarity	Examples (repo-wide)	Auth	Score*
Copilot	Clear, wizard default; copilot-requests:write or COPILOT_GITHUB_TOKEN (quick-start.mdx:123-131)	106	Built-in GH token or fine-grained PAT	9/10
Claude	Partial; ANTHROPIC_API_KEY shown, OAuth-token trap and WIF undocumented inline (quick-start.mdx:135-137; cli.md:240)	54	API key or WIF (link-only, no inline example)	6/10
Codex	Clear; OPENAI_API_KEY/CODEX_API_KEY (quick-start.mdx:141-145)	74	API key	7/10
Pi	Newer; requires cli-proxy: true + github.mode: gh-proxy, spec split across 2 docs (tools.md:217; quick-start.mdx:159)	31 (now a real 5th engine)	Varies by model: prefix	6/10
Custom	No live top-level examples (only an unused shared/genaiscript.md include)	0	N/A	n/a

*Score = subjective 1–10 friction rating for a Claude-Code-only user, tracked for trend purposes.

Tool classification (docs/src/content/docs/reference/tools.md): 12 of ~14 documented tools are engine-agnostic (edit, github, linear, jira, bash, playwright, cache-memory, drive-memory [preview-gated], repo-memory, qmd, agentic-workflows, mcp-servers). Only web-search is explicitly engine-dependent, and only for Codex (:116-128) — no equivalent statement exists for Claude/Gemini/Pi, making full cross-engine support ambiguous. cli-proxy is required: true specifically for Pi, but that requirement is only visible by cross-referencing tools.md:217 against quick-start.mdx:159.

Parity trend: ratio narrowed slightly (51% vs 48%) but both totals dropped, so overall imbalance is flat; daily-workflow gap remains ~2.2x (15 claude vs ~33 copilot files); Pi has emerged as a substantial 5th engine (31 examples) worth tracking going forward alongside claude/copilot/codex.

Auth Gaps

• No admonition-style callout for the CLAUDE_CODE_OAUTH_TOKEN trap (cli.md:240), despite other warnings in the same doc using blockquote callouts.
• quick-start.mdx:133-137 (Claude tab) omits any OAuth/WIF mention; the Gemini tab (:149-153) does surface WIF — inconsistent across tabs.
• A full worked Anthropic WIF YAML example exists, but only at docs/src/content/docs/reference/auth.mdx:237-257 — cli.md and quick-start.mdx only link out to it.
• Confirmed required secrets: Copilot (copilot-requests:write or COPILOT_GITHUB_TOKEN), Claude (ANTHROPIC_API_KEY or WIF), Codex (CODEX_API_KEY/OPENAI_API_KEY), Gemini (GEMINI_API_KEY or WIF), Pi (varies by model: prefix + cli-proxy: true).

Recommended Actions

Priority 1: Add a Starlight :::caution-style admonition at cli.md:240 stating plainly that CLAUDE_CODE_OAUTH_TOKEN (including claude login tokens) is unsupported and to use ANTHROPIC_API_KEY or WIF instead; mirror one line of it in quick-start.mdx's Claude tab (133-137) the way the Gemini tab already does for WIF.

Priority 2: Inline the existing Anthropic WIF YAML example (auth.mdx:237-257) into cli.md/quick-start.mdx instead of link-only; add 2-3 Claude-engine auth/config smoke-test variants mirroring smoke-copilot-aoai-*.md to close the sharpest advanced-config parity gap.

Priority 3: Re-verify architecture.mdx:17's engine list (flagged omitting Pi again after being reported fixed on 2026-09-14); add an explicit note to cli.md that gh aw init scaffolding is Copilot-only.

Review tooling note: .claude/agents/doc-reader.md, .claude/agents/engine-example-counter.md, and .claude/skills/reporting.md are all missing a name: frontmatter field, so none registered as callable this run (filed via missing_tool). Adding name: to each would let future runs use the intended dedicated tooling instead of general-purpose fallback agents.

References:

• §35097042647 (this run)
• §34970007560 (2026-09-15, prior run)
• §34227198249 (2026-09-08, OAuth-token gap downgraded Critical→Major)

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• api.anthropic.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "api.anthropic.com"

See Network Configuration for more information.

Generated by 📝 Claude Code User Documentation Review · claude · agent · 328 AIC · ⌖ 7.9 AIC · ⊞ 6.3K · ◷

• expires on Sep 17, 2026, 4:51 AM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Claude Code User Documentation Review.

A newer discussion is available at Discussion #61598.