🔍 Claude Code User Documentation Review - 2026-03-02

[github-actions[bot]]

Executive Summary

As a Claude Code user reviewing gh-aw documentation for the tenth consecutive day, the overall picture remains stable: the docs are broadly accessible to non-Copilot users, but contain persistent Copilot-centric framing in several specific locations. All 11 previously identified issues remain unresolved.

Key Finding: Claude Code users can successfully adopt gh-aw. There are zero critical blockers. The engine selection wizard, auth docs, and CLI commands are all multi-engine-friendly. However, architecture diagrams and a few page sections are still Copilot-centric, creating unnecessary friction for Claude users.

---

Persona Context

I reviewed this documentation as a developer who:

• ✅ Uses GitHub for version control
• ✅ Uses Claude Code as primary AI assistant
• ❌ Does NOT use GitHub Copilot
• ❌ Does NOT use Copilot CLI
• ❌ Does NOT have Copilot subscription

---

Question 1: Onboarding Experience

Can a Claude Code user understand and get started with gh-aw?

Yes. The Quick Start (docs/src/content/docs/setup/quick-start.mdx) is the strongest entry point for Claude users. Prerequisites explicitly list "Anthropic Claude" as a valid AI account. The gh aw add-wizard command presents an interactive engine selector—Copilot, Claude, or Codex—and walks through setting ANTHROPIC_API_KEY. No Copilot access is required to complete the Quick Start.

Specific Issues Found:

• Issue 1: creating-workflows.mdx line 21 opens the "GitHub Web Interface" section with "If you have access to GitHub Copilot, you can create and edit Agentic Workflows directly from the Web Interface." — This implies Claude users cannot use the web interface, with no alternative path shown for Claude/Codex users.
• Issue 2: creating-workflows.mdx step 3 says "If not using Copilot, also adjust the engine: field" — this frames Copilot as the default/normal path and Claude as the exception requiring extra action.

Recommended Fixes:

• Rewrite the web interface section to note it requires a Copilot subscription, and add a parallel prompt-based workflow for Claude Code users (/agent or direct prompt in Claude Code).
• Change step 3 phrasing to "Set the engine: field to copilot, claude, codex, or gemini depending on your AI provider."

---

Question 2: Inaccessible Features for Non-Copilot Users

What features or steps don't work without Copilot?

Features That Require Copilot:

• GitHub Web Interface workflow creation — Requires active Copilot subscription (creating-workflows.mdx)
• assign-to-agent safe output with name: copilot — Assigns Copilot coding agent to issues/PRs (assign-to-copilot.mdx); no Claude equivalent
• Custom Copilot agent files (.github/agents/*.agent.md) — Engine-specific Copilot feature; Claude engine doesn't use this mechanism

Features That Work Without Copilot (engine-agnostic):

• gh aw init, compile, run, list, status, logs, audit, health
• All tools: edit, github, bash, playwright, web-fetch, cache-memory, repo-memory, agentic-workflows
• All MCP server integrations
• Safe outputs (create-issue, add-comment, create-pull-request, etc.)
• Secrets management (gh aw secrets set/bootstrap)

Missing Documentation:

• No equivalent to Copilot's custom agent files for the Claude engine
• No documentation on Claude-specific args flags (e.g. --max-turns, --model)

---

Question 3: Documentation Gaps and Assumptions

Where does the documentation assume Copilot usage?

Copilot-Centric Language Found In:

• File: docs/src/content/docs/introduction/architecture.mdx lines 177, 205, 248 — Firewall and MCP gateway diagrams show "Copilot CLI" as the only agent type inside the AI agent process container. Claude Code users may think these diagrams don't apply to them.
• File: docs/src/content/docs/introduction/architecture.mdx line 224 — Only configuration example for network: uses engine: copilot; no Claude equivalent shown.
• File: docs/src/content/docs/introduction/how-they-work.mdx line 26 — Lists supported engines as "GitHub Copilot (default), Claude by Anthropic, and Codex" — omits Gemini, which is a supported engine documented in engines.md and auth.mdx.
• File: docs/src/content/docs/reference/engines.md line 75 — The "Extended Coding Agent Configuration" section shows model: gpt-5 # defaults to claude-sonnet-4 with id: copilot — the comment "defaults to claude-sonnet-4" is incorrect for the Copilot engine and confusingly mixes engine names.

Missing Alternative Instructions:

• No Claude Code equivalent for the GitHub web interface workflow creation path
• docs/src/content/docs/setup/cli.md documents many commands but does not include add-wizard, which is prominently featured in the Quick Start

---

Severity-Categorized Findings

🚫 Critical Blockers (Score: 0/10)

No critical blockers exist for Claude Code users. All core workflow creation, compilation, and execution paths work without Copilot.

⚠️ Major Obstacles (Score: 5 obstacles)

Obstacle 1: Architecture diagrams are Copilot-centric

Impact: Confusion when reading security architecture documentation

Current State: architecture.mdx lines 177/205 show COPILOT["Copilot CLI"] and line 248 shows AGENT["Agent container\nCopilot CLI + MCP client\n172.30.0.20"] as nodes in architecture diagrams that describe the general AWF system, not Copilot-specific behavior.

Why It's Problematic: Claude Code users reading security architecture docs will see "Copilot CLI" throughout and may believe the firewall/proxy system only works with Copilot. This is incorrect — these are engine-agnostic infrastructure components.

Suggested Fix: Replace "Copilot CLI" with "AI Engine (Copilot/Claude/Codex)" or "Coding Agent" in the diagrams. The MCP gateway diagram could say "Agent container\nCoding Agent + MCP client".

Affected Files: docs/src/content/docs/introduction/architecture.mdx

Obstacle 2: Creating workflows page gates web interface on Copilot

Impact: Claude Code users skip a valid workflow creation path

Current State: creating-workflows.mdx line 21: "If you have access to GitHub Copilot, you can create and edit Agentic Workflows directly from the Web Interface."

Why It's Problematic: A Claude Code user sees this conditional and skips the section entirely, missing context about how the workflow generation prompt pattern works (which is the same prompt pattern they'd use with Claude Code).

Suggested Fix: Split into "Using Copilot on GitHub.com" (requires Copilot subscription) and "Using Claude Code or other coding agents" (no subscription needed) subsections.

Affected Files: docs/src/content/docs/setup/creating-workflows.mdx

Obstacle 3: `add-wizard` command absent from CLI reference

Impact: Users cannot find documentation for a prominently featured command

Current State: quick-start.mdx tells users to run gh aw add-wizard githubnext/agentics/daily-repo-status as the primary onboarding step. However, cli.md contains no entry for add-wizard — it's not listed in the commands table or the Commands section.

Why It's Problematic: Users following the quick start, then turning to the CLI reference for more detail, find no documentation for this command.

Suggested Fix: Add an add-wizard section to cli.md documenting its behavior (interactive engine selection, secret setup, workflow addition, optional first run).

Affected Files: docs/src/content/docs/setup/cli.md, docs/src/content/docs/setup/quick-start.mdx

Obstacle 4: Incorrect ANTHROPIC_API_KEY setup URL

Impact: Claude users sent to documentation page instead of API key creation page

Current State: auth.mdx line 104: "Create an API key at (platform.claude.com/redacted)

Why It's Problematic: platform.claude.com/docs/en/get-started is a documentation page, not the API key creation interface. Claude users need to go to https://console.anthropic.com/settings/keys to create API keys. This is a dead-end for new users following setup instructions.

Suggested Fix: Update the URL to https://console.anthropic.com/settings/keys

Affected Files: docs/src/content/docs/reference/auth.mdx

Obstacle 5: Troubleshooting has zero Claude-specific entries

Impact: Claude users stuck on engine-specific errors have no guidance

Current State: The troubleshooting section (common-issues.md) contains multiple Copilot-specific entries (Copilot CLI Not Found, Copilot License or Inference Access Issues) but zero Claude-specific entries.

Why It's Problematic: Claude users hitting errors like invalid API key format, rate limits, or model availability issues have no documentation to consult.

Suggested Fix: Add a "Claude / Anthropic Issues" troubleshooting section with common errors: invalid ANTHROPIC_API_KEY, rate limit handling, and model availability.

Affected Files: docs/src/content/docs/troubleshooting/common-issues.md

💡 Minor Confusion Points (11 total)

• Issue 1: how-they-work.mdx line 26 — Lists engines as "Copilot (default), Claude, and Codex" — omits Gemini (now a 4th supported engine)
• Issue 2: engines.md line 75 — model: gpt-5 # defaults to claude-sonnet-4 with id: copilot — comment mixes engine names incorrectly
• Issue 3: architecture.mdx line 224 — Network config example shows only engine: copilot, no Claude equivalent
• Issue 4: creating-workflows.mdx step 3 — Frames Claude as exception ("If not using Copilot, also adjust..."), Copilot as the norm
• Issue 5: creating-workflows.mdx line 77 — Links to "VSCode Agent Mode" (Copilot) docs without Claude Code equivalent link
• Issue 6: Web search guide (web-search.md) — Only shows Copilot engine examples; no Claude equivalent
• Issue 7: cli.md line 161 — secrets bootstrap --engine lists (copilot, claude, codex) — omits Gemini
• Issue 8: custom-agent-for-aw.mdx — Uses /agent agentic-workflows slash command syntax (Copilot Chat/VSCode Agent Mode specific) with no Claude Code equivalent documented
• Issue 9: cli.md secrets example — Shows gh aw secrets set COPILOT_GITHUB_TOKEN as the example command; could use a neutral example
• Issue 10: README implies Copilot by linking to "Related Projects" (AWF) diagram that shows Copilot-centric labels
• Issue 11: how-they-work.mdx "Copilot (default)" — The "(default)" label may confuse new users into thinking Copilot is required

---

Engine Comparison Analysis

Available Engines

• engine: copilot — Excellent documentation; detailed troubleshooting; custom agent files feature
• engine: claude — Good documentation; clear auth setup (except URL); 36 real examples
• engine: codex — Adequate documentation; fewer examples
• engine: gemini — Exists in engines.md and auth.mdx but missing from how-they-work.mdx summary and secrets bootstrap CLI help text

Documentation Quality by Engine

Engine	Setup Docs	Examples	Auth Docs	Overall Score
Copilot	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐	⭐⭐⭐⭐⭐
Claude	⭐⭐⭐⭐	⭐⭐⭐⭐	⭐⭐⭐⭐	⭐⭐⭐⭐
Codex	⭐⭐⭐	⭐⭐⭐	⭐⭐⭐⭐	⭐⭐⭐
Gemini	⭐⭐⭐	⭐⭐	⭐⭐⭐⭐	⭐⭐⭐

Rating Scale: ⭐⭐⭐⭐⭐ (Excellent) to ⭐ (Poor/Missing)

---

Tool Availability Analysis

Engine-Agnostic Tools (work with any engine):

• edit — File editing
• github — GitHub API operations (all toolsets)
• bash — Shell execution
• web-fetch — Web content fetching
• playwright — Browser automation
• cache-memory — Persistent cross-run memory
• repo-memory — Repository-specific memory
• agentic-workflows — Workflow introspection
• Custom MCP servers (all mcp-servers: configurations)

Engine-Specific Features:

• Copilot agent: field — Custom agent files (.github/agents/*.agent.md); no Claude equivalent
• Copilot assign-to-agent safe output — Assigns GitHub Copilot to issues; no Claude equivalent
• web-search: — Works with all engines but some require additional MCP server setup (documentation unclear on which engines)

Unclear/Undocumented:

• web-search: engine compatibility — "Some engines require third-party MCP servers" (tools.md) but doesn't specify which engines need extra setup

---

Authentication Requirements

Current Documentation Coverage

• ✅ Copilot: COPILOT_GITHUB_TOKEN — Detailed instructions with fine-grained PAT setup link
• ✅ Claude: ANTHROPIC_API_KEY — Instructions present but URL points to docs page instead of API key creation page
• ✅ Codex: OPENAI_API_KEY — Clear instructions
• ✅ Gemini: GEMINI_API_KEY — Clear instructions

Missing for Claude Users

• Correct URL for API key creation: should be https://console.anthropic.com/settings/keys, not `(platform.claude.com/redacted)
• No Claude-specific troubleshooting for auth failures (contrast with detailed Copilot license troubleshooting)

Secret Names

Engine	Secret Name	Documented
Copilot	COPILOT_GITHUB_TOKEN	✅ Detailed
Claude	ANTHROPIC_API_KEY	✅ Present (URL needs fix)
Codex	OPENAI_API_KEY	✅ Clear
Gemini	GEMINI_API_KEY	✅ Clear

---

Example Workflow Analysis

Workflow Count by Engine

Engine: copilot  - ~82 workflows
Engine: claude   - 36 workflows (~22%)
Engine: codex    - ~11 workflows
Engine: gemini   - 0 simple engine: gemini lines (some may use block form)
Unspecified/default - ~34 workflows
Total: ~163 workflows

(Note: grep results were truncated; counts are approximate)

Quality of Examples

Claude Examples: 36 workflows covering security analysis, code metrics, doc updates, MCP integrations (Azure, Fabric RTI), safe output optimization, and more. Sufficient diversity. Real production workflows running in this repo (including this very review workflow claude-code-user-docs-review.md).

Copilot Examples: ~82 workflows covering the broadest range of use cases. Serves as the de facto "gallery" of what gh-aw can do.

Codex Examples: ~11 workflows; adequate for demonstrating the engine works but narrower coverage.

---

Recommended Actions

Priority 1: Critical Documentation Fixes

1. Fix ANTHROPIC_API_KEY URL — Change (platform.claude.com/redacted) to https://console.anthropic.com/settings/keys` in auth.mdx — directly blocks Claude users from completing setup
2. Document add-wizard in CLI reference — Add entry to cli.md for the command featured prominently in Quick Start — leaves users with no reference docs for a key command
3. Update architecture diagrams — Replace "Copilot CLI" with "Coding Agent" in architecture.mdx diagrams — prevents Claude users from understanding that security architecture applies to them

Priority 2: Major Improvements

1. Rewrite creating-workflows.mdx web interface section — Add a parallel Claude Code path alongside the Copilot-gated web interface section — removes the gatekeeping language that implies non-Copilot users can't use this path
2. Add Claude troubleshooting section — Add at least 3-4 entries to common-issues.md for Claude-specific errors — currently no guidance for the second most documented engine
3. Add Gemini to how-they-work.mdx — Update line 26 to include Gemini as a fourth engine — inconsistency between overview page and full reference docs

Priority 3: Nice-to-Have Enhancements

1. Add Claude Code equivalent to custom-agent-for-aw.mdx — Document how Claude Code users configure their agent behavior (system prompt, CLAUDE.md, etc.)
2. Add Gemini to secrets bootstrap --engine help text — Update cli.md line 161 to include gemini in the engine list
3. Fix confusing model comment in engines.md — Change model: gpt-5 # defaults to claude-sonnet-4 under the Copilot section — the comment incorrectly implies Copilot defaults to a Claude model

---

Positive Findings

What Works Well for Claude Code Users

• ✅ Quick Start prerequisites are engine-inclusive — explicitly lists "Anthropic Claude" alongside Copilot
• ✅ add-wizard is the best onboarding path — interactive, asks for engine, sets correct secrets automatically
• ✅ Auth reference is comprehensive — covers all 4 engines with equal structure and detail
• ✅ All core tools are engine-agnostic — edit, github, bash, playwright, cache-memory all work identically for Claude
• ✅ 36 real Claude workflows — rich, production-quality examples in the same repo
• ✅ CLI commands are fully multi-engine — init, compile, run, logs, audit all work without modification
• ✅ Engines reference has dedicated Claude section — engine: claude frontmatter is clearly documented
• ✅ FAQ addresses Claude auth confusion — explicitly says CLAUDE_CODE_OAUTH_TOKEN is not supported, use ANTHROPIC_API_KEY
• ✅ This review is itself a Claude workflow — claude-code-user-docs-review.md uses engine: claude and runs daily, demonstrating production-grade Claude usage

---

Conclusion

Can Claude Code Users Successfully Adopt gh-aw?

Answer: Yes, with minor friction

A Claude Code user who follows the Quick Start (gh aw add-wizard) will successfully set up and run their first workflow without any Copilot dependency. The authentication docs cover Claude correctly (aside from the URL issue). All core CLI commands, tools, and MCP integrations are fully engine-agnostic.

The friction comes from: (1) having to navigate past Copilot-first language in a few specific pages, (2) the incorrect Anthropic API key URL, and (3) the absence of Claude-specific troubleshooting. None of these are show-stoppers, but collectively they create a slightly unwelcoming experience for users who have consciously chosen Claude over Copilot.

Overall Assessment Score: 7/10 (stable — 10th consecutive day)

Breakdown:

• Clarity for non-Copilot users: 7/10
• Claude engine documentation: 8/10
• Alternative approaches provided: 7/10
• Engine parity: 7/10

Trend Note

This score has been 7/10 for 10 consecutive days. All 11 identified issues have been reported in previous reviews. The stability suggests these are either accepted design trade-offs (Copilot-first since Copilot is the default engine) or items in a backlog. The most actionable quick win remains fixing the ANTHROPIC_API_KEY URL in auth.mdx.

---

Appendix: Files Reviewed

Complete List of Documentation Files Analyzed

• README.md
• docs/src/content/docs/setup/quick-start.mdx
• docs/src/content/docs/setup/creating-workflows.mdx
• docs/src/content/docs/setup/cli.md
• docs/src/content/docs/introduction/how-they-work.mdx
• docs/src/content/docs/introduction/architecture.mdx
• docs/src/content/docs/introduction/overview.mdx
• docs/src/content/docs/reference/tools.md
• docs/src/content/docs/reference/engines.md
• docs/src/content/docs/reference/auth.mdx
• docs/src/content/docs/reference/frontmatter.md
• docs/src/content/docs/reference/assign-to-copilot.mdx
• .github/workflows/*.md (engine field survey, truncated at ~163 files)

---

Report Generated: 22598742665
Workflow: claude-code-user-docs-review
Engine Used: claude (eating our own dog food 🐕)
Consecutive days at score 7/10: 10

AI generated by Claude Code User Documentation Review

• expires on Mar 3, 2026, 10:38 PM UTC

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-03-03T22:38:58.042Z.

Closed by Workflow