Summary
Manual inspection of the gh-aw CLI help output (/tmp/gh-aw/agent/all-help.txt, 426 per-command files in /tmp/gh-aw/agent/help-output/) and comparison against docs/src/content/docs/setup/cli.md found 4 confirmed issues: 1 high, 2 medium, 1 low.
Methodology note: The task specified running three "Rig custom harness" invocations (typo/grammar, flag-consistency, docs-vs-help). These were attempted but could not execute in this sandboxed session — the Rig launcher (.github/skills/rig/rig.ts) requires a COPILOT_CONNECTION_TOKEN for its Copilot SDK client, which is only injected into an internal driver subprocess environment and is not exposed to the workflow shell, causing AUTHENTICATION_FAILED on every invocation attempt. This is an environment/infrastructure limitation, not a CLI defect. To still meet the inspection goal, the same three analysis areas were performed manually using grep/view against the same pre-collected help dataset, which is an equivalent (if slower) method for these categories of findings.
Severity Breakdown
| Severity |
Count |
| High |
1 |
| Medium |
2 |
| Low |
1 |
docs-vs-help
1. [High] Broken Markdown table — explanatory sentence injected mid-table
Location: docs/src/content/docs/setup/cli.md, lines 126–132 (Global Options table)
Quoted text:
| Flag | Description |
|------|-------------|
| `-h`, `--help` | Show help (`gh aw help [command]` for command-specific help) |
For invalid nested command paths, `gh aw` now fails explicitly instead of falling back to parent help output. For example, `gh aw secrets gh --help` returns an unknown-command error rather than reprinting `gh aw secrets` help.
| `-v`, `--verbose` | Enable verbose output showing detailed information |
| `--banner` | Display ASCII logo banner with purple GitHub color theme |
Expected: A well-formed Markdown table with all three rows (-h/--help, -v/--verbose, --banner) contiguous, with the explanatory prose placed before or after the table (not inside it).
Actual: A full paragraph of prose is inserted between the first and second table rows, which breaks Markdown table parsing — the -v, --verbose and --banner rows will likely render as plain text or a second, malformed table rather than being part of the same table.
Suggested fix: Move the "For invalid nested command paths..." sentence to a paragraph immediately after the table (or before it), so the three flag rows remain contiguous.
Priority: High (visibly broken rendering in published docs).
2. [Medium] forecast documented Options list is missing --concurrency
Location: docs/src/content/docs/setup/cli.md, line 661 (forecast command Options); actual flag confirmed in /tmp/gh-aw/agent/help-output/forecast--.txt
Quoted text (actual CLI help):
--concurrency int Maximum number of concurrent usage-artifact downloads (0 = use default)
Expected: The forecast section's Options line in the docs should list all real flags, including --concurrency, matching the pattern used for --days, --eval, --period, --sample, --timeout, --json/-j, --repo/-r.
Actual: --concurrency is omitted from the documented Options list for forecast.
Suggested fix: Add --concurrency (with its description) to the forecast Options list in the docs.
Priority: Medium (undocumented but real, useful flag; users tuning download parallelism won't discover it from docs).
3. [Medium] Inconsistent --repo flag description across commands: [HOST/]owner/repo format vs owner/repo format
Location: Multiple commands' help output in /tmp/gh-aw/agent/all-help.txt
Quoted text:
-r, --repo string Target repository ([HOST/]owner/repo format). Defaults to current repository # most commands (e.g. enable, disable, update, upgrade, secrets set)
-r, --repo string Target repository (owner/repo format) # doctor
-r, --repo string Target repository (owner/repo format) # env get
-r, --repo string Target repository (owner/repo format) # env update (as well, minor variant)
-r, --repo string Target repository (owner/repo format). Defaults to current repository # secrets bootstrap (variant without [HOST/])
Expected: A single, consistent description format for the --repo/-r flag across all commands (either all should support and document [HOST/]owner/repo GitHub Enterprise Server host prefixes, or the docs should explain why some commands only accept owner/repo without a host).
Actual: doctor, env get, and env update document --repo as plain owner/repo format (no [HOST/] prefix), while most other commands (e.g. enable, disable, update, upgrade, deploy) document it as [HOST/]owner/repo format. It's unclear whether this reflects an actual functional difference (i.e., doctor/env truly don't support GHE host prefixes) or is simply inconsistent wording for the same behavior.
Suggested fix: Either (a) make the flag help text consistent across all commands if the behavior is identical, or (b) if doctor/env get/env update genuinely don't support [HOST/] prefixes, keep the distinction but call it out explicitly (e.g., in a changelog/dev note) so it's clearly intentional rather than a documentation oversight.
Priority: Medium (could mislead GHES users about supported repo targeting syntax for specific commands).
flag-consistency
4. [Low] -l short flag mapped to two different long flags across commands
Location: project new vs logs/other commands using -l
Quoted text:
-l, --link string Repository to link project to (format: owner/repo) # project new
-l, --logical-repo string ... # (another command, e.g. logs)
Expected: Ideally, short flags map to the same long-flag concept across the CLI for muscle-memory consistency, though this is a minor/cosmetic concern since -l is scoped per-command (no functional conflict, as Cobra flags are command-local).
Actual: -l is used for --link in project new and for --logical-repo in another command context, which could cause minor confusion for users who habitually use -l across different gh aw subcommands.
Suggested fix: No functional fix required; consider documenting short-flag meanings are command-scoped, or align short-flag choices where the same short letter is used for unrelated concepts in different commands, if maintaining a mental map across commands is a documentation goal.
Priority: Low (cosmetic, no functional bug).
Inspection Metadata
- Commands inspected: init, add (add-wizard), new, secrets (set, bootstrap, get via env), doctor, fix, compile, validate, lint, trial, run, list, status, logs, audit, outcomes, health, checks, forecast, experiments, enable, disable, remove, update, deploy, upgrade, env (get, update), mcp, mcp-server, pr transfer, domains, version, completion, project (new), hash-frontmatter — using the pre-collected
/tmp/gh-aw/agent/all-help.txt (21,768 lines) and 426 per-command files under /tmp/gh-aw/agent/help-output/.
- Docs file compared:
docs/src/content/docs/setup/cli.md (1017 lines), full document reviewed section by section (Global Options through Related Documentation).
- Method: Manual
grep/pattern analysis in lieu of the requested Rig custom harness invocations, which failed with AUTHENTICATION_FAILED due to a missing COPILOT_CONNECTION_TOKEN in the workflow shell environment (token is scoped to an internal driver subprocess only). No repository code, workflow, or infrastructure files were modified; all commands used were read-only (grep, view, sed).
- Date: Run ID 30820310346.
Generated by ✅ CLI Consistency Checker · auto · 224.9 AIC · ⌖ 5.71 AIC · ⊞ 7.3K · ◷
Summary
Manual inspection of the
gh-awCLI help output (/tmp/gh-aw/agent/all-help.txt, 426 per-command files in/tmp/gh-aw/agent/help-output/) and comparison againstdocs/src/content/docs/setup/cli.mdfound 4 confirmed issues: 1 high, 2 medium, 1 low.Methodology note: The task specified running three "Rig custom harness" invocations (typo/grammar, flag-consistency, docs-vs-help). These were attempted but could not execute in this sandboxed session — the Rig launcher (
.github/skills/rig/rig.ts) requires aCOPILOT_CONNECTION_TOKENfor its Copilot SDK client, which is only injected into an internal driver subprocess environment and is not exposed to the workflow shell, causingAUTHENTICATION_FAILEDon every invocation attempt. This is an environment/infrastructure limitation, not a CLI defect. To still meet the inspection goal, the same three analysis areas were performed manually usinggrep/viewagainst the same pre-collected help dataset, which is an equivalent (if slower) method for these categories of findings.Severity Breakdown
docs-vs-help
1. [High] Broken Markdown table — explanatory sentence injected mid-table
Location:
docs/src/content/docs/setup/cli.md, lines 126–132 (Global Options table)Quoted text:
Expected: A well-formed Markdown table with all three rows (
-h/--help,-v/--verbose,--banner) contiguous, with the explanatory prose placed before or after the table (not inside it).Actual: A full paragraph of prose is inserted between the first and second table rows, which breaks Markdown table parsing — the
-v, --verboseand--bannerrows will likely render as plain text or a second, malformed table rather than being part of the same table.Suggested fix: Move the "For invalid nested command paths..." sentence to a paragraph immediately after the table (or before it), so the three flag rows remain contiguous.
Priority: High (visibly broken rendering in published docs).
2. [Medium]
forecastdocumented Options list is missing--concurrencyLocation:
docs/src/content/docs/setup/cli.md, line 661 (forecastcommand Options); actual flag confirmed in/tmp/gh-aw/agent/help-output/forecast--.txtQuoted text (actual CLI help):
Expected: The forecast section's Options line in the docs should list all real flags, including
--concurrency, matching the pattern used for--days,--eval,--period,--sample,--timeout,--json/-j,--repo/-r.Actual:
--concurrencyis omitted from the documented Options list forforecast.Suggested fix: Add
--concurrency(with its description) to theforecastOptions list in the docs.Priority: Medium (undocumented but real, useful flag; users tuning download parallelism won't discover it from docs).
3. [Medium] Inconsistent
--repoflag description across commands:[HOST/]owner/repo formatvsowner/repo formatLocation: Multiple commands' help output in
/tmp/gh-aw/agent/all-help.txtQuoted text:
Expected: A single, consistent description format for the
--repo/-rflag across all commands (either all should support and document[HOST/]owner/repoGitHub Enterprise Server host prefixes, or the docs should explain why some commands only acceptowner/repowithout a host).Actual:
doctor,env get, andenv updatedocument--repoas plainowner/repo format(no[HOST/]prefix), while most other commands (e.g.enable,disable,update,upgrade,deploy) document it as[HOST/]owner/repo format. It's unclear whether this reflects an actual functional difference (i.e.,doctor/envtruly don't support GHE host prefixes) or is simply inconsistent wording for the same behavior.Suggested fix: Either (a) make the flag help text consistent across all commands if the behavior is identical, or (b) if
doctor/env get/env updategenuinely don't support[HOST/]prefixes, keep the distinction but call it out explicitly (e.g., in a changelog/dev note) so it's clearly intentional rather than a documentation oversight.Priority: Medium (could mislead GHES users about supported repo targeting syntax for specific commands).
flag-consistency
4. [Low]
-lshort flag mapped to two different long flags across commandsLocation:
project newvslogs/other commands using-lQuoted text:
Expected: Ideally, short flags map to the same long-flag concept across the CLI for muscle-memory consistency, though this is a minor/cosmetic concern since
-lis scoped per-command (no functional conflict, as Cobra flags are command-local).Actual:
-lis used for--linkinproject newand for--logical-repoin another command context, which could cause minor confusion for users who habitually use-lacross differentgh awsubcommands.Suggested fix: No functional fix required; consider documenting short-flag meanings are command-scoped, or align short-flag choices where the same short letter is used for unrelated concepts in different commands, if maintaining a mental map across commands is a documentation goal.
Priority: Low (cosmetic, no functional bug).
Inspection Metadata
/tmp/gh-aw/agent/all-help.txt(21,768 lines) and 426 per-command files under/tmp/gh-aw/agent/help-output/.docs/src/content/docs/setup/cli.md(1017 lines), full document reviewed section by section (Global Options through Related Documentation).grep/pattern analysis in lieu of the requested Rig custom harness invocations, which failed withAUTHENTICATION_FAILEDdue to a missingCOPILOT_CONNECTION_TOKENin the workflow shell environment (token is scoped to an internal driver subprocess only). No repository code, workflow, or infrastructure files were modified; all commands used were read-only (grep,view,sed).