Skip to content

feat: add --json flag to help command for agent introspection#432

Merged
BYK merged 1 commit intomainfrom
feat/help-json
Mar 16, 2026
Merged

feat: add --json flag to help command for agent introspection#432
BYK merged 1 commit intomainfrom
feat/help-json

Conversation

@BYK
Copy link
Member

@BYK BYK commented Mar 16, 2026

Phase 1 of #346: Add structured JSON output to the help command so AI agents can discover CLI commands at runtime.

What

Three modes of sentry help --json:

  • Full tree: sentry help --json → all routes, commands, flags, positional params as JSON
  • Route group: sentry help --json issue → group metadata with subcommands
  • Specific command: sentry help --json issue list → single command's full metadata

Framework-injected flags (help, helpAll, log-level, verbose) are stripped from JSON output so agents see only user-facing flags.

Architecture

  • src/lib/introspect.ts (new) — shared route-tree introspection module with types, type guards, extraction functions, and JSON cleaning utilities. Used at runtime by help --json and at build time by generate-skill.ts.
  • src/commands/help.ts — adds output: "json" and --json flag. Dynamic imports for the route tree and introspect module to avoid circular deps.
  • src/lib/help.ts — consolidated to import shared types from introspect.ts instead of duplicating them.
  • script/generate-skill.ts — imports shared types and functions from introspect.ts, removing ~100 lines of duplication. No functional change to output.

Tests

  • 33 unit tests for introspection functions
  • 12 property-based tests (fast-check) for invariants
  • 10 integration tests for help --json end-to-end

@github-actions
Copy link
Contributor

github-actions bot commented Mar 16, 2026

Semver Impact of This PR

🟡 Minor (new features)

📋 Changelog Preview

This is how your changes will appear in the changelog.
Entries from this PR are highlighted with a left border (blockquote style).


New Features ✨

Init

  • Show feedback hint after successful setup by betegon in #430
  • Add --team flag to relay team selection to project creation by MathurAditya724 in #403
  • Enforce canonical feature display order by betegon in #388
  • Accept multiple delimiter formats for --features flag by betegon in #386
  • Add git safety checks before wizard modifies files by betegon in #379
  • Add experimental warning before wizard runs by betegon in #378
  • Add init command for guided Sentry project setup by betegon in #283

Issue List

  • Auto-compact when table exceeds terminal height by BYK in #395
  • Redesign table to match Sentry web UI by BYK in #372

Other

  • (auth) Allow re-authentication without manual logout by BYK in #417
  • (trial) Auto-prompt for Seer trial + sentry trial list/start commands by BYK in #399
  • Add --json flag to help command for agent introspection by BYK in #432
  • Add sentry span list and sentry span view commands by betegon in #393
  • Support SENTRY_HOST as alias for SENTRY_URL by betegon in #409
  • Add --dry-run flag to mutating commands by BYK in #387
  • Return-based output with OutputConfig on buildCommand by BYK in #380
  • Add --fields flag for context-window-friendly JSON output by BYK in #373
  • Magic @ selectors (@latest, @most_frequent) for issue commands by BYK in #371
  • Input hardening against agent hallucinations by BYK in #370
  • Add response caching for read-only API calls by BYK in #330

Bug Fixes 🐛

Dsn

Init

  • Make URLs clickable with OSC 8 terminal hyperlinks by MathurAditya724 in #423
  • Remove implementation detail from help text by betegon in #385
  • Truncate uncommitted file list to first 5 entries by MathurAditya724 in #381

Other

  • (api) Convert --data to query params for GET requests by BYK in #383
  • (docs) Remove double borders and fix column alignment on landing page tables by betegon in #369
  • (trace) Show span IDs in trace view and fix event_id mapping by betegon in #400
  • Show human-friendly names in trial list and surface plan trials by BYK in #412
  • Add trace ID validation to trace view + UUID dash-stripping by BYK in #375

Documentation 📚

  • Update AGENTS.md with patterns from span commands work by BYK in #433
  • Update credential storage docs and remove stale config.json references by betegon in #408

Internal Changes 🔧

Init

  • Remove --force flag by betegon in #377
  • Remove dead determine-pm step label by betegon in #374

Tests

  • Consolidate unit tests subsumed by property tests by BYK in #422
  • Remove redundant and low-value tests by BYK in #418

Other

  • (log/list) Convert non-follow paths to return CommandOutput by BYK in #410
  • Unify commands as generators with HumanRenderer factory, remove stdout plumbing by BYK in #416
  • Convert list command handlers to return data instead of writing stdout by BYK in #404
  • Split api-client.ts into focused domain modules by BYK in #405
  • Migrate non-streaming commands to CommandOutput with markdown rendering by BYK in #398
  • Convert Tier 2-3 commands to return-based output and consola by BYK in #394
  • Convert remaining Tier 1 commands to return-based output by BYK in #382
  • Converge Tier 1 commands to writeOutput helper by BYK in #376

Other

  • Minify JSON on read and pretty-print on write in init local ops by MathurAditya724 in #396

🤖 This preview updates automatically when you update the PR.

@BYK BYK force-pushed the feat/help-json branch from 19d54e6 to ed25c2f Compare March 16, 2026 15:34
@github-actions
Copy link
Contributor

github-actions bot commented Mar 16, 2026

Codecov Results 📊

111 passed | Total: 111 | Pass Rate: 100% | Execution Time: 0ms

📊 Comparison with Base Branch

Metric Change
Total Tests
Passed Tests
Failed Tests
Skipped Tests

✨ No test changes detected

All tests are passing successfully.

✅ Patch coverage is 100.00%. Project has 1100 uncovered lines.
✅ Project coverage is 95.05%. Comparing base (base) to head (head).

Coverage diff
@@            Coverage Diff             @@
##          main       #PR       +/-##
==========================================
+ Coverage    95.02%    95.05%    +0.03%
==========================================
  Files          163       164        +1
  Lines        22103     22244      +141
  Branches         0         0         —
==========================================
+ Hits         21002     21144      +142
- Misses        1101      1100        -1
- Partials         0         0         —

Generated by Codecov Action

@BYK BYK force-pushed the feat/help-json branch from ed25c2f to 3c1867f Compare March 16, 2026 15:51
@BYK
Copy link
Member Author

BYK commented Mar 16, 2026

Addressing review comments

Re: circular import — Moved the introspection logic into src/lib/help.ts as introspectAllCommands() and introspectCommand() functions. Since lib/help.ts already imports routes from app.ts directly (no circular dep — the cycle is only app.ts → commands/help.ts → app.ts), there's no dynamic import needed for the route introspection anymore.

The only remaining dynamic import in commands/help.ts is for the --helpAll delegation to Stricli, which is pre-existing.

Re: using standard yield-based output — The command now uses yield new CommandOutput(...) for all code paths. No more writeJson() calls. The JSON/human branching is fully handled by the framework:

  • --json: yields introspection data → framework serializes to JSON
  • Human mode (no args): yields banner string → framework passes to human renderer
  • Human mode (with args): delegates to Stricli --helpAll (no yield)

@BYK BYK force-pushed the feat/help-json branch from 3c1867f to 1ce73e0 Compare March 16, 2026 16:12
@BYK
Copy link
Member Author

BYK commented Mar 16, 2026

Round 2 review fixes

All three review comments addressed:

  1. No more flags.json check — The command now always yields CommandOutput with introspection data. The human renderer uses a _banner field (stripped from JSON via jsonExclude) for the branded banner display. No manual output branching.

  2. All flags visible — Removed FRAMEWORK_FLAGS, filterFrameworkFlags(), cleanCommandInfoForJson(), and cleanRouteInfoForJson() from introspect.ts. The JSON output now shows all flags including framework-injected ones (help, helpAll, log-level, verbose).

  3. Clean functions removed — Since we no longer filter, the clean wrappers are obsolete and deleted. Tests updated accordingly.

@BYK BYK force-pushed the feat/help-json branch from 1ce73e0 to 6934a57 Compare March 16, 2026 17:21
@BYK
Copy link
Member Author

BYK commented Mar 16, 2026

Round 3 review fixes

Human output no longer emits JSON. Implemented a proper formatHelpHuman renderer in lib/help.ts that produces human-readable output for all introspection types:

  • Full tree (no args): branded banner (via _banner field, stripped from JSON via jsonExclude)
  • Route group (e.g. sentry help issue): formatted command list with aligned descriptions
  • Single command (e.g. sentry help issue list): signature, description, visible flags with aliases and defaults
  • Error: plain error message

Example human output for sentry help issue:

sentry issue

  Manage Sentry issues

  Commands:
    list     List issues in a project
    explain  Analyze an issue's root cause using Seer AI
    plan     Generate a solution plan using Seer AI
    view     View details of a specific issue

No more JSON.stringify in the human renderer.

@BYK BYK marked this pull request as ready for review March 16, 2026 17:30
@BYK BYK force-pushed the feat/help-json branch from 6934a57 to 673ef28 Compare March 16, 2026 17:43
@BYK BYK force-pushed the feat/help-json branch from 673ef28 to 9355ec7 Compare March 16, 2026 17:56
@BYK BYK force-pushed the feat/help-json branch from 9355ec7 to 20a857c Compare March 16, 2026 18:01
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

There are 2 total unresolved issues (including 1 from previous review).

Fix All in Cursor

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

@BYK BYK force-pushed the feat/help-json branch from 20a857c to dba5458 Compare March 16, 2026 18:09
Add structured JSON output to the help command so AI agents can discover
CLI commands at runtime without relying on static docs.

New features:
- `sentry help --json` — emit full command tree as JSON
- `sentry help --json <group>` — emit route group metadata
- `sentry help --json <group> <cmd>` — emit specific command metadata
- Framework-injected flags (help, helpAll, log-level, verbose) are
  stripped from JSON output

Architecture:
- Extract shared route-tree introspection into src/lib/introspect.ts
- Consolidate duplicated type guards from help.ts and generate-skill.ts
- generate-skill.ts now imports from introspect.ts (no functional change)

Tests: 33 unit + 12 property-based + 10 integration tests
@BYK BYK force-pushed the feat/help-json branch from dba5458 to 91e97c2 Compare March 16, 2026 18:14
@BYK BYK merged commit bdbbd83 into main Mar 16, 2026
22 checks passed
@BYK BYK deleted the feat/help-json branch March 16, 2026 19:02
betegon pushed a commit that referenced this pull request Mar 17, 2026
Phase 1 of #346: Add structured JSON output to the `help` command so AI
agents can discover CLI commands at runtime.

## What

Three modes of `sentry help --json`:

- **Full tree**: `sentry help --json` → all routes, commands, flags,
positional params as JSON
- **Route group**: `sentry help --json issue` → group metadata with
subcommands
- **Specific command**: `sentry help --json issue list` → single
command's full metadata

Framework-injected flags (`help`, `helpAll`, `log-level`, `verbose`) are
stripped from JSON output so agents see only user-facing flags.

## Architecture

- **`src/lib/introspect.ts`** (new) — shared route-tree introspection
module with types, type guards, extraction functions, and JSON cleaning
utilities. Used at runtime by `help --json` and at build time by
`generate-skill.ts`.
- **`src/commands/help.ts`** — adds `output: "json"` and `--json` flag.
Dynamic imports for the route tree and introspect module to avoid
circular deps.
- **`src/lib/help.ts`** — consolidated to import shared types from
`introspect.ts` instead of duplicating them.
- **`script/generate-skill.ts`** — imports shared types and functions
from `introspect.ts`, removing ~100 lines of duplication. No functional
change to output.

## Tests

- 33 unit tests for introspection functions
- 12 property-based tests (fast-check) for invariants
- 10 integration tests for `help --json` end-to-end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant