feat: CLI v2 — nested commands, queries, and rich output by sroussey · Pull Request #65 · workglow-dev/sec

sroussey · 2026-03-06T04:58:34Z

Summary

Complete redesign of the CLI for better ergonomics:

Three-command happy path: sec init → sec bootstrap → sec sync covers 90% of use cases
Nested subcommand groups: bootstrap, sync, update, fetch, query, db replace 14 flat commands
Full query system: sec query entities|filings|offerings|crowdfunding|facts|persons with --format table|csv|json
Rich output utilities: table renderer, progress bars, spinners, status messages
Global flags: --json, --verbose, --dry-run, --no-color, --concurrency
Proper error handling: exit codes 0/1/2, errors to stderr, runCommand wrapper
Interactive init wizard: guided first-run setup with .env.local generation
Database inspection: sec db status and sec db stats

Before → After

Before	After
`setup-db`	`sec db setup` (or `sec init`)
`bootstrap-download submissions`	`sec bootstrap download submissions`
`bootstrap-all-cik-names`	`sec bootstrap ingest cik-names`
`bootstrap-submissions`	`sec bootstrap ingest submissions`
`submissions 1018724`	`sec fetch submissions 1018724`
`update-all-submissions`	`sec update submissions`
(no query support)	`sec query entities Tesla`
(no daily pipeline)	`sec sync`

Stats

53 files changed, +6,353 / -444 lines
429 tests passing (53 new)
14 old command files removed, replaced by 6 command groups + 7 query modules

Test plan

All 429 tests pass (bun test)
Build succeeds (bun run build)
CLI help shows all groups and global flags
Integration smoke test verifies command hierarchy end-to-end
Query modules tested with in-memory repos via resetDependencyInjectionsForTesting()

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Replace flat command registration with nested subcommand groups (bootstrap, sync, update, fetch, query, db). Reuse all existing task logic — just reorganize the Commander tree. Old command files are preserved but no longer imported. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Add queryEntities function with support for CIK lookup, SIC/state filtering, partial name search, sorting, and pagination. Wire into the query entities CLI subcommand with table/json/csv output. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Add queryFilings() with support for CIK, form type, date range, and text search filters. Wire into the query filings subcommand. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…facts, persons) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

All 14 old command files replaced by nested group structure in src/cli/groups/. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Verifies the command hierarchy works end-to-end by spawning the CLI as a subprocess and checking help output for all command groups, global options, version, and subcommands (bootstrap, query, fetch, update, db). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Copilot

Pull request overview

Redesigns the sec CLI into a v2 command hierarchy with shared global flags, richer output helpers, and a new query surface over stored SEC data—backed by updated specs/docs and a CLI smoke test.

Changes:

Replaces the legacy flat command set with nested command groups (bootstrap, sync, update, fetch, query, db, init).
Adds shared CLI infrastructure (global options parsing, runCommand wrapper, table/progress utilities) and new query modules + tests.
Updates documentation/specs and adds an integration test for the CLI help hierarchy.

Reviewed changes

Copilot reviewed 52 out of 53 changed files in this pull request and generated 16 comments.

Show a summary per file

File	Description
src/sec.ts	CLI entrypoint updated to v2 + global options hookup
src/commands/index.ts	Switches command registration to new grouped CLI modules
src/commands/UpdateAllSubmissions.ts	Removes legacy flat command
src/commands/UpdateAllForms.ts	Removes legacy flat command
src/commands/UpdateAllCompanyFacts.ts	Removes legacy flat command
src/commands/Submissions.ts	Removes legacy flat command
src/commands/SetupDB.ts	Removes legacy flat command
src/commands/Form.ts	Removes legacy flat command
src/commands/Doc.ts	Removes legacy flat command
src/commands/DailyIndex.ts	Removes legacy flat command
src/commands/CompanyFacts.ts	Removes legacy flat command
src/commands/BootstrapSubmissions.ts	Removes legacy flat command
src/commands/BootstrapDownload.ts	Removes legacy flat command
src/commands/BootstrapCompanyFacts.ts	Removes legacy flat command
src/commands/BootstrapCikLastUpdate.ts	Removes legacy flat command
src/commands/BootstrapAllCikNames.ts	Removes legacy flat command
src/cli/runCommand.ts	Adds centralized command wrapper for exit codes + stderr errors
src/cli/runCommand.test.ts	Tests for `runCommand` behavior
src/cli/queries/PersonQuery.ts	Adds “persons” query module
src/cli/queries/PersonQuery.test.ts	Tests for persons query
src/cli/queries/OfferingQuery.ts	Adds “offerings” query module
src/cli/queries/OfferingQuery.test.ts	Tests for offerings query
src/cli/queries/FilingQuery.ts	Adds “filings” query module
src/cli/queries/FilingQuery.test.ts	Tests for filings query
src/cli/queries/FactsQuery.ts	Adds “facts” query module
src/cli/queries/FactsQuery.test.ts	Tests for facts query
src/cli/queries/EntityQuery.ts	Adds “entities” query module (with sorting support)
src/cli/queries/EntityQuery.test.ts	Tests for entities query
src/cli/queries/DbStatus.ts	Adds DB inspection helpers (status/stats)
src/cli/queries/DbStatus.test.ts	Tests for DB inspection helpers
src/cli/queries/CrowdfundingQuery.ts	Adds “crowdfunding” query module
src/cli/queries/CrowdfundingQuery.test.ts	Tests for crowdfunding query
src/cli/output/index.ts	Barrel exports for output helpers
src/cli/output/TableRenderer.ts	Adds table/csv/json renderer utility
src/cli/output/TableRenderer.test.ts	Tests for table renderer
src/cli/output/Progress.ts	Adds status/progress/spinner helper primitives
src/cli/output/Progress.test.ts	Tests for progress/spinner helpers
src/cli/groups/update.ts	Adds `sec update …` command group
src/cli/groups/sync.ts	Adds `sec sync` pipeline command
src/cli/groups/query.ts	Adds `sec query …` command group wired to query modules
src/cli/groups/init.ts	Adds interactive `sec init` wizard + `.env.local` generation
src/cli/groups/init.test.ts	Tests for `.env.local` generation logic
src/cli/groups/fetch.ts	Adds `sec fetch …` command group
src/cli/groups/db.ts	Adds `sec db …` command group (setup/status/stats/reset)
src/cli/groups/bootstrap.ts	Adds `sec bootstrap …` command group + pipeline
src/cli/cli.integration.test.ts	CLI smoke/integration test verifying command tree + global flags
src/cli/GlobalOptions.ts	Adds global CLI flags + parsing
src/cli/GlobalOptions.test.ts	Tests for global flags
package.json	Dependency adjustments (incl. bunset bump)
bun.lock	Lockfile updates for dependency changes
docs/plans/2026-03-05-cli-v2-plan.md	Implementation plan doc added
docs/plans/2026-03-05-cli-v2-design.md	CLI v2 design doc added
SPEC.md	CLI spec updated to v2 structure

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-06T05:09:46Z

+        } finally {
+          rl.close();


The readline interface is closed twice (rl.close() inside the try block and again in finally). This can lead to inconsistent behavior across Node/Bun versions. Prefer closing exactly once (e.g., remove the inner rl.close() and rely on finally, or guard with a boolean).

Suggested change

} finally {

rl.close();

use the finally version

Copilot · 2026-03-06T05:09:46Z

+      const limit = parseInt(options.limit);
+      const offset = parseInt(options.offset);


limit/offset are parsed with parseInt without validation. If the user passes a non-numeric value, NaN will flow into .slice() and silently return empty results. Use a commander option parser (like the existing parseIntOption) or explicitly validate and throw a user-friendly error when parsing fails.

Suggested change

const limit = parseInt(options.limit);

const offset = parseInt(options.offset);

const limit = Number.parseInt(options.limit, 10);

if (Number.isNaN(limit)) {

throw new Error(`Invalid value for --limit: "${options.limit}" is not a valid integer.`);

}

const offset = Number.parseInt(options.offset, 10);

if (Number.isNaN(offset)) {

throw new Error(`Invalid value for --offset: "${options.offset}" is not a valid integer.`);

}

use parseIntOption

Copilot · 2026-03-06T05:09:47Z

+      console.log(
+        renderTable(result.rows as Record<string, unknown>[], columns, {
+          format: options.format as "table" | "csv" | "json",
+          total: result.total,
+          offset,
+          limit,
+        })
+      );


options.format is an unvalidated string from commander but is cast to "table" | "csv" | "json" and passed through. An invalid value will currently surface as undefined output (via renderTable). Validate --format against allowed values (or set commander .choices([...])) and fail fast with a clear error message.

Copilot · 2026-03-06T05:09:47Z

+| `--from <date>`    | Filing date start (YYYY-MM-DD)    |
+| `--to <date>`      | Filing date end (YYYY-MM-DD)      |


SPEC documents sec query filings date flags as --from/--to, but the implementation uses --after/--before. Update SPEC.md to match the implemented flags (or add aliases in the CLI for compatibility).

Suggested change

| `--from <date>` | Filing date start (YYYY-MM-DD) |

| `--to <date>` | Filing date end (YYYY-MM-DD) |

| `--after <date>` | Filing date start (YYYY-MM-DD) |

| `--before <date>` | Filing date end (YYYY-MM-DD) |

Copilot · 2026-03-06T05:09:47Z

+    const start = offset + 1;
+    const end = offset + count;
+    lines.push("");
+    lines.push(`Showing ${start}-${end} of ${options.total} results`);
+
+    if (end < options.total) {


renderTextTable can produce an invalid pagination range when rows.length === 0 but options.total is provided (e.g., "Showing 1-0 of 10 results"). Adjust the footer logic to handle empty pages (e.g., show "Showing 0-0" or omit the range when count is 0).

Suggested change

const start = offset + 1;

const end = offset + count;

lines.push("");

lines.push(`Showing ${start}-${end} of ${options.total} results`);

if (end < options.total) {

const start = count === 0 ? 0 : offset + 1;

const end = count === 0 ? 0 : offset + count;

lines.push("");

lines.push(`Showing ${start}-${end} of ${options.total} results`);

if (count > 0 && end < options.total) {

Copilot · 2026-03-06T05:09:49Z

+
+Detects existing `.env.local` and offers to reconfigure or skip. PostgreSQL path prompts for connection string or individual parameters. Validates database connectivity before writing config.
+
+When `--json` is passed: outputs config as JSON, no interactivity (for scripting). Non-zero exit if any step fails.


This design doc claims sec init --json outputs config as JSON with no interactivity, but the current init implementation is always interactive and does not handle --json. Either implement the documented behavior or adjust the design doc to reflect the current CLI capabilities.

Suggested change

When `--json` is passed: outputs config as JSON, no interactivity (for scripting). Non-zero exit if any step fails.

`sec init` is always interactive in v2. Non-zero exit if any step fails.

Copilot · 2026-03-06T05:09:49Z

+| `--verbose`     | `-v`  | Enable detailed log output                        |
+| `--dry-run`     |       | Show what would be done without making changes    |
+| `--no-color`    |       | Disable colored output                            |
+| `--concurrency` | `-c`  | Max parallel operations (default varies by command)|


Copilot · 2026-03-06T05:09:49Z

+export function buildEnvConfig(config: InitConfig): string {
+  const lines: string[] = [
+    `SEC_DB_TYPE="${config.dbType}"`,
+    `SEC_DB_FOLDER="${config.dbFolder}"`,
+    `SEC_DB_NAME="${config.dbName}"`,
+    `SEC_RAW_DATA_FOLDER="${config.rawDataFolder}"`,
+  ];
+
+  if (config.dbType === "postgres") {
+    if (config.pgUrl) {
+      lines.push(`SEC_PG_URL="${config.pgUrl}"`);
+    } else {
+      if (config.pgHost) lines.push(`SEC_PG_HOST="${config.pgHost}"`);
+      if (config.pgPort) lines.push(`SEC_PG_PORT="${config.pgPort}"`);
+      if (config.pgUser) lines.push(`SEC_PG_USER="${config.pgUser}"`);
+      if (config.pgPassword) lines.push(`SEC_PG_PASSWORD="${config.pgPassword}"`);
+      if (config.pgDatabase) lines.push(`SEC_PG_DATABASE="${config.pgDatabase}"`);
+    }
+  }
+
+  return lines.join("\n") + "\n";


buildEnvConfig interpolates user-provided values into double-quoted .env lines without escaping. If a value contains " or newlines (e.g., a PostgreSQL password), it can break the file format or inject additional variables. Escape " and \n (or switch to a safe .env encoder) before writing.

Copilot · 2026-03-06T05:09:50Z

+    .command("submissions")
+    .description("Update all submissions for all companies")
+    .option("--concurrency <n>", "Override default concurrency")
+    .action(async () => {
+      await runCommand(async () => {
+        await runTasks(new UpdateAllSubmissionsTask());
+      });
+    });


These subcommands define a --concurrency option but the handler ignores it, and the underlying tasks currently hardcode their concurrency limits. This makes the CLI misleading. Either remove the option or plumb the value through (e.g., extend task input to accept a concurrency limit and use it in wf.map({ concurrencyLimit })).

Copilot · 2026-03-06T05:09:50Z

+#### `sec query entities [cik]`
+
+List or look up entities.
+
+| Argument | Required | Description                       |
+| -------- | -------- | --------------------------------- |
+| `cik`    | No       | Specific CIK to look up          |
+
+| Option             | Description                   |
+| ------------------ | ----------------------------- |
+| `--name <pattern>` | Filter by name (LIKE pattern) |
+| `--sic <code>`     | Filter by SIC code            |
+| `--state <code>`   | Filter by state of incorporation |
+| `--limit <n>`      | Max rows (default: 25)        |
+| `--offset <n>`     | Skip rows (default: 0)        |
+| `--format <fmt>`   | Output format: table, csv, json |


SPEC documents sec query entities [cik] with a positional CIK argument and --name, but the implemented CLI is sec query entities [search] with --cik (and no --name). Please align the spec to the actual command signature/flags (or update the command to match the spec).

Suggested change

#### `sec query entities [cik]`

List or look up entities.

| Argument | Required | Description |

| -------- | -------- | --------------------------------- |

| `cik` | No | Specific CIK to look up |

| Option | Description |

| ------------------ | ----------------------------- |

| `--name <pattern>` | Filter by name (LIKE pattern) |

| `--sic <code>` | Filter by SIC code |

| `--state <code>` | Filter by state of incorporation |

| `--limit <n>` | Max rows (default: 25) |

| `--offset <n>` | Skip rows (default: 0) |

| `--format <fmt>` | Output format: table, csv, json |

#### `sec query entities [search]`

List or look up entities.

| Argument | Required | Description |

| -------- | -------- | --------------------------------------- |

| `search` | No | Free-text search term to filter entities |

| Option | Description |

| --------------- | ------------------------------------ |

| `--cik <cik>` | Filter by exact CIK |

| `--sic <code>` | Filter by SIC code |

| `--state <code>`| Filter by state of incorporation |

| `--limit <n>` | Max rows (default: 25) |

| `--offset <n>` | Skip rows (default: 0) |

| `--format <fmt>`| Output format: table, csv, json |

sroussey · 2026-03-06T05:14:38Z

@copilot open a new pull request to apply changes based on the comments in this thread

Copilot · 2026-03-06T05:14:45Z

@sroussey I've opened a new pull request, #66, to work on those changes. Once the pull request is ready, I'll request review from you.

…, and docs alignment (#66) * Initial plan * fix: apply review feedback — validation, escaping, type correctness, and docs alignment Co-authored-by: sroussey <127349+sroussey@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: sroussey <127349+sroussey@users.noreply.github.com>

sroussey · 2026-03-06T05:31:42Z

@copilot why is the test ending in an error when it has no failures?

Copilot · 2026-03-06T05:31:49Z

@sroussey I've opened a new pull request, #67, to work on those changes. Once the pull request is ready, I'll request review from you.

Updated the test suite for runCommand to ensure process.exitCode is reset to 0 after each test and after all tests have completed, improving test reliability.

…and handling Updated the command registration process to initialize dependency injection only once, triggered by a preAction hook. This change enhances performance and ensures that the DI setup is not repeated unnecessarily for each command execution.

sroussey and others added 14 commits March 5, 2026 05:16

feat(cli-v2): add global options infrastructure

0dfd0ee

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

feat(cli-v2): add table renderer with table/csv/json formats

0cc31c9

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

feat(cli-v2): add progress bar and spinner utilities

eabe363

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

feat(cli-v2): add runCommand error wrapper with exit codes

4a467ee

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

feat(cli-v2): add output barrel export

f22c435

feat(cli-v2): add interactive init wizard

f88c906

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

feat(cli-v2): implement filing query command

aa96b2f

Add queryFilings() with support for CIK, form type, date range, and text search filters. Wire into the query filings subcommand. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

feat(cli-v2): implement all query commands (offerings, crowdfunding, …

b93d9d3

…facts, persons) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

feat(cli-v2): implement db status and db stats commands

405a887

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

refactor(cli-v2): remove old flat command files

cc09eb7

All 14 old command files replaced by nested group structure in src/cli/groups/. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

docs: update SPEC.md to v2 CLI design

e50f354

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

github-actions Bot assigned sroussey Mar 6, 2026

sroussey requested a review from Copilot March 6, 2026 04:59

Copilot started reviewing on behalf of sroussey March 6, 2026 05:00 View session

Copilot AI reviewed Mar 6, 2026

View reviewed changes

Copilot AI mentioned this pull request Mar 6, 2026

fix: CLI v2 review feedback — input validation, escaping, type safety, and docs alignment #66

Merged

Copilot AI mentioned this pull request Mar 6, 2026

fix: prevent bun test exit code 1 when all tests pass #67

Merged

sroussey and others added 3 commits March 6, 2026 06:04

test(cli-v2): enhance runCommand tests with exit code handling

358798d

Updated the test suite for runCommand to ensure process.exitCode is reset to 0 after each test and after all tests have completed, improving test reliability.

fix: prevent bun test exit code 1 when all tests pass (#67)

43e8221

sroussey merged commit c461792 into main Mar 6, 2026
1 check passed

		const limit = parseInt(options.limit);
		const offset = parseInt(options.offset);

-      const limit = parseInt(options.limit);
-      const offset = parseInt(options.offset);
+      const limit = Number.parseInt(options.limit, 10);
+      if (Number.isNaN(limit)) {
+        throw new Error(`Invalid value for --limit: "${options.limit}" is not a valid integer.`);
+      }
+      const offset = Number.parseInt(options.offset, 10);
+      if (Number.isNaN(offset)) {
+        throw new Error(`Invalid value for --offset: "${options.offset}" is not a valid integer.`);
+      }

		\| `--from <date>` \| Filing date start (YYYY-MM-DD) \|
		\| `--to <date>` \| Filing date end (YYYY-MM-DD) \|


		Detects existing `.env.local` and offers to reconfigure or skip. PostgreSQL path prompts for connection string or individual parameters. Validates database connectivity before writing config.

		When `--json` is passed: outputs config as JSON, no interactivity (for scripting). Non-zero exit if any step fails.

	When `--json` is passed: outputs config as JSON, no interactivity (for scripting). Non-zero exit if any step fails.
	`sec init` is always interactive in v2. Non-zero exit if any step fails.

Conversation

sroussey commented Mar 6, 2026

Summary

Before → After

Stats

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

sroussey Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

sroussey Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

sroussey commented Mar 6, 2026

Uh oh!

Copilot AI commented Mar 6, 2026

Uh oh!

sroussey commented Mar 6, 2026

Uh oh!

Copilot AI commented Mar 6, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants