Skip to content

test(cli): shared CLI contract schema-golden test (v0.19 Workstream 8)#2668

Merged
devarismeroxa merged 2 commits into
mainfrom
test/cli-contract-schema-golden
Jul 23, 2026
Merged

test(cli): shared CLI contract schema-golden test (v0.19 Workstream 8)#2668
devarismeroxa merged 2 commits into
mainfrom
test/cli-contract-schema-golden

Conversation

@devarismeroxa

Copy link
Copy Markdown
Contributor

Summary

Implements the schema-golden test for v0.19 Workstream 8
(v019-plans/workstreams/cli-contract.md) — the harmonization guarantee for
every other v0.19 workstream's --json surface.

Corrected premise, verified in the repo (not "one schema," per the plan's
own §4 finding):
there are two distinct, both-deliberate --json families
in production today:

  • Family A — envelope (cecdysis.CommandWithResult): 15 commands
    (13 pre-existing + init + pipelines init, both migrated onto the
    envelope in this PR — previously zero --json support on either).
    Validated against one committed schema,
    cmd/conduit/cecdysis/testdata/envelope.schema.json.
  • Family B — client-result passthrough
    (cecdysis.CommandWithExecuteWithClientResult): 13 commands, raw
    protojson/go-json, intentionally never wrapped in the envelope (confirmed
    correct, load-bearing behavior, not a migration backlog item). Asserted to
    never accidentally match the envelope schema.
  • 1 named exception: quickstart --json means "log format," not the
    envelope — documented, not silently ignored, not silently fixed.

What's implemented

  1. Completeness walk (cmd/conduit/cli/schema_golden_test.go): recursively
    walks (&root.RootCommand{}).SubCommands() and classifies every leaf.
    Fails the build if a command registers --json but implements neither
    family and isn't a named exception. Verified this predicate actually
    fires (temporarily un-exempted quickstart, confirmed the test failed
    with the expected message, restored it).
  2. Shared schema-validation harness
    (cmd/conduit/internal/testutils/envelope.go): ValidateEnvelope/
    MatchesEnvelope, test-only (never linked into the conduit binary —
    testutils has zero non-test importers, verified). Promotes
    github.com/santhosh-tekuri/jsonschema/v6 from an already-present
    indirect
    dependency to direct — no new dependency, just wiring one that
    was already in go.sum.
  3. conduit init / conduit pipelines init migrated onto Family A
    (previously zero --json support on either), human-mode text preserved
    byte-for-byte.
  4. Every one of the 26 pre-existing Family A/B commands gets an
    envelope-conformance (or negative) assertion added to its own existing
    test file — no production behavior changed, per AC-3. New lint/dry-run
    fixtures added where no test file existed before.
  5. Doc addendum: docs/design-documents/20260707-cli-output-conventions.md
    §6 documents the two-family split and the quickstart exception.

Risk tier

Tier 2 (CLI/test infrastructure, per the plan's own assessment — no
data-path invariant touched).

Adversarial self-review

  • Verified the completeness walk's failure predicate actually fires (not
    just that it passes) by temporarily removing the quickstart exception
    entry and confirming the expected failure message, then restoring it.
  • Verified make generate produces zero drift beyond this diff's own
    files (git status --short after make generate shows only the files
    this PR intentionally touches).
  • Verified testutils (home of the new schema validator) has zero
    non-test importers, so jsonschema/v6 does not enter the shipped
    conduit binary despite being promoted to a direct module dependency.
  • connectors bundle and connector|processor new have no genuine
    end-to-end success-path test today (bundling/scaffolding need real
    artifacts/network+toolchain) — for these I added a schema-conformance
    check against the real result type with realistic synthetic data
    (mirroring the pre-existing TestResultMarshalsToTheDocumentedEnvelope
    precedent in scaffoldcmd), documented as such rather than silently
    passed off as a full E2E test.
  • Checked error paths in the two migrated init commands: a HARD failure
    (unwritable directory, unknown connector) still returns a non-nil error
    from ExecuteWithResult, correctly routed through the existing
    conduiterrResultError mapping — no new error-shape code needed.
  • No serialized-state, protocol, or config-schema changes; this is CLI
    output/test surface only.

Test plan

  • go build ./...
  • golangci-lint run ./cmd/conduit/... — 0 issues
  • go test -race ./cmd/conduit/... — all green
  • make generate && git status --short — zero unexpected drift
  • markdownlint-cli2 (both local and CI-pinned 0.18.1) on the doc
    addendum — 0 errors
  • Manually confirmed the completeness walk's failure predicate fires

Roadmap: v0.19 Workstream 8.

🤖 Generated with Claude Code

https://claude.ai/code/session_015GQFzakPShAYj8CcwajYDD

Adds the schema-golden test that proves the "--json contract" claim instead
of asserting it: a completeness walk recursively classifies every leaf in
(&root.RootCommand{}).SubCommands() into Family A (cecdysis.CommandWithResult,
validated against the one committed envelope.schema.json), Family B
(cecdysis.CommandWithExecuteWithClientResult, raw protojson/go-json
passthrough, asserted to NOT match the envelope), or a named exception
(quickstart --json, a pre-existing log-format toggle, documented not fixed).
An unclassified --json-registering command fails the build.

- cmd/conduit/cecdysis/testdata/envelope.schema.json: the one committed
  Family A schema (draft 2020-12, additionalProperties:false).
- cmd/conduit/internal/testutils/envelope.go: ValidateEnvelope/MatchesEnvelope,
  shared by every command's own test file (test-only; never linked into the
  conduit binary). Promotes github.com/santhosh-tekuri/jsonschema/v6 from an
  already-present indirect dependency to direct.
- cmd/conduit/cecdysis/decorators.go: exports MarshalJSON so Family B tests
  can assert on the exact --json bytes without re-deriving the marshaling
  choice.
- cmd/conduit/cli/schema_golden_test.go: the completeness walk.
- Migrates `conduit init` and `conduit pipelines init` onto
  cecdysis.CommandWithResult (previously zero --json support), with
  human-mode output text preserved unchanged.
- Extends the 26 existing Family A/B commands' own test files with an
  envelope-conformance (or negative) assertion each, plus new lint/dry-run
  fixtures where none existed.
- Documents the two-family split and the quickstart exception in
  docs/design-documents/20260707-cli-output-conventions.md §6.

Roadmap: v0.19 Workstream 8 (v019-plans/workstreams/cli-contract.md).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015GQFzakPShAYj8CcwajYDD
@devarismeroxa
devarismeroxa requested a review from a team as a code owner July 23, 2026 17:44
…ema-golden

# Conflicts:
#	cmd/conduit/root/pipelines/init.go
@devarismeroxa
devarismeroxa merged commit 6da2c72 into main Jul 23, 2026
6 checks passed
@devarismeroxa
devarismeroxa deleted the test/cli-contract-schema-golden branch July 23, 2026 18:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant