Skip to content

fix(cli): match Go machine-format encoder output for -o toml/yaml/json (CLI-1975) - #6002

Merged
Coly010 merged 11 commits into
developfrom
columferry/cli-1975--o-toml-o-yaml-emit-wrong-key-casing-on-struct-payloads-o
Aug 3, 2026
Merged

fix(cli): match Go machine-format encoder output for -o toml/yaml/json (CLI-1975)#6002
Coly010 merged 11 commits into
developfrom
columferry/cli-1975--o-toml-o-yaml-emit-wrong-key-casing-on-struct-payloads-o

Conversation

@Coly010

@Coly010 Coly010 commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

⚖ Parity ruling applied

This issue was decision-gated. Colum ruled (2026-07-30) to take the issue's recommended option: remap the -o toml / -o yaml encoders to Go field-name casing and structure — full Go parity. The alternative (declaring the TS snake_case casing intentional) was rejected. The Go CLI (apps/cli-go/) is the byte-parity oracle: for every affected command the machine output now matches what the Go binary prints, including field-name casing, ordering, zero-value inclusion, datetime rendering, and null rendering.

User-visible change: scripts consuming -o toml / -o yaml from the affected commands will now see Go-shaped keys (ProjectRef / [[branches]]-style PascalCase in TOML, projectref-style lowercased names in YAML) instead of the snake_case keys the TS CLI emitted until now — i.e. scripts written against the long-lived Go CLI work again. -o json values additionally carry Go's default HTML escaping (< > & as \u003c-style escapes); any JSON parser decodes these transparently.

Fixes CLI-1975

What changed

Go's utils.EncodeOutput hands raw Go structs to BurntSushi TOML and yaml.v3, and neither library reads json: tags — keys come from the Go struct field names. The TS legacy shell was instead re-encoding the decoded snake_case JSON. This PR closes that gap for every struct-payload command.

New shared encoder (legacy/shared/legacy-go-struct-output.encoders.ts)

A pure, spec-driven pair of encoders that reproduce the Go binary byte-for-byte, driven by per-payload-family Go struct specs (field order = Go declaration order, mirrored from apps/cli-go/pkg/api/types.gen.go and command-local structs):

  • TOML (BurntSushi v1.6.0 semantics): PascalCase field names, primitives before sub-tables, 2-space nested table indentation, blank line before top-level tables and every [[array-of-tables]] element, nil pointers/slices/maps omitted, empty decoded arrays as key = [], time.Time as native RFC3339Nano datetimes, Go float 'g' formatting with the TOML .0 rule, BurntSushi's exact string-escape table, and Go's runtime failure for populated nullable.Nullable fields.
  • YAML (yaml.v3 v3.0.1 semantics): lowercased-whole-field-name keys, explicit null for nil pointers, []/{} for nil slices/maps, yaml.v3's 4-column indentation algorithm (+2 inside sequence items), its scalar-quoting resolver (old-bools, base-0 ints, floats, base-60, timestamps → double-quoted; emitter-plain-disallowed → single-quoted), block literals with chomping/indentation indicators, map[bool]T rendering for oapi nullable fields, yaml.v3's natural map-key sort, and Go strconv.FormatFloat(_, 'g', -1, bits) float formatting incl. float32 rounding.

Every golden byte string in the unit tests was captured empirically from a scratch Go program running the repo's own utils.EncodeOutput with the exact library versions pinned in apps/cli-go/go.mod.

Commands rewired to the spec encoders

  • branches list/create/update (shared branches.go-payload.ts)
  • orgs list/create (orgs.go-payload.ts)
  • projects list (Go's linkedProject embed: inlined V1ProjectWithDatabaseResponse fields first, Linked last), projects create (shared create core), projects api-keys (-o yaml only — Go's -o toml|env encode the SUPABASE_*_KEY env map, unchanged)
  • secrets list, sso list/show/add/update/remove (sso.go-payload.ts), domains get/create/activate/reverify, ssl-enforcement get/update (ssl-enforcement.go-payload.ts), functions list (replaces its bespoke per-command key maps), backups list, snippets list, services, vanity-subdomains get/activate/check-availability (YAML was snake_case; TOML gains Go's declaration order)

Nil-vs-empty slice semantics follow each Go command: append-built lists (branches list, projects list) emit nothing for -o toml when empty (Go nil slice), decoded lists emit key = [].

Bug-for-bug parity notes

  • snippets list -o toml now fails with Go's exact error (failed to output toml: toml: cannot encode a map with non-string key type) whenever a snippet carries a description — BurntSushi cannot encode nullable.Nullable[string] (map[bool]string), and the Management API always sends the key. Go fails identically.
  • projects api-keys -o yaml renders nullable fields as yaml.v3 renders map[bool]T: apikey: + indented true: <value> when set, {} when absent, false: "" for explicit JSON null.

JSON HTML escaping

encodeGoJson (-o json) and encodeGoStructJsonBody (raw-HTTP request bodies for sso add/update) now produce Go's encoding/json default escaping: < > &\u003c \u003e \u0026, \u0008/\u000c for backspace/form feed, and escaped U+2028/U+2029 — materially visible in sso … metadata_xml. Both now route through the shared Go-faithful JSON walker (legacy-go-json.ts, which gains a compact mode); functions list's bespoke post-escaper was deleted. The cli-e2e replay server compares parsed bodies, so recorded fixtures are unaffected.

Tests & docs

  • New unit suite for the encoders with Go-captured golden bytes (quoting matrix, block literals, floats incl. -0, nullable shapes, nil/empty slices, natural key sort, hostnames nesting).
  • Integration tests upgraded to byte-exact assertions for branches list (toml+yaml, incl. a zero-value branch and the empty-list -o toml no-output case), sso show (json escape + yaml + toml), backups list (incl. [[Backups]]), snippets (both the Go failure and the description-absent success bytes), plus casing fixes across orgs/projects/secrets/services/ssl-enforcement/domains/vanity tests. The previously-wrong branches list toml assertion (name = "feat-1") is now the full Go-golden document.
  • An explicit exempt-proof test: branches get -o toml (map payload) keeps its env-map keys verbatim — the struct remap must not apply to map payloads (sso info, status, postgres-config, network-bans, branches get are unchanged).
  • SIDE_EFFECTS.md parity claims updated (domains' "intentional snake_case divergence" note deleted; backups/secrets/sso output-shape descriptions now byte-accurate).

Deliberately out of scope (pre-existing divergences, noted for the record)

  • network-restrictions get/update: Go never encodes -o output for these commands (it always prints three fixed Printf lines), so there is no Go byte oracle; the TS handlers' existing -o json|yaml|toml|env support is left untouched. Follow-up candidate.
  • branches list -o json with zero branches: TS emits [], Go emits null (append-built nil slice). JSON values were out of CLI-1975's scope (HTML escaping only).
  • projects list -o json: TS sorts keys alphabetically so linked sorts mid-object; Go emits it last. Same out-of-scope reasoning.
  • domains data.errors/data.messages are modeled as raw JSON values; Go's generated element type marshals as an empty struct — unobservable because these arrays are empty on every reachable path (both sides emit []).

@Coly010

Coly010 commented Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9b583fd807

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread apps/cli/src/legacy/commands/domains/domains.emit.ts
Comment thread apps/cli/src/legacy/shared/legacy-go-struct-output.encoders.ts
Comment thread apps/cli/src/legacy/shared/legacy-go-struct-output.encoders.ts Outdated
… (review: codex)

- yamlIsTimestamp now mirrors time.Parse's calendar validation (days-in-month
  with proleptic-Gregorian leap years) and zone-offset range checks (hour > 24,
  minute > 60 rejected), so strings like "2025-02-31" stay plain exactly as
  yaml.v3 leaves them.
- TOML map keys sort by UTF-8 byte order (sort.Strings) and yaml.v3's
  keyList.Less rune comparisons use code points, not JS UTF-16 units, fixing
  astral-vs-high-BMP key ordering in both encoders.
- yamlHasSpecialChars/yamlDoubleQuoted now port libyaml's is_printable: astral
  characters, U+FEFF, and U+FFFE/U+FFFF double-quote with \x/\u/\U escapes.

All new golden bytes captured from Go's own utils.EncodeOutput with the pinned
BurntSushi toml v1.6.0 / yaml.v3 v3.0.1.
@Coly010

Coly010 commented Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e020a45acf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread apps/cli/src/legacy/shared/legacy-go-struct-output.encoders.ts
Comment thread apps/cli/src/legacy/shared/legacy-go-struct-output.encoders.ts
Comment thread apps/cli/src/legacy/shared/legacy-go-struct-output.encoders.ts
Comment thread apps/cli/src/legacy/shared/legacy-go-struct-output.encoders.ts Outdated
Comment thread apps/cli/src/legacy/shared/legacy-go-struct-output.encoders.ts Outdated
@Coly010

Coly010 commented Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a479e84f36

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread apps/cli/src/legacy/shared/legacy-go-struct-output.encoders.ts Outdated
Comment thread apps/cli/src/legacy/commands/backups/list/list.handler.ts
Comment thread apps/cli/src/legacy/commands/sso/add/add.handler.ts Outdated
@Coly010

Coly010 commented Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c701b9ae64

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread apps/cli/src/legacy/shared/legacy-go-struct-output.encoders.ts
Comment thread apps/cli/src/legacy/shared/legacy-go-struct-output.encoders.ts
Comment thread apps/cli/src/legacy/shared/legacy-go-struct-output.encoders.ts
Comment thread apps/cli/src/legacy/shared/legacy-go-struct-output.encoders.ts
Comment thread apps/cli/src/legacy/shared/legacy-go-struct-output.encoders.ts Outdated
Comment thread apps/cli/src/legacy/shared/legacy-go-struct-output.encoders.ts Outdated
Coly010 added 2 commits July 31, 2026 02:20
…5--o-toml-o-yaml-emit-wrong-key-casing-on-struct-payloads-o
@Coly010
Coly010 marked this pull request as ready for review July 31, 2026 09:55
@Coly010
Coly010 requested a review from a team as a code owner July 31, 2026 09:55

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b7522fec63

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread apps/cli/src/legacy/shared/legacy-go-struct-output.encoders.ts Outdated
@github-actions

github-actions Bot commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Supabase CLI preview

npx --yes https://pkg.pr.new/supabase/cli/supabase@1d72d37c01b7a32690c7fe71781cf35ded9d8294

Preview package for commit 1d72d37.

@Coly010

Coly010 commented Jul 31, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

Round-6 finding addressed in dddbf6b: yamlKeyLess digit-run accumulation now wraps with BigInt.asIntN(64, …) to reproduce yaml.v3's unchecked Go int64 arithmetic (Go-probe verified), with a golden unit test pinning the probed order.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dddbf6b880

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread apps/cli/src/legacy/shared/legacy-go-struct-output.encoders.ts
@Coly010

Coly010 commented Jul 31, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

Round-7 finding adjudicated: the partial-flush premise was probe-confirmed (exactly 4096 bytes, not the accumulated prefix), the suggested fix therefore refuted, and the bound documented on encodeLegacyGoToml in c7968ee (docs-only).

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Another round soon, please!

Reviewed commit: c7968ee982

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@Coly010 Coly010 self-assigned this Jul 31, 2026

@kanadgupta kanadgupta left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of future-proofing suggestions from Claude but LGTM!

Comment thread apps/cli/src/legacy/shared/legacy-go-struct-output.encoders.ts
Comment thread apps/cli/src/legacy/commands/branches/branches.go-payload.ts
…gic (CLI-1975)

Hoist the map-sort + direct/sub table partitioning shared by tomlTable
and tomlArrayOfTables into tomlOrderedEntries; tomlInlineTable keeps its
own copy since its position-based separator logic doesn't unify cleanly
(review kanadgupta). Also notes in the Go Parity Checklist that a
types.gen.go regen requires re-auditing every go-payload.ts spec.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 95d24f6b75

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread apps/cli/src/legacy/shared/legacy-go-struct-output.encoders.ts
Comment thread apps/cli/src/legacy/shared/legacy-go-struct-output.encoders.ts
Comment thread apps/cli/src/legacy/shared/legacy-go-struct-output.encoders.ts
@Coly010
Coly010 enabled auto-merge August 3, 2026 10:06
Parses the real Go struct declarations out of types.gen.go and structurally
compares them against the LegacyGoType specs the *.go-payload.ts files
hand-declare, so a future field add/remove/reorder/rename in the Go API
types desyncs a spec loudly instead of silently corrupting -o yaml/-o toml
output (review kanadgupta, PR #6002).
…5--o-toml-o-yaml-emit-wrong-key-casing-on-struct-payloads-o

# Conflicts:
#	apps/cli/src/legacy/commands/sso/add/add.handler.ts
#	apps/cli/src/legacy/commands/sso/update/update.handler.ts

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1d72d37c01

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread apps/cli/src/legacy/shared/legacy-go-struct-output.encoders.ts
@Coly010
Coly010 added this pull request to the merge queue Aug 3, 2026
Merged via the queue into develop with commit 56296d9 Aug 3, 2026
28 checks passed
@Coly010
Coly010 deleted the columferry/cli-1975--o-toml-o-yaml-emit-wrong-key-casing-on-struct-payloads-o branch August 3, 2026 12:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants