Skip to content

test: expected-output fixtures for JSON inputs (salvaged from #2509)#2961

Merged
schani merged 1 commit into
masterfrom
json-expected-output-fixtures
Jul 20, 2026
Merged

test: expected-output fixtures for JSON inputs (salvaged from #2509)#2961
schani merged 1 commit into
masterfrom
json-expected-output-fixtures

Conversation

@schani

@schani schani commented Jul 20, 2026

Copy link
Copy Markdown
Member

This salvages the surviving half of #2509 by @Adam724: the ability to assert what a fixture's output should be when it legitimately differs from the input.

Problem

The JSON fixture harness asserts round-trip output ≈ input, with unconditional leniency: extra null properties in the output are always allowed, and allowMissingNull languages may drop nulls. That makes correct behaviors where the output should differ from the input untestable. The concrete case from #2509: Go's omitempty should drop null/empty fields, and nothing could assert that it does.

Mechanism

This ports the JSON Schema fixtures' existing .out.<feature>.json convention to JSON inputs, instead of #2509's parallel path-keyed option map:

  • A JSON input foo.json may come with an expected-output file foo.out.<key>.json next to it.
  • The file applies to a run when <key> is one of the language's features, or the name of a renderer option that this particular run sets (via a pinned quickTestRendererOptions entry).
  • When it applies, the run's output must match the expected file strictly — none of the usual null leniency. When absent, behavior is unchanged.
  • .out. files are excluded from test-input enumeration.

Test case

test/inputs/json/priority/omit-empty.json is #2509's test input (a null field that omitempty drops):

{ "results": [ { "age": 52, "name": null }, { "age": 27 } ] }

A pinned quickTestRendererOptions entry runs it for Go with omit-empty, and omit-empty.out.omit-empty.json asserts the null field is actually dropped:

{ "results": [ { "age": 52 }, { "age": 27 } ] }

The plain (no-option) runs of this input still use the ordinary lenient round-trip comparison in every language, including Go: by default the Go renderer only tags non-null/union/any optionals with ,omitempty, so the null survives the round trip there.

The input lives in priority/, not misc/, so it runs in CI — QUICKTEST skips misc/, which is why #2509's version of this test never actually ran in CI.

Relation to #2509

  • Superseded half: Enhance testing capabilities #2509's per-test CLI options mechanism (test/lib/optionMap.ts) has since been covered by quickTestRendererOptions in test/languages.ts, so it is not ported.
  • Salvaged half: the expected-output (*.in.json / *.out.json) idea, redesigned onto the existing .out.<feature>.json convention. The test input and expected output are @Adam724's, credited as co-author.

Testing

  • FIXTURE=golang script/test (full), FIXTURE=python script/test, FIXTURE=typescript script/test, FIXTURE=schema-golang script/test all green.
  • Negative check: with a stale expected file that keeps the null, the pinned Go run fails and the plain runs pass, confirming the strict comparison engages exactly where the expected file applies.

🤖 Generated with Claude Code

The JSON fixture harness asserts round-trip output ~ input, with
unconditional leniency: extra null properties in the output are always
allowed, and allowMissingNull languages may drop nulls.  Correct
behaviors where the output legitimately differs from the input were
therefore untestable -- concretely, that Go's omitempty drops null
fields.

Port the JSON Schema fixtures' `.out.<feature>.json` convention to JSON
inputs: an input `foo.json` may come with an expected-output file
`foo.out.<key>.json`, which applies to a run when `<key>` is one of the
language's `features` or the name of a renderer option that the
particular run sets (via a pinned `quickTestRendererOptions` entry).
When it applies, the output must match strictly, without the null
leniency; when absent, behavior is unchanged.  `.out.` files are
excluded from test-input enumeration.

Use it to assert the omit-empty behavior from #2509: a pinned Go run of
priority/omit-empty.json with omit-empty enabled must drop the null
field, per omit-empty.out.omit-empty.json.  The input is in priority/
(not misc/) so it also runs under QUICKTEST in CI.

Salvaged from #2509; the test case is Adam724's.

Co-authored-by: Adam724 <alaspina6@gmail.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@schani
schani merged commit 3de9cc2 into master Jul 20, 2026
28 checks passed
@schani
schani deleted the json-expected-output-fixtures branch July 20, 2026 16:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant