Skip to content

fix: reject stray commas in object, array, call and parameter lists - #1113

Open
He-Pin wants to merge 2 commits into
databricks:masterfrom
He-Pin:fix-empty-object-comma-parse
Open

fix: reject stray commas in object, array, call and parameter lists#1113
He-Pin wants to merge 2 commits into
databricks:masterfrom
He-Pin:fix-empty-object-comma-parse

Conversation

@He-Pin

@He-Pin He-Pin commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Motivation

{,} (and { , }) parsed as an empty object, while go-jsonnet and jrsonnet both report a parse error:

Program sjsonnet (before) sjsonnet (after) go-jsonnet jrsonnet
{,} { } parse error Unexpected: "," while parsing field definition expected field name, got ','
{ , } { } parse error parse error parse error
local f(a) = a; f(,) runtime arity error parse error parse error parse error
(function(,) 1)() 1 parse error parse error parse error
[1,,] [1] parse error parse error parse error

Root cause: the comma-list grammars combined rep(sep = ",") (accepts zero elements) with an optional trailing comma, so a lone comma was consumed as the "trailing" comma of an empty list. The Jsonnet grammar requires at least one element before a comma. The array grammar had a related hole: after the first comma, expr.rep(0, sep = ",") ~ ",".? accepted zero elements followed by a comma, so [1,,] parsed as [1].

Modification

sjsonnet/src/sjsonnet/Parser.scala — capture the trailing comma (,.!) in the four affected grammars (objinside, args, params, arrBody`) and fail with a descriptive parse error when a comma appears with zero preceding elements:

  • Expected at least one object member before ','
  • Expected at least one argument before ','
  • Expected at least one parameter before ','
  • Expected at least one array element before ','

Implementation note: the obvious rep(1, sep = ",") ~ ",".? wrapped in an outer .? was tried first and rejected — it triggers a Scala 2.13 lambda-lift compiler bug (Could not find proxy for case val x1: fastparse.ParsingRun) with fastparse's implicit plumbing. The capture-and-check shape keeps the original flat parser structure and compiles on all Scala versions. For arrBody, cut = true is set before the failure so the error propagates through the enclosing .? instead of being swallowed.

Result

  • {,}, { , }, f(,), function(,), [1,,] are parse errors.
  • Valid trailing commas unchanged (pinned by a new positive test): {a: 1,}, {a: 1, b: 2,}, [1, 2,], f(1, 2,), function(a,) e, function(x=1,) e, [1, for x in xs], plus object comprehensions, object locals and asserts with commas.
  • Other comma misuses were already rejected and still are ({,a: 1}, {a: 1,, b: 2}, [,], [1,,2], {,,}).

Test plan

  • New: new_test_suite/error.parse_object_lone_comma.jsonnet, error.parse_call_lone_comma.jsonnet, error.parse_params_lone_comma.jsonnet, error.parse_array_double_comma.jsonnet (+ goldens)
  • New: new_test_suite/parse_trailing_commas_valid.jsonnet (+ golden) pinning the preserved valid forms
  • ./mill 'sjsonnet.jvm[_].test' — all Scala versions (2.12.21 / 2.13.18 / 3.3.8) pass
  • ./mill 'sjsonnet.js[_].compile' 'sjsonnet.native[_].compile' 'sjsonnet.wasm[_].compile' — pass
  • ./mill __.checkFormat — clean

Found by four-way differential testing (sjsonnet vs go-jsonnet vs jrsonnet vs spec).

He-Pin added 2 commits August 6, 2026 14:49
…r lists

Motivation:
`{,}` (and `{ , }`) parsed as an empty object while go-jsonnet and
jrsonnet both report a parse error: the comma-list grammars combined
`rep(sep = ",")` (which accepts zero elements) with an optional trailing
comma, so a single comma was consumed as the "trailing" comma of an
empty list. The same hole accepted `f(,)` and `function(,)` (rejected
only later, or at runtime). The Jsonnet grammar requires at least one
element before a comma.

Modification:
Capture the trailing comma in the three affected grammars (objinside,
args, params) and fail with a descriptive parse error when a comma
appears with zero preceding elements. A straight `rep(1, ...) ~ ... .?`
restructuring was rejected because it triggers a Scala 2.13 lambda-lift
compiler bug ("Could not find proxy for case val x1: ParsingRun") with
fastparse's implicit plumbing; the capture-and-check shape keeps the
original flat structure.

Result:
`{,}`, `{ , }`, `f(,)`, `function(,)` are parse errors; valid trailing
commas are unchanged (`{a: 1,}`, `[1, 2,]`, `f(1, 2,)`,
`function(a,)`, object comprehensions, object locals/asserts).

References:
Found by four-way differential testing (sjsonnet vs go-jsonnet vs
jrsonnet vs spec).
Motivation:
The same trailing-comma capture hole existed in arrBody: after the
first comma, `expr.rep(0, sep = ",") ~ ",".?` accepted zero elements
followed by a comma, so `[1,,]` (and `[1, ,]`) parsed as `[1]` while
go-jsonnet, C++ jsonnet and jrsonnet all report a parse error.

Modification:
Capture the trailing comma in the inner rest-of-array grammar and fail
with "at least one array element before ','" when the repetition is
empty, mirroring the objinside/args/params fixes. Set cut=true so the
error propagates through the enclosing `.?`. Valid forms are unaffected:
`[1,]`, `[1, 2,]`, and `[e, for x in xs]`.

Result:
`[1,,]` is a parse error, matching all three reference implementations.
Found by the same four-way differential testing.

References:
Extends PR databricks#1113 (lone commas in objects, calls, params).
@He-Pin He-Pin changed the title fix: reject lone commas in object bodies, call arguments and parameter lists fix: reject stray commas in object, array, call and parameter lists Aug 7, 2026
@He-Pin
He-Pin marked this pull request as ready for review August 7, 2026 06:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant