feat(grammar): ordered-properties mode for JsonSchemaOutputConstraint (#425) - #431
Merged
Conversation
…#425) Opt-in ordered mode: declared properties must appear in declaration order -- optional properties are skippable, but a later property can never precede an earlier one, and a required property is never skippable (the next-key candidate window runs from just past the last emitted key up to and including the first required key, so the mask can never steer generation into an unrepairable skip). Lets a streaming consumer act on an early field (e.g. a short spoken "say" line sent to TTS) before a later, larger field ("show") finishes streaming -- previously any declared key was legal in any position, so the model could emit the large field first and defeat the latency win. - ToolSchemaCompiler: CompiledObject.Ordered, threaded recursively so nested objects are ordered too; ordered-mode key candidates collapse to a small window instead of an every-unemitted-key bitmask. - JsonToolArgumentConstraint: NextKeyCandidates() drives both StepObject and CollectObject. Also gates ',' on a candidate existing (both modes): once every reachable key is emitted, ',' previously committed to an impossible key and dead-ended the constraint into giving up; now '}' is forced instead. - JsonSchemaOutputConstraint(vocab, schema, orderedProperties: true) + exposed OrderedProperties for wiring tests. - CLI: --json-schema-ordered alongside -j/--json-schema/--jf. - Server: json_schema.ordered / response_format.ordered (flat json_object extension) on /v1/chat/completions. Closes #425. The issue's lower-priority related asks (non-empty required strings, toolcall|schema alternation) are intentionally not included and can be split into their own issues. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018EWo1niax1g8A5E79RSALJ
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces an ordered-properties mode (addressing issue #425) to enforce JSON schema property emission in declaration order. This allows streaming consumers to process early fields before later, larger ones finish. The changes span the CLI, Core grammar constraint logic, Server endpoints, and associated unit tests. No review comments were provided, so I have no feedback to provide.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
…eview round)
Findings from the multi-angle review of the ordered-properties PR:
- fix(grammar): GemmaToolArgumentConstraint had the same all-keys-emitted
trailing-comma hole the PR fixed in the JSON walker -- and worse: the
mask kept whitespace legal in OExpectKey, so the dead-state escape
never fired and generation livelocked burning whitespace tokens to the
limit (fail-closed), never closing the call. Same gate applied to its
StepObject/CollectObject; ToolGrammarMockTests pin flipped.
(The JSON walker's old failure mode was the same livelock, not a
"constraint gives up" dead state -- test comments corrected.)
- refactor(grammar): NextKeyCandidates/HasNextKey hoisted onto
CompiledObject (the shared compiled form both walkers consume), with
an O(1) HasNextKey for the three call sites that only need emptiness
(the per-simulated-byte comma gate among them). TryCompileObject dual
overloads merged into one defaulted signature. CompiledObject.Ordered
now documents it is enforced by the JSON walker only.
- fix(server): ordered opt-in is now position-agnostic -- flat
response_format.ordered and nested json_schema.ordered are both
honored for BOTH response_format shapes, instead of silently no-oping
when placed on the "wrong" level for the given type.
- fix(cli): --json-schema-ordered without --json-schema/--json-schema-file
is now a hard error instead of a silent no-op, matching the flag
family's fail-loudly contract.
- docs(grammar): ordered-mode docs no longer overstate coverage --
FreeValue-degraded subtrees (open/untyped/too-deep) stay unordered.
- tests: merged-token (",\") assertion at the comma gate; server ordered
wiring tests deduplicated into a helper + flat-on-json_schema case;
CLI ordered-without-schema error case.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018EWo1niax1g8A5E79RSALJ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements #425: an opt-in ordered mode for
JsonSchemaOutputConstraintwhere declared properties must appear in declaration order -- optional properties are skippable, but never reordered, and a required property is never skippable. This lets a streaming consumer act on an early field (the issue''s TTSsayline) before a later, larger field (show) finishes streaming.How it works
ToolSchemaCompiler.TryCompileObject(obj, ordered)stampsCompiledObject.Orderedrecursively (typed nested objects are ordered too; FreeValue-degraded subtrees stay unordered, as documented).CompiledObject.NextKeyCandidates(emitted)computes next-key candidates on the shared compiled form: unordered keeps the existing every-unemitted-key bitmask; ordered narrows to the window from just past the highest emitted key up to and including the first required key -- so the mask itself makes skipping a required key impossible.HasNextKey(emitted)is the O(1) emptiness check for hot call sites.Comma gate (both modes, both walkers)
The review round upgraded this from "latent dead state" to a confirmed fail-closed livelock: at
OExpectCommaOrClosea,was always legal even when no key could follow (all declared keys emitted). After sampling it, the machine sat inOExpectKeywhere the mask kept only whitespace legal (}is not accepted there, EOG is forbidden mid-object), so the dead-state escape never fired and generation burned whitespace tokens until the token limit. Now,is masked whenever the candidate window is empty, forcing}.The multi-angle review found the same livelock in
GemmaToolArgumentConstraint(the sibling walker over the sameCompiledObject); the identical gate is applied there too, with its test pin flipped.QwenCoderToolArgumentConstraintalready had the gate viaUnemittedMask.Opt-in surfaces
new JsonSchemaOutputConstraint(vocab, schema, orderedProperties: true)--json-schema-ordered(with-j/--json-schema/--jf; a hard error if no schema flag is given)ordered: true-- honored both nested onjson_schemaand flat onresponse_format, for bothjson_schemaandjson_objectshapes (position-agnostic)Default behavior is unchanged (any-order), except the comma-gate livelock fix above.
Review round (multi-agent, xhigh)
10 finder angles + adversarial verification over the diff. The automaton core survived all five correctness angles (mask/replay consistency incl. merged BPE tokens, ordered-window induction proving no reachable dead state, prefix-key resolution, LZCNT edge cases, free-value nesting, reset re-engagement). Findings applied:
orderedwas position-dependent (flat flag silently ignored fortype:"json_schema")--json-schema-orderedwithout a schema flag silently no-opedNextKeyCandidates/HasNextKeyhoisted ontoCompiledObject;TryCompileObjectoverloads merged; O(1) emptiness on the per-simulated-byte comma gateresponse_formatsection drift (pre-existing, predates feat(core): caller-supplied output-level ITokenConstraint (whole-turn grammar, not just tool args) #423)Tests
Tests.Core 273 / Tests.Cli 60 / Tests.Server 154 all pass; solution builds clean under warnings-as-errors. New coverage: declaration-order masking (first key, required-not-skippable, optional-skip, nested objects, EOG completion), merged-token comma gate, Gemma comma gate, CLI ordered-without-schema error, server ordered wiring for all three flag placements.
Out of scope
The issue''s lower-priority related asks --
minLength/non-empty required strings, and a top-leveltoolcall | schemaalternation -- are intentionally not included, per the "happy to split out" note.Closes #425.
🤖 Generated with Claude Code
https://claude.ai/code/session_018EWo1niax1g8A5E79RSALJ