Skip to content

Fix _emit_string round-trip for number/date-shaped string values#68

Merged
rmichaelthomas merged 1 commit into
mainfrom
fix/emit-string-numeric-date-roundtrip
Jul 22, 2026
Merged

Fix _emit_string round-trip for number/date-shaped string values#68
rmichaelthomas merged 1 commit into
mainfrom
fix/emit-string-numeric-date-roundtrip

Conversation

@rmichaelthomas

Copy link
Copy Markdown
Owner

The defect

renderer._emit_string(s) decides whether a string value needs quotes to survive the round-trip property parse(tokenize(render(ast))) == ast. The previous rule quoted iff the value contained a space, matched a reserved word, or differed from its lowercased form.

That rule missed two self-typing lexical shapes. lexer._classify resolves a bare unquoted word as: reserved-word tables → _NUMBER_RE_DATE_REUNKNOWN. So a string whose content is number-shaped or date-shaped, emitted bare, re-lexed as a NUMBER or DATE token — not a string:

  • _emit_string("2025-07-01")2025-07-01 → re-lexes as DATE → parses to DateLiteral, not the original string.
  • _emit_string("75")75 → re-lexes as NUMBER → parses to NumberLiteral.
  • _emit_string("-3.5") → same break (full _NUMBER_RE shape: sign + decimal).

This is a Calendar-Era (v29) regression in kind: _DATE_RE was added to the lexer as a new self-typing shape, and _emit_string was never taught about it. The number case shares the root cause.

The fix

Added one clause to _emit_string's quoting condition: also quote when the value matches _NUMBER_RE or _DATE_RE.

Import, not copy. _NUMBER_RE/_DATE_RE are imported from .lexer rather than re-declared in renderer.py — mirroring the function's existing pattern of importing ALL_RESERVED from vocabulary instead of restating the reserved-word list. A local copy would create a second source of truth that must be updated in lockstep every time the lexer's literal grammar grows, which is exactly the failure that produced this bug. lexer.py imports only re and vocabulary (not renderer), so renderer → lexer is one-directional and adds no import cycle.

Out of scope (per the build spec): no rename of _NUMBER_RE/_DATE_RE, no changes to lexer.py, no changes to the QuotedString/BareWord/RememberValueNode with-vs-from routing logic (a separate, already-correct round-trip guard), no vocabulary or AST changes.

Invariants confirmed

  • grep -nE 'compile\(r?["\x27]\^' src/liminate/renderer.py → no matches (regex sourced only via import, not copied).
  • python3 -c "import liminate.renderer, liminate.lexer" → imports cleanly, no circular import.
  • _emit_string('2025-07-01') → '"2025-07-01"', _emit_string('75') → '"75"', _emit_string('-3.5') → '"-3.5"', _emit_string('active') → 'active' (matches spec exactly).
  • Real NumberLiteral/DateLiteral nodes still render bare (_emit_string is only reached from BareWord/QuotedString branches).

Tests

Added to tests/test_renderer.py:

  • Unit-level _emit_string quoting check for date/number-shaped content plus the negative/decimal shape.
  • Three new round-trip cases in the existing ROUND_TRIP_SENTENCES parametrized suite.
  • Explicit AST-level guards that a date/number-shaped QuotedString value stays a QuotedString through render + re-parse (not a DateLiteral/NumberLiteral).
  • Regression guard that genuine NumberLiteral/DateLiteral nodes keep rendering bare.

pytest tests/test_renderer.py -v → 77 passed (8 new).
pytest -q (full suite) → 1845 passed, no regressions.

Release note

This warrants the dormant 0.16.1 publish (per the build spec) — the fix changes rendered output for a previously-broken case, so it needs a version bump to propagate. PyPI release remains the architect's manual gate; not part of this PR.

🤖 Generated with Claude Code

lexer._classify resolves a bare unquoted word as reserved-word tables
-> _NUMBER_RE -> _DATE_RE -> UNKNOWN. renderer._emit_string didn't know
about the number/date shapes, so a string value like "2025-07-01" or
"75" rendered bare and re-lexed as a DATE/NUMBER token on the next
parse instead of staying a string.

Import _NUMBER_RE/_DATE_RE from .lexer rather than re-declaring them,
mirroring how _emit_string already imports ALL_RESERVED from
vocabulary instead of restating the reserved-word list.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@rmichaelthomas
rmichaelthomas merged commit 866d5ba into main Jul 22, 2026
2 checks passed
@rmichaelthomas
rmichaelthomas deleted the fix/emit-string-numeric-date-roundtrip branch July 22, 2026 03:31
rmichaelthomas added a commit that referenced this pull request Jul 22, 2026
Publishes the _emit_string round-trip fix for number/date-shaped
string values (PR #68), which landed on main after the v0.17.0 bump.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant