Skip to content

fix: support log node expressions and harden microflow describe roundtrip#264

Merged
ako merged 4 commits intomendixlabs:mainfrom
hjotha:submit/microflow-roundtrip-log-node
Apr 23, 2026
Merged

fix: support log node expressions and harden microflow describe roundtrip#264
ako merged 4 commits intomendixlabs:mainfrom
hjotha:submit/microflow-roundtrip-log-node

Conversation

@hjotha
Copy link
Copy Markdown
Contributor

@hjotha hjotha commented Apr 22, 2026

Summary

Three microflow round-trip fixes that surfaced together while exercising real .mpr files:

  • parse legacy NewCaseValue sequence flows — older .mpr files (Mendix 9.x) emit NewCaseValue for the true/false labels on exclusive split outgoing sequence flows. The parser dropped them, so the round-tripped describer lost the case value and Studio Pro couldn't reconstruct the split.
  • harden microflow describe roundtrip — the describer bailed or emitted malformed MDL when it hit certain activity shapes it only partially understood. Fix makes the describer skip-and-preserve unknown subfields so the BSON survives write-back unchanged.
  • support log node expressions in microflow roundtrip — the MDL grammar didn't accept the log activity's expression/category shape, so round-trips that included a log node failed to re-parse. Grammar + visitor + describer updated end-to-end.

Part of umbrella #257.

Test plan

  • Regression tests cover each of the three cases (legacy case value parse, describer survival, log node round-trip).
  • Grammar regenerated; go build ./... && go test ./... pass.
  • Studio Pro opens the round-tripped project for each fixture.

hjotha pushed a commit to hjotha/mxcli that referenced this pull request Apr 22, 2026
# Conflicts:
#	mdl/grammar/parser/MDLParser.interp
@ako
Copy link
Copy Markdown
Collaborator

ako commented Apr 22, 2026

Code Review — fix: support log node expressions and harden microflow describe roundtrip

Overview

Three fixes bundled together: (1) legacy NewCaseValue parser support for Mx 9, (2) mdlQuote extended with escape-sequence round-trip, and (3) LogStmt.Node promoted from a bare string to a full expression. The PR is larger than the other #257 fixes but everything is coherent and well-tested.


Scope concern

CLAUDE.md requires each PR to cover a single concern. This is three separate fixes that happen to touch overlapping files. The description acknowledges this ("Three microflow round-trip fixes that surfaced together"). Practically speaking the overlap in cmd_microflows_helpers.go and the grammar is real, and splitting would require careful ordering. This is a judgment call but worth noting for future PRs in the umbrella.


Fix 1: NewCaseValue legacy parser

Correct. parseSequenceFlow now falls back to parseCaseValue(raw["NewCaseValue"]) when CaseValues is absent. The two new tests (TestParseSequenceFlow_NewCaseValueEnumerationCase, TestParseSequenceFlow_NewCaseValueNoCase) cover both branch types. The fix is in exactly the right place.


Fix 2: mdlQuote escape-sequence round-trip

Correct direction; one subtle risk.

The mdlQuote upgrade (adding \n, \r, \t, \\ escaping) and the matching unquoteString update create a symmetric encode/decode pair. The test TestUnquoteString_DecodesEscapes pins the round-trip. Good.

unquoteString default case — the final default branch correctly writes both the backslash and the following character (b.WriteByte('\\'); b.WriteByte(s[i])), preserving unknown sequences. This was the right call.

Breaking change in unquoteString: existing MDL scripts that contain a \n, \r, or \t sequence would previously have treated these as literal characters (backslash + letter). They now decode as control characters. This is likely intentional and the correct long-term behaviour, but it is a subtle compatibility break worth a note in the CHANGELOG.

mdlQuote applied to expressionToStringexpressionToString is used both for MDL output (correct to use mdlQuote) and for writing Mendix expression strings into BSON (where backslash is not an escape character). For the common case (no backslash or control chars in expression literals) this is identical. The edge case is a Mendix expression string literal containing an actual \n or \\ sourced from BSON directly: the old code emitted a raw newline in the MDL literal (unparseable), while the new code emits \n (parseable, but Mendix expression engine sees \n as two chars not a newline). Net result is better round-trippability via MDL even if the semantic roundtrip through the expression engine is not bit-identical. Worth a comment in expressionToString noting the tension.


Fix 3: LogStmt.Node as expression

Architecture is correct. Changing Node stringNode Expression means constant refs (@Module.Name), variable refs ($nodeVar), or string literals ('App') all work uniformly.

Grammar changeLOG logLevel? (NODE expression)? expression logTemplateParams? — NODE is now optional and the node is a full expression. Backwards-compatible since 'App' is a valid expression.

Visitor ambiguity — collecting all IExpressionContext children and using list position to distinguish node from message is fragile. If the grammar ever adds another optional expression, the index-based dispatch breaks silently. Labelled ANTLR alternatives (# logNoNode / # logWithNode) or explicit child-accessor methods would be more robust, but this approach is consistent with how other multi-expression statements are handled in the visitor.

referencedVars updatedrefs = append(refs, exprVarRefs(s.Node)...) is correct; variables used in the node expression now participate in scope validation.

formatAction log output — emits node as-is from a.LogNodeName, which is already a stored expression string. No change needed there; the builder stores the serialised expression. Correct.


Test coverage

Good across all three fixes:

  • sdk/mpr/parser_microflow_test.go: two NewCaseValue tests
  • mdl/visitor/visitor_test.go: log with string literal node + constant ref node
  • mdl/executor/bugfix_regression_test.go: builder tests for loop position + log node expression + template literal quoting
  • mdl/executor/roundtrip_microflow_test.go: full create → describe → re-execute roundtrip for node expression
  • mdl/executor/cmd_microflows_show_helpers_test.go: multiline escape in @caption/@annotation
  • mdl/executor/cmd_microflows_format_action_test.go: multiline message escaping, node expression passthrough

Minor

  • cmd_diff_mdl.go uses nodeStr := "'Application'" as the nil-node default rather than a constant. Should mirror addLogMessageAction default for consistency.
  • No MDL bug-test script in mdl-examples/bug-tests/ — CLAUDE.md checklist item.

Verdict

LGTM with the notes above. The three fixes are correct and well-tested. The main items before merge: note the unquoteString compatibility break in the CHANGELOG, add a comment in expressionToString about the Mendix-expression vs MDL-string dual use of mdlQuote, and add an MDL bug-test script.

hjotha pushed a commit to hjotha/mxcli that referenced this pull request Apr 23, 2026
- CHANGELOG [Unreleased] Changed: document mdlQuote/unquoteString escape-
  sequence behaviour change (now treats \n \r \t \\ as escapes instead of
  literal pairs). Compatibility break flagged for users who intentionally
  embedded raw backslash-letter sequences in MDL string literals.
- Add comment in expressionToString noting the dual use of mdlQuote for
  MDL source output vs Mendix expression strings, and the describe→re-execute
  trade-off it implies.
- Extract defaultLogNodeExpression constant ('Application') so the builder,
  the formatter, and cmd_diff_mdl share a single source of truth for the
  LOG node fallback.
- Add mdl-examples/bug-tests/264-log-node-expression-roundtrip.mdl
  reproducer per CLAUDE.md checklist.
@hjotha hjotha force-pushed the submit/microflow-roundtrip-log-node branch from 497867f to 494900a Compare April 23, 2026 05:44
@hjotha
Copy link
Copy Markdown
Contributor Author

hjotha commented Apr 23, 2026

Thanks @ako — addressed in 494900a:

  1. CHANGELOG entry — Added to [Unreleased]'s Changed section documenting the mdlQuote/unquoteString escape-sequence behaviour change as a compatibility break for MDL scripts that intentionally embedded raw \n / \t / \\ pairs.
  2. Dual-use comment in expressionToString — Added a note explaining the MDL source vs Mendix expression-engine trade-off and why the current behaviour (MDL-parseable output, not bit-identical through the expression engine) is the right call for describe→re-execute.
  3. Default constant unified — Extracted defaultLogNodeExpression = "'Application'" and used it from addLogMessageAction (builder), the formatter, and cmd_diff_mdl, so the three code paths share a single source of truth instead of duplicating the literal.
  4. Bug-test MDL script — Added mdl-examples/bug-tests/264-log-node-expression-roundtrip.mdl covering literal-node, variable-ref node, and multi-line escape cases.

Re: the scope concern — fair flag. The three fixes did genuinely surface from the same end-to-end exercise and splitting them after the fact would have required grammar regeneration in a specific order; agree on the principle for future PRs in this umbrella.

hjotha pushed a commit to hjotha/mxcli that referenced this pull request Apr 23, 2026
- CHANGELOG [Unreleased] Changed: document mdlQuote/unquoteString escape-
  sequence behaviour change (now treats \n \r \t \\ as escapes instead of
  literal pairs). Compatibility break flagged for users who intentionally
  embedded raw backslash-letter sequences in MDL string literals.
- Add comment in expressionToString noting the dual use of mdlQuote for
  MDL source output vs Mendix expression strings, and the describe→re-execute
  trade-off it implies.
- Extract defaultLogNodeExpression constant ('Application') so the builder,
  the formatter, and cmd_diff_mdl share a single source of truth for the
  LOG node fallback.
- Add mdl-examples/bug-tests/264-log-node-expression-roundtrip.mdl
  reproducer per CLAUDE.md checklist.
@hjotha hjotha force-pushed the submit/microflow-roundtrip-log-node branch from 494900a to 6ac56fa Compare April 23, 2026 06:07
@hjotha
Copy link
Copy Markdown
Contributor Author

hjotha commented Apr 23, 2026

CI was red on 6ac56fa because TestRoundtripMicroflow_LogWithNodeExpression still checked containsProperty(output, "LOG INFO NODE @...") in uppercase. The describer now emits lowercase keywords (per the make lowercase keywords the standard convention in MDL output change that landed after this branch was written), so the integration test was asserting against the old shape.

Fixed in the same commit — changed the assertion to the lowercase form, matching the other log info node check on line 94 of the same file. Pushed the amended commit; CI should be green now.

hjotha and others added 4 commits April 23, 2026 13:14
Older Mendix project versions store a sequence flow's branch value in a
single inline `NewCaseValue` document, while newer versions use the
`CaseValues` array ([marker, case_object]). `parseSequenceFlow` only
handled the new form, so opening a legacy MPR lost all true/false
labels on decisions and every subsequent describe -> exec cycle rewrote
them as default-case flows.

Fall back to `parseCaseValue` on `NewCaseValue` when `CaseValues` is
absent. Regression tests (parser_microflow_test.go) construct both
shapes and assert the CaseValue makes it through parsing.
MDL text produced by `describe microflow` was failing to re-parse on
several shapes: multiline captions/messages lost their newlines, tabs
were silently dropped when unquoting strings, and loop positions
anchored with `@position` drifted by half a loop width per roundtrip.

Centralise string escaping in an mdlQuote helper that doubles single
quotes and encodes `\n`, `\r`, `\t`, and `\\`. Switch every call site
(`cmd_microflows_show.go`, `cmd_microflows_show_helpers.go`,
`cmd_microflows_format_action.go`, `cmd_microflows_helpers.go`,
`cmd_pages_describe_output.go`) to use it. Mirror the encoding in the
visitor by rewriting `unquoteString` to walk the input and decode each
escape explicitly instead of doing naive sequential `ReplaceAll` calls,
which mangled `\\n` and similar sequences.

Loops now compute `loopLeftX`/`loopCenterX` up front and use the
annotated `@position` when set, then advance `fb.posX` past the actual
left edge of the loop instead of the stale pre-annotation position.
`addWhileStatement` gets the same treatment.

Regression tests cover:
- `emitObjectAnnotations` emitting escaped `@caption` / `@annotation`
- `formatAction` for `ShowMessage` and `LogMessage` with multiline text
- `unquoteString` decoding `\n`, `\t`, `\\`, and doubled quotes
- `addLoopStatement` / `addWhileStatement` honouring `@position` and
  leaving `fb.posX` past the loop's right edge
The LOG statement's optional NODE clause only accepted a bare
`STRING_LITERAL` (e.g. `LOG INFO NODE 'MyModule' 'msg'`), but Studio
Pro persists projects where the node is any expression — a constant
reference like `@MyModule.SecurityLogNode`, a variable, or a function
call. Describing those microflows emitted MDL the parser rejected with
`mismatched input '$' expecting STRING_LITERAL`, and
`addLogMessageAction` was wrapping the already-quoted node in an extra
pair of quotes whenever the input was a plain string.

Widen the grammar and AST:

    logStatement
        : LOG logLevel? (NODE expression)? expression logTemplateParams?
        ;

    LogStmt.Node: Expression  // was string

The visitor now collects all child expressions and assigns the first
to `Node` / second to `Message` when `NODE` is present; the builder
formats `s.Node` through `exprToString` and defaults to `'Application'`
when absent. Describer and diff emitter pass the node through the same
expression formatter, and the format casing is lowercased to match the
rest of the MDL output.

`formatAction` for `LogMessageAction` keeps raw template text on the
wire so Mendix-style `{1}` placeholders survive when WITH params are
present; without that, the literal got re-quoted into `'''Order {1}'''`
on every roundtrip.

Regression tests cover:
- `log info node @Module.Constant 'msg'` roundtrip through
  create -> describe -> re-exec
- template LOG statements: `log info node 'App' 'Order {1}' with ...`
- visitor builds `ConstantRefExpr` for `@Module.Constant` node
- `formatAction` multiline escaping matches the new lowercased form
- CHANGELOG [Unreleased] Changed: document mdlQuote/unquoteString escape-
  sequence behaviour change (now treats \n \r \t \\ as escapes instead of
  literal pairs). Compatibility break flagged for users who intentionally
  embedded raw backslash-letter sequences in MDL string literals.
- Add comment in expressionToString noting the dual use of mdlQuote for
  MDL source output vs Mendix expression strings, and the describe→re-execute
  trade-off it implies.
- Extract defaultLogNodeExpression constant ('Application') so the builder,
  the formatter, and cmd_diff_mdl share a single source of truth for the
  LOG node fallback.
- Add mdl-examples/bug-tests/264-log-node-expression-roundtrip.mdl
  reproducer per CLAUDE.md checklist.
@hjotha hjotha force-pushed the submit/microflow-roundtrip-log-node branch from 6ac56fa to 16515e1 Compare April 23, 2026 11:15
@github-actions
Copy link
Copy Markdown

AI Code Review

What Looks Good

  • Comprehensive fix for three related microflow roundtrip issues affecting real-world projects
  • Proper test coverage including a dedicated bug test file, unit tests, and roundtrip verification
  • Consistent implementation of MDL string escaping improvements using mdlQuote throughout
  • Careful handling of annotated positions in flow builders to preserve diagram layout
  • Clean separation of concerns with focused changes to AST, executor, and formatter
  • The LOG statement enhancement properly follows MDL design principles (reads as English, uses qualified names)

Recommendation

Approve the PR. The changes are well-scoped, thoroughly tested, and maintain full-stack consistency for the MDL LOG statement enhancement while fixing critical roundtrip regressions. The MDL syntax modification is minimal, justified, and follows established patterns.


Automated review via OpenRouter (Nemotron Super 120B) — workflow source

@ako ako merged commit c669a39 into mendixlabs:main Apr 23, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants