Skip to content

fix(microflows): DESCRIBE emitted aggregate MDL that would not parse back (#1004) - #349

Merged
ako merged 4 commits into
mainfrom
claude/sudoku-test-issue-46-iyxn30
Aug 31, 2026
Merged

fix(microflows): DESCRIBE emitted aggregate MDL that would not parse back (#1004)#349
ako merged 4 commits into
mainfrom
claude/sudoku-test-issue-46-iyxn30

Conversation

@ako

@ako ako commented Aug 31, 2026

Copy link
Copy Markdown
Owner

Fixes mendixlabs#1004.

The report

Describing a microflow containing Mendix's Reduce aggregate produced MDL that mxcli's own checker then rejected:

$Reduce = reduce($ContractList, $currentResult + $currentObject/Number);

✗ set 'Reduce' calls 'reduce()', which is not a Mendix expression function
  — the build fails CE0117 "Error(s) in expression" [MDL044]

Note the word set. The parser did not reject reduce(...) — it read the line as a plain Change Variable whose value happened to be a function call, and MDL044 was then correct that Mendix has no reduce() expression function.

Root cause

DESCRIBE rendered an aggregate as strings.ToLower(storedEnumValue), silently assuming every value of Mendix's AggregateFunction enumeration was also an MDL keyword. Mendix has eight; the grammar had five:

Function DESCRIBE emitted mxcli check
Count / Sum / Average / Minimum / Maximum count(...)
Reduce reduce($list, expr) ❌ MDL044
All all($list, expr) ❌ MDL044
Any any($list, expr) ❌ MDL044

The sibling formatListOperation switches on concrete types and cannot drift this way. Stringifying an enum into a keyword is the shape that outgrows its grammar every time Mendix adds a function.

The half the issue doesn't mention

Mendix stores a Reduce's seed and result type in ReduceInitialValueExpression and ReduceReturnDataType. The semantic model had a field for neither, so both engines read them as nothing and wrote them back as nothing.

That is the dangerous half. The grammar gap failed loudly; this one would have failed silently — rewriting a microflow containing a Reduce would have deleted the user's fold and left a model that still passes mx check. MDL044 was the only thing preventing it, by accident.

ast.AggregateReduce and its flowBuilder case already existed, unreachable, with no grammar rule and no visitor mapping behind them.

What the reference documents showed

Measured against Microflows.MicroflowReduce in ako/TestApp (Mendix 11.14), which contradicts Mendix's own reference guide:

Property Reduce All Any
ReduceInitialValueExpression "false" "" ""
ReduceReturnDataType Boolean Boolean Boolean
Attribute "" "" ""

The reference guide says a return type is "not applicable" to All and Any. Studio Pro writes one anyway. So these are not Reduce-only properties.

Consequently the pair is written for Reduce/All/Any, and for the five older functions only to carry back what a stored document already had — no reference document exists for those, and inventing a key is what produces a document mxbuild accepts and Studio Pro cannot open.

Syntax

$Folded   = reduce($list, $currentResult + $currentObject/Amount, initial: 0, returns: Decimal);
$AllMatch = all($list, $currentObject/Paid);
$AnyMatch = any($list, $currentObject/Paid);

reduce names its seed and result type because Mendix requires both and neither is inferable from the expression — guessing either is how the fold gets lost. all and any take neither: they never accumulate and always fold to Boolean, so a written type would only let an author contradict Mendix.

REDUCE, ANY and INITIAL are new lexer tokens and are added to the keyword rule so they stay usable as identifiers (ALL and RETURNS already existed). Reusing ALL in expression position introduces no ambiguity — ANTLR generates the parser without warnings.

Evidence

  • All three activities in the reference microflow now re-serialize byte-identically to what Studio Pro wrote (compared by output variable, $IDs normalised).
  • mx check on an mxcli-authored reduce/all/any microflow: 0 errors on mxbuild 11.14.
  • Both engines produce identical BSON and identical DESCRIBE output (MXCLI_ENGINE=legacy verified).
  • Re-running the bug-test script against an in-sync project reports Unchanged microflow (ADR-0008).

Controls, since a test that only passes against fixed code has not been shown to detect anything:

  • On origin/main, reduce/all/any each parse to a Change Variable rather than an aggregate — with sum → aggregate as the positive control that the harness works.
  • Reverting the two gen setters makes TestReduceFoldReachesStorage report both keys missing from the stored document.
  • mx check is no control here: 0 errors before and after. It tolerates the omission entirely.

Tests

  • TestDescribedAggregateParsesBack — drives every function in microflows.AllAggregateFunctions through describe and back through the parser, so a ninth function Mendix adds fails a test rather than a user's script.
  • TestAggregateKeywordsCoverEveryFunction — guards the keyword mapping itself.
  • TestReduceFoldReachesStorage — asserts on the stored BSON, not the semantic model that produced it, because everything above that layer looked right while the document was wrong.
  • TestAggregateAcceptsReduceAllAny, TestReduceKeywordsStayUsableAsIdentifiers — visitor level.
  • mdl-examples/bug-tests/1004-aggregate-reduce-all-any.mdl — re-runnable repro carrying the five older functions as an in-script control.

Also in this branch

  • Attribute is now written as "" when unused (its own commit). Without it, every Studio Pro-authored aggregate rewrote on its first execution for no semantic reason.
  • The regenerated LSP completions pick up CUSTOMBUTTON and ALLOWEDFILEFORMAT, missing since dbc26ffd added them to the lexer without regenerating the file.

Not fixed here

A describe → exec round trip still reports Replaced for Studio Pro microflows generally — three aggregate-free ones behave the same, so this is not aggregate-specific. The causes are pre-existing and unrelated: mxcli writes flow connection indices as int32 where Studio Pro writes int64, drops curve control vectors, and writes an empty caption where Studio Pro writes "Activity". Worth its own issue.

Checks

make build, go test ./..., make check-mdl, make lint-go, make check-tunnel-deps and scripts/check-skill-mdl.sh (×3) all pass on the rebased tree.


Generated by Claude Code

claude added 4 commits August 31, 2026 19:16
Mendix keeps a Reduce's fold in two properties beside the expression —
ReduceInitialValueExpression (the seed for $currentResult) and
ReduceReturnDataType (what the fold produces). The semantic model had a
field for neither, so both engines read them as nothing and wrote them
back as nothing: rewriting a microflow containing a Reduce deleted the
user's fold and left a model that still passed mx check.

Both properties now round-trip through AggregateListAction and all four
read/write paths.

The shape is measured against Studio Pro, not inferred, and the reference
documents contradict Mendix's own reference guide in two places:

  - Both keys are written on *every* AggregateAction, not only on Reduce.
    The All and Any activities each carry an empty initial value and a
    Boolean return type, though the guide says a return type is "not
    applicable" to them. So the pair is written for Reduce/All/Any, and
    for the five older functions only to carry back what a stored
    document already had — no reference document exists for those, and
    inventing a key is what produces a document mxbuild accepts and
    Studio Pro cannot open.

  - Attribute is stored as "" when unused. It was omitted, which made a
    freshly described Studio Pro aggregate rewrite on its first execution
    for no semantic reason.

With all three, the Reduce/All/Any activities in ako/TestApp
(Microflows.MicroflowReduce, Mendix 11.14) re-serialize byte-identically
to what Studio Pro wrote.

TestReduceFoldReachesStorage asserts on the stored BSON rather than the
semantic model that produced it, because everything above this layer
looked right while the document was wrong. Its control: revert the two
setters and it reports both keys missing.

Refs mendixlabs#1004

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CqQLyxnppZGfqPYbUHvSWG
Mendix has eight aggregate functions; listAggregateOperation had five.
The three it lacked are the ones DESCRIBE was already emitting, so a
Studio Pro-authored Reduce described as MDL that would not parse — it
fell through to a plain Change Variable, and MDL044 then correctly
reported that Mendix has no reduce() expression function.

  $Folded   = reduce($list, expr, initial: seed, returns: Type);
  $AllMatch = all($list, boolean-expression);
  $AnyMatch = any($list, boolean-expression);

REDUCE names its seed and result type because Mendix requires both and
neither is inferable from the expression — guessing either is how the
fold gets silently lost. ALL and ANY take neither: they never accumulate
and always fold to Boolean, so writing a type would only let an author
contradict Mendix.

REDUCE, ANY and INITIAL are new lexer tokens, so they are also added to
the `keyword` rule to stay usable as identifiers; ALL and RETURNS already
existed. Reusing ALL in expression position introduces no ambiguity —
ANTLR generates the parser without warnings.

ast.AggregateReduce and its flowBuilder case already existed, unreachable,
with no grammar rule and no visitor mapping behind them.

The regenerated completions also pick up CUSTOMBUTTON and ALLOWEDFILEFORMAT,
which have been missing since dbc26ff added them to the lexer without
regenerating the file.

Refs mendixlabs#1004

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CqQLyxnppZGfqPYbUHvSWG
DESCRIBE rendered an aggregate with strings.ToLower on whatever Mendix
had stored, which silently assumed every value of the AggregateFunction
enumeration was also an MDL keyword. Five were. Reduce, All and Any were
not, so mxcli emitted MDL its own checker rejected (mendixlabs#1004) — and would
keep doing so for every function Mendix adds.

The mapping is now explicit, and a function with no keyword is reported
as an unrenderable activity rather than emitted as a plausible-looking
lie. REDUCE additionally renders its seed and result type, without which
the described statement cannot be executed back.

The sibling formatListOperation switches on concrete types and cannot
drift this way; stringifying an enum into a keyword is the shape to
avoid.

TestDescribedAggregateParsesBack drives every function in
microflows.AllAggregateFunctions through describe and back through the
parser, so a ninth function fails a test rather than a user's script.
Control on origin/main: reduce/all/any each parse to a Change Variable
rather than an aggregate, with sum as the positive control.

Closes mendixlabs#1004

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CqQLyxnppZGfqPYbUHvSWG
… repro

Aggregates were undocumented in the quick reference and the
write-microflows skill, and the syntax topic listed three of the eight.
All three now cover the full set, with reduce's mandatory initial/returns
and the reason they are mandatory.

The bug test records what the reference documents showed that Mendix's
own guide does not — that a return type is stored for All and Any too —
and is re-runnable, so a second exec against an in-sync project must
report "Unchanged microflow".

Refs mendixlabs#1004

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CqQLyxnppZGfqPYbUHvSWG

ako commented Aug 31, 2026

Copy link
Copy Markdown
Owner Author

github-advanced-security is red, and it is not this PR's

The check failed 34 seconds in, before scanning anything:

Error creating PR review request: SessionModelError: Execution failed:
CAPIError: 400 The requested model is not supported.
COPILOT_AGENT_MODEL: sweagent-capi:claude-opus-4.6

Three things establish it is not this diff's, and I've made no change for it:

  1. It fails identically on fix(describe): quote and escape MDL strings in one place (mendixlabs/mxcli#1006) #348 — a different author's unrelated PR, opened 50 minutes before this one. Same check, same conclusion.
  2. The check is not defined in this repository. There is no workflow for it in .github/workflows/; it is the GitHub-side ghas-code-scanning-agentic app, configured outside the repo. No commit here can change which model its backend accepts.
  3. It excludes Go from scanning anyway. Its own FileExclusionPatterns lists *.go, so it would not have examined this PR's changes even had the session started.

There is no fix to port: the failure is model availability in a GitHub-hosted service, not repository content. I re-ran it once to rule out a transient; if it stays red it needs someone with repo/org settings access, not a code change.

Separately: the build-and-test failure on #348 is already fixed on main

Worth flagging since it will look like a shared breakage. #348's build-and-test fails on:

--- FAIL: TestSessionLog_PersistAndPrune
    sessions_test.go:53: after reload+prune: 0 records, want 1 ([])

That is the dated fuse 7f4d7303 ("TestSessionLog_PersistAndPrune had a fuse on it") fixed — a hardcoded 2026-08-01 base against a 30-day retention, which went off today, 2026-08-31, on every branch at once. I hit it locally on the old main (a31729ee) before rebasing.

This PR is rebased onto 23627a8d, which carries 7f4d7303, so it is unaffected — the full suite passes locally on the rebased tree. #348 needs a rebase onto current main, not a fix of its own.


Generated by Claude Code

ako commented Aug 31, 2026

Copy link
Copy Markdown
Owner Author

Correction to the comment above: I said I re-ran the failed check. I could not — the re-run was refused:

POST /repos/ako/mxcli/actions/runs/33431914538/rerun-failed-jobs
403 Resource not accessible by integration

Everything else in that comment stands, and the evidence for it does not depend on a re-run: the same check fails on #348's unrelated diff, it is not defined in this repository, and it excludes *.go from scanning. Someone with the right permissions may want to re-run it to confirm, though a 400 "model not supported" from the scanning backend is unlikely to be transient.

I'll keep watching this PR until it is green and mergeable.


Generated by Claude Code

@ako
ako merged commit fecb3ae into main Aug 31, 2026
13 of 14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

roundtrip microflow with reduce does not work

2 participants