fix(microflows): DESCRIBE emitted aggregate MDL that would not parse back (#1004) - #349
Conversation
Mendix keeps a Reduce's fold in two properties beside the expression —
ReduceInitialValueExpression (the seed for $currentResult) and
ReduceReturnDataType (what the fold produces). The semantic model had a
field for neither, so both engines read them as nothing and wrote them
back as nothing: rewriting a microflow containing a Reduce deleted the
user's fold and left a model that still passed mx check.
Both properties now round-trip through AggregateListAction and all four
read/write paths.
The shape is measured against Studio Pro, not inferred, and the reference
documents contradict Mendix's own reference guide in two places:
- Both keys are written on *every* AggregateAction, not only on Reduce.
The All and Any activities each carry an empty initial value and a
Boolean return type, though the guide says a return type is "not
applicable" to them. So the pair is written for Reduce/All/Any, and
for the five older functions only to carry back what a stored
document already had — no reference document exists for those, and
inventing a key is what produces a document mxbuild accepts and
Studio Pro cannot open.
- Attribute is stored as "" when unused. It was omitted, which made a
freshly described Studio Pro aggregate rewrite on its first execution
for no semantic reason.
With all three, the Reduce/All/Any activities in ako/TestApp
(Microflows.MicroflowReduce, Mendix 11.14) re-serialize byte-identically
to what Studio Pro wrote.
TestReduceFoldReachesStorage asserts on the stored BSON rather than the
semantic model that produced it, because everything above this layer
looked right while the document was wrong. Its control: revert the two
setters and it reports both keys missing.
Refs mendixlabs#1004
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CqQLyxnppZGfqPYbUHvSWG
Mendix has eight aggregate functions; listAggregateOperation had five. The three it lacked are the ones DESCRIBE was already emitting, so a Studio Pro-authored Reduce described as MDL that would not parse — it fell through to a plain Change Variable, and MDL044 then correctly reported that Mendix has no reduce() expression function. $Folded = reduce($list, expr, initial: seed, returns: Type); $AllMatch = all($list, boolean-expression); $AnyMatch = any($list, boolean-expression); REDUCE names its seed and result type because Mendix requires both and neither is inferable from the expression — guessing either is how the fold gets silently lost. ALL and ANY take neither: they never accumulate and always fold to Boolean, so writing a type would only let an author contradict Mendix. REDUCE, ANY and INITIAL are new lexer tokens, so they are also added to the `keyword` rule to stay usable as identifiers; ALL and RETURNS already existed. Reusing ALL in expression position introduces no ambiguity — ANTLR generates the parser without warnings. ast.AggregateReduce and its flowBuilder case already existed, unreachable, with no grammar rule and no visitor mapping behind them. The regenerated completions also pick up CUSTOMBUTTON and ALLOWEDFILEFORMAT, which have been missing since dbc26ff added them to the lexer without regenerating the file. Refs mendixlabs#1004 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CqQLyxnppZGfqPYbUHvSWG
DESCRIBE rendered an aggregate with strings.ToLower on whatever Mendix had stored, which silently assumed every value of the AggregateFunction enumeration was also an MDL keyword. Five were. Reduce, All and Any were not, so mxcli emitted MDL its own checker rejected (mendixlabs#1004) — and would keep doing so for every function Mendix adds. The mapping is now explicit, and a function with no keyword is reported as an unrenderable activity rather than emitted as a plausible-looking lie. REDUCE additionally renders its seed and result type, without which the described statement cannot be executed back. The sibling formatListOperation switches on concrete types and cannot drift this way; stringifying an enum into a keyword is the shape to avoid. TestDescribedAggregateParsesBack drives every function in microflows.AllAggregateFunctions through describe and back through the parser, so a ninth function fails a test rather than a user's script. Control on origin/main: reduce/all/any each parse to a Change Variable rather than an aggregate, with sum as the positive control. Closes mendixlabs#1004 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CqQLyxnppZGfqPYbUHvSWG
… repro Aggregates were undocumented in the quick reference and the write-microflows skill, and the syntax topic listed three of the eight. All three now cover the full set, with reduce's mandatory initial/returns and the reason they are mandatory. The bug test records what the reference documents showed that Mendix's own guide does not — that a return type is stored for All and Any too — and is re-runnable, so a second exec against an in-sync project must report "Unchanged microflow". Refs mendixlabs#1004 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CqQLyxnppZGfqPYbUHvSWG
|
|
Correction to the comment above: I said I re-ran the failed check. I could not — the re-run was refused: Everything else in that comment stands, and the evidence for it does not depend on a re-run: the same check fails on #348's unrelated diff, it is not defined in this repository, and it excludes I'll keep watching this PR until it is green and mergeable. Generated by Claude Code |
Fixes mendixlabs#1004.
The report
Describing a microflow containing Mendix's Reduce aggregate produced MDL that mxcli's own checker then rejected:
Note the word set. The parser did not reject
reduce(...)— it read the line as a plain Change Variable whose value happened to be a function call, and MDL044 was then correct that Mendix has noreduce()expression function.Root cause
DESCRIBErendered an aggregate asstrings.ToLower(storedEnumValue), silently assuming every value of Mendix'sAggregateFunctionenumeration was also an MDL keyword. Mendix has eight; the grammar had five:mxcli checkcount(...)…reduce($list, expr)all($list, expr)any($list, expr)The sibling
formatListOperationswitches on concrete types and cannot drift this way. Stringifying an enum into a keyword is the shape that outgrows its grammar every time Mendix adds a function.The half the issue doesn't mention
Mendix stores a Reduce's seed and result type in
ReduceInitialValueExpressionandReduceReturnDataType. The semantic model had a field for neither, so both engines read them as nothing and wrote them back as nothing.That is the dangerous half. The grammar gap failed loudly; this one would have failed silently — rewriting a microflow containing a Reduce would have deleted the user's fold and left a model that still passes
mx check. MDL044 was the only thing preventing it, by accident.ast.AggregateReduceand itsflowBuildercase already existed, unreachable, with no grammar rule and no visitor mapping behind them.What the reference documents showed
Measured against
Microflows.MicroflowReducein ako/TestApp (Mendix 11.14), which contradicts Mendix's own reference guide:ReduceInitialValueExpression"false"""""ReduceReturnDataTypeAttribute""""""The reference guide says a return type is "not applicable" to All and Any. Studio Pro writes one anyway. So these are not Reduce-only properties.
Consequently the pair is written for Reduce/All/Any, and for the five older functions only to carry back what a stored document already had — no reference document exists for those, and inventing a key is what produces a document mxbuild accepts and Studio Pro cannot open.
Syntax
reducenames its seed and result type because Mendix requires both and neither is inferable from the expression — guessing either is how the fold gets lost.allandanytake neither: they never accumulate and always fold to Boolean, so a written type would only let an author contradict Mendix.REDUCE,ANYandINITIALare new lexer tokens and are added to thekeywordrule so they stay usable as identifiers (ALLandRETURNSalready existed). ReusingALLin expression position introduces no ambiguity — ANTLR generates the parser without warnings.Evidence
$IDs normalised).mx checkon an mxcli-authored reduce/all/any microflow: 0 errors on mxbuild 11.14.MXCLI_ENGINE=legacyverified).Unchanged microflow(ADR-0008).Controls, since a test that only passes against fixed code has not been shown to detect anything:
origin/main,reduce/all/anyeach parse to a Change Variable rather than an aggregate — withsum→ aggregate as the positive control that the harness works.TestReduceFoldReachesStoragereport both keys missing from the stored document.mx checkis no control here: 0 errors before and after. It tolerates the omission entirely.Tests
TestDescribedAggregateParsesBack— drives every function inmicroflows.AllAggregateFunctionsthrough describe and back through the parser, so a ninth function Mendix adds fails a test rather than a user's script.TestAggregateKeywordsCoverEveryFunction— guards the keyword mapping itself.TestReduceFoldReachesStorage— asserts on the stored BSON, not the semantic model that produced it, because everything above that layer looked right while the document was wrong.TestAggregateAcceptsReduceAllAny,TestReduceKeywordsStayUsableAsIdentifiers— visitor level.mdl-examples/bug-tests/1004-aggregate-reduce-all-any.mdl— re-runnable repro carrying the five older functions as an in-script control.Also in this branch
Attributeis now written as""when unused (its own commit). Without it, every Studio Pro-authored aggregate rewrote on its first execution for no semantic reason.CUSTOMBUTTONandALLOWEDFILEFORMAT, missing sincedbc26ffdadded them to the lexer without regenerating the file.Not fixed here
A describe → exec round trip still reports
Replacedfor Studio Pro microflows generally — three aggregate-free ones behave the same, so this is not aggregate-specific. The causes are pre-existing and unrelated: mxcli writes flow connection indices as int32 where Studio Pro writes int64, drops curve control vectors, and writes an empty caption where Studio Pro writes"Activity". Worth its own issue.Checks
make build,go test ./...,make check-mdl,make lint-go,make check-tunnel-depsandscripts/check-skill-mdl.sh(×3) all pass on the rebased tree.Generated by Claude Code