Sync ako/mxcli: six build-breakers, three new capabilities, and a registry-integrity test - #873
Merged
Conversation
…ree actions Closes the write-only gap: every microflow action mxcli can author on the default engine can now also be read back. TRANSFORM JSON was the serious one, and worse than the coverage audit suggested. The audit looked for missing READER cases and flagged `transform` as authorable-but-unreadable; in fact the modelsdk WRITER had no case for it either, so the action fell through to `default: return nil` and the enclosing ActionActivity was written with no action at all. `mxcli exec` reported "Created microflow" and mxbuild then failed CE0008 "No action defined." — the mendixlabs#850 shape. The grammar, builder, DESCRIBE formatter and the legacy writer all handled it, which is exactly why it looked supported. Readers added for TransformJsonAction, CallExternalAction and RestOperationCallAction, mirroring the keys each writer builds. Two of those diverge from what symmetry would suggest: an external action stores its result under `VariableName`, and REST's two mapping lists key their names differently (`Parameter` vs `QueryParameter`). CallExternalAction.ResultDataType is deliberately NOT reconstructed. It is resolved from the consumed service's cached $metadata at write time, so inferring it from the stored VariableDataType would let a stale value round-trip as if it had been authored. Verified on Mendix 11.13.0: `transform` now builds at 0 errors (CE0008 first became CE1613 for a missing transformer — the action existing at all was the fix), and describe→exec→describe is stable. All three round-trip tests fail against the pre-fix code. Coverage: 44 of 58 action types read by the default engine, and zero of the remainder are authorable — the 14 left have no model type in either direction. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013uQvFDd5R4eNqqita59jM8
`mxcli check` segfaulted on the one-line script `SQL DISCONNECT source;` —
and that line came from mxcli's own `mxcli syntax sql` example.
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation]
... visitor.(*Builder).ExitSqlDisconnect
Two defects.
Grammar: every sqlStatement alternative took a bare IDENTIFIER for the
connection alias, driver and table name. `source` lexes as SOURCE_KW
(MDLLexer.g4:317), so no alternative matched. IMPORT FROM, in the same
file, already used identifierOrKeyword for the very same alias — so
`import from source ...` worked while `sql source select ...` did not.
Aliases, drivers and table names are user-chosen words and now take
identifierOrKeyword throughout, which also unblocks `table`, `query`,
`view`, `index` and `key`.
Robustness: ANTLR error-recovers and keeps walking the tree, so the
listener ran against a context whose IDENTIFIER() was nil and .GetText()
took the process down. A statement that fails to parse must be a reported
error, never a crash. The sibling handlers already guarded their children
(`if len(ids) < 2 { return }`); DISCONNECT, GENERATE CONNECTOR and the
query passthrough did not. All five now read children through one helper
that tolerates a missing node.
Note the index shift in ExitSqlGenerateConnector: the alias joins the
AllIdentifierOrKeyword list, so the module moves from [0] to [1] and the
table/view names from [1:] to [2:]. Getting that wrong would silently
generate a connector into a module named after the connection, so it has
its own test.
Found by the new syntax-example guard (following commit). The panic
aborted the test binary, which had been masking ten further failures in
entries sorted after "sql" — a crash in a table-driven test is not one
failure, it is an unknown number.
Repro: mdl-examples/bug-tests/sql-keyword-alias-crash.mdl
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LUToAkUx54bNkNjsBpufRH
The feature registry is hand-maintained while the grammar moves
underneath it, and nothing tied the two together: the existing tests check
structure — fields populated, aliases resolve, see-also targets exist —
never whether the documented MDL is real. 26 of 120 entries had an
example that did not parse.
That matters more than a stale doc usually would, because the registry is
the first surface an agent consults, so a gap reads as "unsupported"
rather than "undocumented". Findings from a contact-management app built
with mxcli reported three features as missing that all already existed;
one was written off as a critical gap and worked around with a bespoke
microflow.
TestExamplesParse now feeds every Example through the parser. Examples
come in several legitimate shapes, so each blank-line-separated block is
tried as a statement, a microflow activity, a page widget, a workflow
activity and a retrieve clause; a failure means it parses as none of
them. Only Example is checked — Syntax carries metasyntax by design.
115 entries are actively checked; 5 are exempt with a stated reason
(error/fix prose, and `oql`, whose example is a shell command).
Corrections it forced out:
- `Binds:` in 6 entries. The parser hard-rejects it: "'Binds:' is no
longer supported, use 'Attribute:' instead". Anyone copying
`mxcli syntax page` hit an error immediately.
- Workflow decision outcomes were missing the `->` arrow. A user-task
outcome genuinely has none, so the two read alike and are separate
grammar rules.
- ALTER WORKFLOW was wrong four ways: `SET DUE DATE = '...'` (no `=`),
`INSERT <activity> AFTER <name>` (operands reversed), a `BEFORE` that
does not exist, and DROP/REPLACE missing the ACTIVITY keyword.
- `IMAGE 'logo'` — the image name is an identifier; only the path is a
string.
- A microflow datasource written as a call, `MICROFLOW M.GetData()`;
an unnamed TEXTBOX; a stray `;` after USE BUILDING BLOCK, which is a
page-body element; and literal `(...)`/`{ ... }` placeholders left in
Example fields where they belong in Syntax.
And the three gaps the app findings actually tripped on:
- `Action: SAVE_CHANGES CLOSE_PAGE` was absent — only the two halves
were listed separately, so closing a pop-up from a save button looked
impossible. This is the one that cost a bespoke microflow.
- RETRIEVE over an association (`FROM $Obj/Module.Assoc`) was absent,
so it read as unsupported despite working and being documented in the
write-microflows skill.
- OR REPLACE / OR MODIFY sit on the top-level createStatement rule and
apply to every CREATE, but were documented unevenly across entries
and never for OR REPLACE. Documented once as `create-modifiers`
rather than repeated across 27 entries, where it would drift again.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LUToAkUx54bNkNjsBpufRH
Close the write-only action gap: TransformJson write + the last three readers
Two conflicts in cmd/mxcli/syntax/features_page.go, both from main and this
branch correcting the same stale entries independently.
page.datasource: main fixed the Syntax field (dropping the spurious parens
from `MICROFLOW Module.MF()` and explaining when they apply); this branch
fixed the Example (replacing literal `{ ... }` placeholders with real
widget bodies and naming an anonymous TEXTBOX). Kept both halves — main's
Syntax, this branch's Example, which is the one TestExamplesParse holds to
parsing.
fragment: main had also switched `Binds:` to `Attribute:` here, so only
the CREATE PAGE header differed; kept this branch's real header over the
literal `(...)`.
The generated parser needed regenerating (`make grammar`) — main added
SYNCHRONIZE, `publish microflow`, and MOVE JAVA/ODATA rules. The syntax
entries main added along with them pass the new example guard unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LUToAkUx54bNkNjsBpufRH
Fix: hold every `mxcli syntax` example to actually parsing (+ SQL keyword-alias crash)
`ALTER PAGE … SET Action = microflow M.F ON btn` did not parse:
line 2:25 extraneous input 'M' expecting {DROP, ADD, SET, INSERT, REPLACE, '}'}
alterPageAssignment special-cases DATASOURCE, VISIBLE and EDITABLE and then
falls through to `identifierOrKeyword EQUALS propertyValueV3` — and
propertyValueV3 has no `microflow <name>` form, so the value position could
not hold an action at all. CREATE PAGE has had `ACTION COLON actionExprV3`
the whole time.
The documented workaround, REPLACE, works but is not equivalent: it rebuilds
the widget from what the statement says, so every property the author does
not restate — ButtonStyle, Class, design properties, tooltip — is silently
dropped. Retargeting one button should not require restating the button.
alterPageAssignment now reuses actionExprV3, the same rule CREATE PAGE uses,
and the executor builds through the CREATE PAGE builder
(pageBuilder.buildClientActionV3) rather than a second switch. Every action
form is therefore available here the day it is available there — including
the combined SAVE_CHANGES CLOSE_PAGE, which the reporter of the originating
findings had worked around with a bespoke microflow.
That delegation is the point. mendixlabs#855 is the identical bug for DataSource,
filed separately: SET carried a narrower vocabulary than REPLACE, so each
missing case surfaced as its own report. Enumerating cases in SET guarantees
a next one; delegating means the pair cannot drift. (mendixlabs#855 itself is already
fixed on main.)
SetWidgetAction refuses a widget with no Action property rather than writing
one. Studio Pro resolves every stored property against the type's property
list and throws on one it does not know, while mxbuild's deserializer
tolerates it — so a silent write would build clean and fail to open. The
MCP backend refuses the op outright: the pg LightPage does not model widget
actions, and per ADR-0005 an op the storage cannot express is an error, not
a no-op reporting success.
Verified on Mendix 11.13.0: mx check 0 errors, and Class survives four
successive retargets without ever being restated. Causation proven by
reverting grammar+visitor together and reproducing the original parse error
— reverting the grammar alone yields a compile error, not a test failure,
because the visitor references ctx.ActionExprV3().
Two pre-existing behaviours confirmed as not caused by this change, each
against a CREATE PAGE control: a SHOW_PAGE action auto-binds $currentObject
and reports CE1571 outside a dataview, and OPEN_LINK (LinkClientAction) is
not yet serializable by the modelsdk codec. CREATE PAGE reproduces both
identically.
Repro: mdl-examples/bug-tests/alter-page-set-action.mdl
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LUToAkUx54bNkNjsBpufRH
…hree real built-ins
`mxcli check` reported MDL044 for a call to a name Mendix has no expression
function for, but the rule was not on the exec allowlist, so `mxcli exec` wrote
the microflow anyway and the failure surfaced only from mxbuild:
[error] [CE0117] "Error(s) in expression."
at Create variable activity 'Create Boolean variable'
Same shape as mendixlabs#833 — a rule that fires in `check` and is unreachable from exec —
so the fix is the same: add it to execEnforcedMicroflowRules.
Promoting it was not safe as it stood. MDL044's sole allow-list is exprcheck's
funcTable, and the table was missing three genuine Mendix built-ins — isNew,
isSynced and isSyncing (the object-state "special checks"). While the rule only
reported, that was a nuisance; as a write barrier it would have refused valid
MDL, which is the MDL009 trap that kept the mendixlabs#833 promotion narrow. Each of the
three was built against mxbuild 11.13.0 and came back at 0 errors before being
added.
The control that keeps the table honest: `trunc` looks like a sibling of
round/floor/ceil and was even listed in roundingFuncs, but Mendix has no such
function — `trunc($D)` is CE0117 on 11.13.0. It stays out of funcTable and is
removed from roundingFuncs, where its presence invited "fixing" the MDL044
report by adding it.
Verified against mxbuild 11.13.0 with a control binary: with MDL044 off the
allowlist, exec writes the microflow and `mx check` reports 1 error (CE0117);
with it on, exec refuses and the project stays at 0 errors.
Fixes mendixlabs#828
A wrong `icon:` reference passed `mxcli check`, including --references, and
first surfaced as a build error:
[error] [CE1613] "The selected custom icon
'Atlas_Core.Atlas_Filled.no-such-icon' no longer exists." at Action
button 'btnBad'
Nothing resolved the reference — the name was written straight through to
BSON. Icon-collection lookup existed only for `show`/`describe`, never in a
validation path.
`mxcli check … -p app.mpr --references` now indexes the project's icon
collections once per run and resolves every reference: page and snippet
widget trees (CREATE PAGE, and ALTER PAGE's SET/INSERT/REPLACE) plus
navigation menus, including sub-items.
This belongs in the --references pass rather than the no-project one: icon
collections are documents in the project (a blank 11.13 app ships three,
~770 icons), so unlike the cross-module GRANT check there is genuinely
nothing to resolve against without -p.
The two failures are reported differently because they need different
fixes. An unknown icon in a known collection gets near-match suggestions
and a pointer to `describe icon collection <name>`; an unknown collection
gets the list of collections the project actually has, since that is where
the typo usually is.
False positives are the real risk in a new check rule, so the valid cases
were swept before shipping: all nine mdl-examples scripts that use icons
pass unchanged, as do valid references alongside the rejected ones in the
same page.
The repro carries only valid icons, deliberately. `make check-mdl` runs
`mxcli check` with no project, so a bad-icon script would pass there and be
reported as a negative test unexpectedly passing — the rejection cases are
unit-tested instead.
Verified on Mendix 11.13.0: the bad reference is reported before any write,
and the valid script builds with 0 errors. Causation proven by unwiring the
call and watching the reported symptom return ("Check passed!" on a script
with a bad icon).
Repro: mdl-examples/bug-tests/icon-reference-validation.mdl
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LUToAkUx54bNkNjsBpufRH
The empty-repo seed prompt had grown to ~180 lines of pasted text: the interview, the provisioning steps, the multi-app deltas and the model proposal all inline. That is awkward to paste from a phone (which is the flow it exists for), and every fix to the procedure required everyone to re-copy a new prompt. Move the whole procedure into a new `bootstrap-app` skill, which ships embedded in the mxcli binary. The prompt is now three steps: install mxcli, `mxcli init --sync-skills` (works in an empty repo — it returns before init's .mpr lookup, so no project is needed), read the skill and follow it. 179 lines of paste become 26, and the procedure is versioned with the binary rather than with whatever text someone copied months ago. The mxcli download stays in the prompt: it is the one step that cannot live in the skill, since the skill is inside the binary. Docs page keeps the rationale and now summarises what the skill does; step-number references updated to point at the skill's steps.
…ty, and author the drop-down filter's association mode
Two unrelated defects behind one report.
1. `column colCustomer (attribute: Order_Customer)` qualified the association
like an attribute and wrote a dangling AttributeRef, so the build failed
[error] [CE1613] "The selected attribute 'ZKT38.Order.Order_Customer' no
longer exists." at Columns (1/1) of data grid 2
This is a refusal, not a fix, because the reference is not representable.
`CustomWidgets$WidgetValue.AttributeRef` is typed `AttributeRef`, not the
polymorphic `MemberRef`: hand-patching a `DomainModels$AssociationRef` into
that slot makes the project unloadable ("Object of type 'AssociationRef'
cannot be converted to type 'AttributeRef'"), and the assembly defining the
type carries no association-valued member at all. The column property's
<associationTypes> is permission for the attribute PATH to traverse a
reference — `attribute: Assoc/Attr`, which already worked — not to bind one.
The refusal names both working forms.
2. The drop-down filter's association mode was unauthorable: every `baseType:
'ref'` property was unmapped in dropdownfilter.def.json, so `check` reported
MDL-WIDGET01 "has no property refEntity" and exec dropped them. It is now a
def.json mode entered by giving the filter a `datasource:` (the option list),
mirroring the ComboBox's association mode, with a DESCRIBE reader so a
describe→edit→exec cycle does not silently revert it to attribute mode.
Verified on mxbuild 11.13.0. `mx dump-mpr` reads back baseType="ref",
refEntity=IndirectEntityRef{steps:[Order_Customer → Customer]},
refOptions=XPathSource{Customer}, refCaption=AttributeRef{Customer.Name}.
Proven in a browser, because 0 errors from `mx check` cannot tell a written
property from a dropped one:
build mx check running app
fixed 0 errors options are Acme/Globex; picking Acme
narrows the grid from 3 rows to 2
pre-fix 1 error (CE0463) project cannot be deployed
Fixes mendixlabs#830
docs: shrink the bootstrap prompt to a hand-off into a skill
Add ALTER PAGE ... SET Action — retarget a widget's action in place
Add icon-collection reference validation to `mxcli check` (CE1613)
… in DESCRIBE
Filed as "line geometry is not exposed in DESCRIBE/CREATE ASSOCIATION".
Investigating turned up something worse than the missing feature: mxcli was not
merely omitting the anchors, it was destroying them.
`DomainModels$Association` stores `ParentConnection`/`ChildConnection` — where
the connector attaches to the FROM and TO entity boxes — as the string "x;y".
Both writers hardcoded "0;50"/"100;50" and neither parser read them back, so
because the writers rebuild the whole element, any association write reset them.
Measured on a blank Mendix 11.13 app, whose Studio-Pro-authored
Administration.AccountPasswordData_Account stores 0;54 / 100;54:
statement before after
alter entity … add attribute (unrelated) 0;54/100;54 0;54/100;54
alter association … set comment 0;54/100;54 0;50/100;50
create or modify association … 0;54/100;54 0;50/100;50
…same, MXCLI_ENGINE=legacy 0;54/100;54 0;50/100;50
Now read and written back verbatim on both engines (guard-don't-drop, ADR-0005);
the hardcoded pair applies only when nothing is stored. The field is a pointer
because {0,0} is a real anchor (top-left) and cannot double as "unset".
Two properties of the format, verified by hand-patching a project and running
`mx check` on 11.13.0: both components must be integers ("0.5;50" dies at LOAD
with StorageLoadException), and there is no range validation ("0;500", "-20;50"
load with 0 errors), so out-of-range values round-trip untouched.
DESCRIBE reports a non-default pair as a comment above the statement rather than
a clause. DESCRIBE output must stay re-executable, and MDL has no anchor syntax
yet — settling that needs an answer we do not have: whether the two integers are
percentages of the entity box or pixels. Every observed value has x in {0, 100},
which reads as percentages, but that is a hypothesis; shipping
`@anchor(parent: bottom-left, …)` on top of it would bake in a vocabulary the
storage does not have, since the values are a continuous pair, not 8 anchors.
Not applicable to cross-module associations: `DomainModels$CrossAssociation` has
no connection properties, and writing them there crashes Studio Pro (#50).
Refs mendixlabs#872
…, measured
The previous commit called the coordinate system a hypothesis and said "every
observed value has x in {0, 100}". The wider sample falsifies the second claim
and settles the first.
Measured across 88 coordinate pairs in four Studio-Pro-authored sources — a
blank 11.13 app plus the Advanced Audit Trail Core 4.0.0, Email Connector 6.4.2
and Workflow Commons 4.11.0 marketplace modules:
x range 0..100, y range 0..100, nothing outside it
85 of 88 pairs pin one coordinate to exactly 0 or 100 while the other varies
— "which edge, and how far along it"; the 3 exceptions are all y=99
distinct x: 0 9 11 17 18 47 49 50 65 77 78 84 87 100
distinct y: 0 16 19 33 38 51 52 53 54 59 60 67 69 76 85 99 100
So x is NOT always an edge, and the units are percentages of the entity box.
Pixels is ruled out by the model rather than by the values: DomainModels$EntityImpl
stores only Location (canvas pixels) and NO size, so the box is dimensioned by
the editor from the name and attribute list — a pixel anchor would have nothing
to measure against and would drift as attributes are added.
None of this changes the code's behaviour: preservation never needed the unit.
It does change the syntax question, and in the same direction — a continuous
pair spanning 14 distinct x values cannot be spelled as eight named anchors, so
the issue's proposed @anchor(parent: bottom-left, ...) is out either way.
Also records the method, since it generalises: `mxcli marketplace download <id>`
is a source of real Studio-Pro-authored models, and a module .mpk carries either
a raw BSON project.mpr or an MPR v1 SQLite one.
Refs mendixlabs#872
…dinates Closes the third slice of mendixlabs#872. The anchors were already preserved and reported; now they can be written, so a scripted domain model can lay out its own connector lines instead of only its entity boxes. @anchor(from: (0, 54), to: (100, 54)) create association Sales.Order_Customer from Sales.Order to Sales.Customer; alter association Sales.Order_Customer set anchor from (50, 100) to (50, 0); x and y are a percentage of the entity box, 0..100 — the units established by measurement in the previous commit. `from` is the anchor on the FROM entity's box, `to` on the TO entity's, matching the words the statement already uses for its two ends. CREATE needed NO grammar change. `@anchor(from:, to:)` already existed for microflow sequence flows, asking the same question and differing only in the value type; `annotation*` is generic across every CREATE statement, `annotationParamName` already admitted FROM and TO, and `(x, y)` is the existing `annotationParenValue` with two positional values. The two forms cannot be confused: the microflow one names its inner params (`(from: right, to: left)`), a coordinate pair is positional. The only new grammar is ALTER's SET ANCHOR, mirroring `alter entity ... set position (x, y)`. Coordinates rather than the named anchors the issue proposed, because the stored pair is continuous: 14 distinct x values across the 88 measured samples, and mxcli's own default 0;50 differs from Studio Pro's 0;54 by four points. Naming an end SETS it; not naming one PRESERVES what is stored. That asymmetry is what keeps the preservation fix intact through the new syntax — a `create or modify association` about the delete behaviour must not flatten a hand-tuned line — and it is why the AST carries pointers, so "not mentioned" and "mentioned as (0, 0)" stay distinguishable. A fractional coordinate is refused at check time with an explanation. Mendix stores two integers and its loader will not OPEN a project whose anchor is fractional, so truncating 0.5 to 0 would write a value nobody asked for into a file that still loads — the worse of the two failures. DESCRIBE now emits the annotation instead of the placeholder comment, closing the round trip. Verified on mxbuild 11.13.0: authored 11;99 / 9;0 stores exactly that at 0 errors, describe re-emits it, and re-executing that output into a fresh project reproduces the same pair. Refs mendixlabs#872
fix: MDL044 as a write barrier (mendixlabs#828), datagrid association binding/filtering (mendixlabs#830), association line anchors (mendixlabs#872)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Build-breakers — mxcli reported success, mxbuild disagreed
New capability — drop-down filter association mode (#830), ALTER PAGE … SET Action, anchor authoring syntax (#872).
Integrity — every mxcli syntax example now held to actually parsing (26 of 120 didn’t); bootstrap prompt reduced to a skill hand-off.
The body also carries the anchor coordinate-system measurement (88 pairs, four Studio Pro sources) and the verification note: pre-fix control binaries where the claim is causal, and a browser assertion for the filter, since 0 errors from mx check can’t distinguish a written widget property from a dropped one.