OData publishing, external-entity imports, dynamic SQL and folders: nine formula1 findings

.claude/skills/fix-issue.md

@@ -405,3 +405,11 @@ extracting `OffsetExpression`/`LimitExpression`.
 | A suite passes under `mxcli test --attach` and fails under `--local`, with assertions that depend on startup state (a loaded cache, seeded reference data) seeing zero rows | The `--local` runner pointed after-startup at its own registration microflow and did **not** chain the project's own, so the app's startup logic never ran. It was a deliberate choice (a known baseline) but was invisible: the run printed only `After-startup set to MxTest.RegisterEndpoint`, never that the user's microflow had been displaced | `cmd/mxcli/testrunner/runner.go` (`runEndpoint`), `cmd/mxcli/testrunner/cleanup_strategy.go` (`describeStartup`) | Capture project state **before** generating the endpoint MDL, and pass `state.afterStartup` to `GenerateEndpointMDL` so the generated flow chains it — the hosted `--test-endpoint` path already did this, and the mismatch between the two was the bug. Add `--skip-app-startup` for a deterministic empty baseline, and always print which of the two happened. Note the startup microflow's writes run at boot, outside any test transaction, so `@cleanup rollback` does not undo them. mxcli-formula1 findings #19 |
 | `mxcli test tests/ -p app/App.mpr --list` fails with `stat tests/: no such file or directory` while the same command without `--list` runs fine | The `--list` branch passed raw `args` to `ListTests`, bypassing `resolveTestPaths` — so a path relative to the project (rather than the working directory) resolved for execution but not for listing | `cmd/mxcli/cmd_test_run.go` (the `if list` branch) | Pass `resolveTestPaths(args, projectPath)` there too. When a command has two entry points into the same input, check both go through the same path resolution. mxcli-formula1 findings #15 |
 | After `SET` became optional, `mx check` on a project built from `02-microflow-examples.mdl` reports six errors that no MDL change caused: `[CE0109] "Undefined variable 'ProductList.Price'."` at four Aggregate list activities, and `[CE0015] "Aggregate function must specify a valid attribute."` at the expression-based one. Identical on both engines. `mxcli check` on the same script is silent | Two conversions existed for one syntax. `$Sum = sum($List.Price)` used to reach the dedicated `aggregateListStatement` rule; making `SET` optional put `setStatement` — alternative 5 of ~50 — in front of it, so ANTLR matched the lower-numbered alternative and the statement fell through to `buildSetStatement`'s fallback conversion, which joined list and attribute into one name and dropped the per-item expression entirely. Underneath, `buildListAggregateAsFunction` never appended the expression argument, so the SET path could not have seen it either | `mdl/grammar/domains/MDLMicroflow.g4` (`setStatement` moved LAST in `microflowStatement`), `mdl/visitor/visitor_microflow_statements.go` (`buildSetAggregate` replaces `extractVariableAndAttribute`), `mdl/visitor/visitor_microflow_expression.go` (`buildListAggregateAsFunction` appends the expression argument) | **A permissive alternative belongs last.** `$X = <expr>` overlaps every `VARIABLE EQUALS <function-call>` statement in the rule — aggregates, list operations, RANGE — and ANTLR's ALL(*) picks the lowest-numbered alternative that matches, so a new general form silently steals from every specific one above it. **The measurement trap that let this ship**: the `SET?` change was swept with a control binary over every `mdl-examples/**/*.mdl` and found no difference — but with `mxcli check`, which parses and validates and never serializes. This defect lives between the AST and the BSON, where only `exec` + `mx check` can see it. Sweeping with `check` proves the grammar still *parses*; it proves nothing about what the visitor *builds*. For a grammar change, the control sweep must run the integration gate (`go test -tags integration -run TestMxCheck_DoctypeScripts`), not `check`. Fix proven by reverting both halves and watching the new tests fail with the reported symptom. Tests `mdl/visitor/visitor_microflow_aggregate_test.go`. Upstream CI on the ako→mendixlabs sync PR |
+| A published service will not build: every whole-number attribute is `[CE5016] "Attribute … has type Integer, but is published as Edm.Int32"`, and an exposed enumeration adds CE5016 plus `[CE4583] "Enumeration 'X' is not published in this service."` | `mendixAttrTypeToEdm` mapped Integer→Int32 (Mendix publishes it as **Int64**, same as Long), and the enum path wrote `Edm.String` while `EnumerationAsString` was hardcoded `false` — the one combination Mendix rejects, since with the flag false it wants the enumeration published as its own EDM enum type. The function's own comment flagged the unverified rows, and the existing unit test *pinned the wrong answer* | `mdl/executor/cmd_odata.go` (`mendixAttrTypeToEdm`, `enumPublishedAsString`, `publishedAttrType`), `model/types.go` (`PublishedMember.EnumerationAsString`), `mdl/backend/modelsdk/odata_write.go` + `sdk/mpr/writer_odata.go` (stop hardcoding the flag) | **Let mxbuild adjudicate the whole table at once**: publish one attribute of every Mendix type in one service and read the CE5016s off the build. That found Integer (reported) *and* Enumeration (only suspected), and confirmed String/Long/Decimal/Boolean/DateTime were already right — five verified rows for one build. Binary turns out to be unpublishable at all (CE5013), whatever type you give it. **A type and a flag that only work as a pair must travel as a pair** — `Edm.String` is ambiguous between String and a flattened enum, so the flag is the only thing distinguishing them and it belongs on the same struct. Watch for an existing test that encodes the bug: this one asserted `Edm.Int32`, so the fix *failed the suite* until the assertion was corrected. Tests `cmd_contract_test.go`, `cmd_odata_edm_type_test.go`. mxcli-formula1 #16 |
+| `create or modify external entity Mod.E (… Countable: false)` — touching only an entity-level property — detonates every attribute: `[CE6612] "Attribute 'circuitId' of external entity 'Stg_Circuit' is not supported."`, one per attribute, leaving a project that cannot build | Not the executor: it already preserves attributes it was not asked to change (`if len(attrs) > 0`). One layer down, `attributeFromGen` handled `StoredValue` and `OqlViewValue` but **not** `Rest$ODataMappedValue`, so every attribute of an external entity read back with no `RemoteName`, and the writer's `isExternal && a.RemoteName != ""` arm then fell through to a plain StoredValue on the next read-modify-write | `mdl/backend/modelsdk/domainmodel.go` (`attributeFromGen` gains the `ODataMappedValue` / `ODataMappedPrimitiveCollectionValue` arms) | **The attribute-level half of #782**, which fixed the entity level and stopped there — when a read-modify-write loses data, check every *nesting level* of the read, not just the one named in the report. A polymorphic `Value` switch that silently ignores a variant is the shape to look for: it compiles, it reads, and it drops. Reproduce with a **local metadata file** (`MetadataUrl: './contract.xml'`) — no server needed, and the import is the same code path. Also learned here: the reported attribute *rename* (`name` → `Stg_Circuitname`) is a different thing entirely — it happens at import, from `reservedEntityAttrNames`, and `name` is **not** actually reserved (verified: Mendix builds an external entity with an attribute literally named `name`). Tests `external_entity_read_test.go`. mxcli-formula1 #25 |
+| `execute database query … dynamic $Sql` reaches the runtime as the literal string `'$Sql'` — `Parser Error: syntax error at or near "$"` from the database, not from Mendix. Runtime-built SQL, and therefore query pushdown, is impossible | The builder quoted any dynamic query not already starting with a quote — right for `dynamic 'SELECT …'`, wrong for an expression — and the AST kept one `DynamicQuery` string whichever branch of the grammar produced it, so nothing downstream could tell them apart | `mdl/ast/ast_microflow.go` (`DynamicQueryIsExpression`), `mdl/visitor/visitor_microflow_actions.go` (set it in the `expr` branch), `mdl/executor/cmd_microflows_builder_calls.go` (`dynamicQueryExpression`) | **When a grammar has two alternatives that mean different things, the AST must record which one fired** — a shared field plus a "does it look quoted?" heuristic is a guess, and the workaround users find (`dynamic '' + $Sql`, which starts with a quote so the heuristic leaves it alone) is proof the heuristic is the bug. Verified by reading the stored BSON rather than by describe: `DynamicQuery\x00\x05\x00\x00\x00$Sql` — five bytes, no quotes. Tests `cmd_microflows_dynamic_query_test.go`. mxcli-formula1 #21 |
+| `create odata client` against a service behind `authentication basic` prints `Warning: could not fetch $metadata: … HTTP 401`, creates the client anyway, and the following `create external entities from …` imports nothing — from a script that reports success | The statement's `HttpUsername`/`HttpPassword`/`HEADERS` are stored for the runtime, but the design-time fetch was a bare `client.Get`. The fetch failure is only a warning, so the empty client propagates silently | `mdl/executor/cmd_odata.go` (`metadataFetchAuth`, `metadataAuthFromStmt`, `fetchODataMetadata`), `mdl/ast/ast_odata.go` + `mdl/visitor/visitor_odata.go` (`HttpUsernameIsLiteral` / `HeaderDef.ValueIsLiteral`) | **Only a literal is usable at design time.** The visitor strips a quoted literal's quotes, so `'f1api'` and `Module.ApiUser` both arrive as bare strings — the AST has to record which was written, or mxcli sends a *constant's name* as the password. Unresolved names are reported instead, which is also the honest answer: mxcli has no runtime to resolve a constant against. **A warning on a step something else silently depends on needs to say what breaks next** — the message now names the empty client and the import that will do nothing. Verified against a real basic-auth server that 401s without credentials and 403s without the custom header, so both had to arrive. Tests `cmd_odata_metadata_auth_test.go`. mxcli-formula1 #23 |
+| Re-running `create or modify odata service` after editing a `publish entity` block changes nothing — the served `$metadata` is identical, and only `drop odata service` + create picks the edit up | The modify branch updated the service's scalar properties and never touched `EntityTypes` / `EntitySets` | `mdl/executor/cmd_odata.go` (modify branch rebuilds published entities via `astEntityDefToModel`, and carries `AllowedModuleRoles` through) | **Replace, don't merge**: a member removed from the script has to leave the service, which merging cannot express — the script is the description of the service. **Carry through what the statement cannot express**: role grants come from a separate `grant access on odata service` and would otherwise be dropped by a modify (reported; *not* reproduced on 11.12.1 — kept as a guard, and the commit says so rather than claiming a fix). Verified: same script yields `Label as 'label'` before and `Label as 'label' (Filterable, Sortable)` after, build stays at 0 errors. Tests `cmd_odata_modify_members_test.go`. mxcli-formula1 #26 |
+| `create external entities from` a contract that restricts capabilities produces a project that will not build: `'Seasons' is marked Countable=False in the OData service, but True in the app`, `'latitude' is marked Filterable=False …` — one per restricted resource or property | Insert/Update/Delete restrictions were parsed; **Count/Filter/Sort were not**, so the import had nothing to honour and defaulted all three to true — on the one command whose entire job is fidelity to the contract | `mdl/types/edmx.go` (`EdmEntitySet.Countable`, `NonFilterableProperties`, `NonSortableProperties` + the three `applyCapabilityAnnotations` arms), `mdl/executor/cmd_contract.go` | An unannotated set still means countable/filterable/sortable — **silence in a contract is not a restriction**, it is OData's own default, so `nil` and `false` must stay distinguishable (`*bool`, as with the publish-side query options). The generated entity is compared against the contract at *build* time, so anything the contract can say is something the importer must be able to read. Tests `mdl/types/edmx_test.go`. mxcli-formula1 #24 |
+| A contract property called `name` is generated as `Stg_Drivername` / `Circuitname` — prefixed with the remote type. A page written against the published `$metadata` then fails with `The selected attribute 'F1Live.Drivers.name' no longer exists`, and the *same* field carries a different name in every module because the remote type names differ | `attrNameForOData` disambiguates any name in `reservedEntityAttrNames`, and `name` was on that list with the comment "Mendix system-managed attribute for the object name". It is not: Mendix builds an external entity with an attribute literally named `name` | `mdl/executor/cmd_contract.go` (`reservedEntityAttrNames` loses one entry; the import now reports the renames it does make) | **Test the whole list at once, not the reported entry.** One contract with a property per listed name, prefixing disabled, then `mx check`: CE7247 "The name 'x' is a reserved word" for `id`/`owner`/`changedBy`/`changedDate`/`createdDate`/`type`/`context`, and silence for `name`. That turns "is the list wrong?" into "which rows are wrong?" for the cost of a single build, and it *earns* the seven entries that stay rather than leaving them as folklore. Two existing tests pinned the old behaviour and had to be corrected — a hand-maintained list of platform rules will accrete guesses unless each row can point at an error code. **Migration**: a re-import renames the attribute back, so references to the prefixed name must follow. Tests `cmd_contract_reserved_test.go`. mxcli-formula1 #28 |
+| `MOVE JAVA ACTION …` / `MOVE ODATA SERVICE …` is a parse error (`no viable alternative at input 'MOVEJAVA'`), and neither `CREATE` form takes a folder clause — so those documents can never leave the module root from MDL | The `moveStatement` rule listed seven doctypes and nothing else; the missing ones were never unimplemented, just unlisted | `mdl/grammar/MDLParser.g4` (two alternatives), `mdl/ast/ast.go`, `mdl/visitor/visitor_entity.go` (dispatch **and** the MOVE FOLDER discriminator), `mdl/executor/cmd_move.go`, backend `MoveJavaAction` / `MovePublishedODataService`, `sdk/mpr` exports `MoveUnitByID` | Both reduce to the existing reparent primitive — a top-level document move is one containment row, so a new doctype is a list entry plus a lookup, not new machinery. **Watch the discriminator**: `MOVE FOLDER` is told apart from a document move by the *absence* of a doctype keyword, so every keyword added to the rule must also be added to that condition or a folder move starts parsing as a document move. **Verify placement by differential count, not by reading the model**: run the script with and without the MOVE lines and diff `select ContainmentName, count(*) from Unit` — three new Folders rows (a nested path creates two) and an unchanged Documents count says reparented rather than copied or dropped. Grepping blobs for names is a trap; stock modules are full of the same words. Tests `visitor_move_doctypes_test.go`, example in `18-folder-examples.mdl` — which must sit **before** that script's `drop module`, a mistake the integration gate caught and `mxcli check` did not. mxcli-formula1 #32 |

.claude/skills/mendix/manage-security.md

@@ -95,6 +95,10 @@ create module role MyModule.Admin description 'Full administrative access';
 create module role MyModule.User;
 create module role MyModule.Viewer description 'Read-only access';
 
+-- `or modify` updates an existing role's description instead of failing, so the
+-- whole security script stays re-runnable rather than needing a run-once file.
+create or modify module role MyModule.ApiUser description 'API consumer';
+
 -- Remove a module role
 drop module role MyModule.Viewer;
 ```

.claude/skills/mendix/organize-project.md

@@ -168,8 +168,17 @@ move page OldModule.CustomerPage to NewModule;
 | Nanoflow      | `folder 'path'` (keyword) | `move nanoflow ...` |
 | Snippet       | `folder: 'path'` (property) | `move snippet ...` |
 | Enumeration   | N/A | `move enumeration ...` |
+| Constant      | N/A | `move constant ...` |
+| Database connection | N/A | `move database connection ...` |
+| Java action   | N/A | `move java action ...` |
+| OData service (published) | N/A | `move odata service ...` |
 | Entity        | N/A | `move entity ...` (module only, no folders) |
 
+**Java actions and published OData services have no folder clause on `create`**, so
+`move` is the only way to place them — before this they were stuck at the module
+root forever. Both are plain document units, so the move is model-level only: it
+changes containment and nothing else.
+
 **Note:** Pages and snippets use property syntax (`folder: 'path'` inside parentheses). Microflows and nanoflows use keyword syntax (`folder 'path'` before `begin`). Entities are embedded in domain models and can only be moved to a different module (no folder support).
 
 ## Example: Reorganize a Module

cmd/mxcli/syntax/features_misc.go

@@ -211,6 +211,8 @@ SHOW STRUCTURE DEPTH 1 ALL;`,
 			"move folder", "drop folder",
 		},
 		Syntax: `MOVE <doctype> Module.Name TO FOLDER 'Path';
+-- doctype: PAGE | MICROFLOW | NANOFLOW | SNIPPET | ENUMERATION | CONSTANT
+--        | DATABASE CONNECTION | JAVA ACTION | ODATA SERVICE | ENTITY | FOLDER
 MOVE <doctype> Module.Name TO TargetModule;
 MOVE <doctype> OldModule.Name TO FOLDER 'Path' IN NewModule;
 MOVE FOLDER Module.FolderName TO FOLDER 'Path';
@@ -224,6 +226,11 @@ MOVE MICROFLOW MyModule.ACT_ProcessOrder TO FOLDER 'Orders/Processing';
 -- Move entity to different module
 MOVE ENTITY OldModule.Customer TO NewModule;
 
+-- Java actions and published OData services have no folder clause on CREATE,
+-- so MOVE is the only way to place them
+MOVE JAVA ACTION MyModule.ODataQuery TO FOLDER 'Support';
+MOVE ODATA SERVICE MyModule.PublicApi TO FOLDER 'Api/Published';
+
 -- Check impact before cross-module move
 SHOW IMPACT OF OldModule.CustomerPage;
 MOVE PAGE OldModule.CustomerPage TO NewModule;

cmd/mxcli/syntax/features_security.go

@@ -21,8 +21,8 @@ func init() {
 		Keywords: []string{
 			"module role", "create role", "drop role",
 		},
-		Syntax:  "CREATE MODULE ROLE <module>.<role> [DESCRIPTION '<text>'];\nDROP MODULE ROLE <module>.<role>;",
-		Example: "CREATE MODULE ROLE Shop.Admin DESCRIPTION 'Full access';\nCREATE MODULE ROLE Shop.User DESCRIPTION 'Read-only access';",
+		Syntax:  "CREATE [OR MODIFY] MODULE ROLE <module>.<role> [DESCRIPTION '<text>'];\nDROP MODULE ROLE <module>.<role>;",
+		Example: "CREATE MODULE ROLE Shop.Admin DESCRIPTION 'Full access';\n-- OR MODIFY makes a security script re-runnable:\nCREATE OR MODIFY MODULE ROLE Shop.User DESCRIPTION 'Read-only access';",
 		SeeAlso: []string{"security.user-role", "security.entity-access"},
 	})
 

docs/01-project/MDL_QUICK_REFERENCE.md

@@ -355,7 +355,7 @@ Nested folders use `/` separator: `'Parent/Child/Grandchild'`. Missing folders a
 | Show demo users | `show demo users;` | Configured demo users |
 | Show access on element | `show access on microflow\|nanoflow\|page\|entity Mod.Name;` | Which roles can access |
 | Show security matrix | `show security matrix [in module];` | Full access overview |
-| Create module role | `create module role Mod.Role [description 'text'];` | |
+| Create module role | `create [or modify] module role Mod.Role [description 'text'];` | `or modify` updates an existing role instead of failing, so a security script can be re-run |
 | Drop module role | `drop module role Mod.Role;` | |
 | Create user role | `create user role Name (Mod.Role, ...) [manage all roles];` | Aggregates module roles |
 | Alter user role | `alter user role Name add\|remove module roles (Mod.Role, ...);` | |