Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .claude/skills/fix-issue.md
Original file line number Diff line number Diff line change
Expand Up @@ -364,6 +364,8 @@ cases for these three BSON types — they fell to `default: return nil`.
| The SessionStart hook `mxcli init` writes cannot survive an idle reap: it is guarded on `test -x ./mxcli`, and `.gitignore` excludes that binary (~85 MB) on purpose. The container is reclaimed, the repo re-cloned without it, the guard fails and the hook no-ops through `\|\| true` — the next session has no mxcli, no MxBuild cache, no database, and no message saying so | The hook inlined the whole bring-up in one shell line, so "binary missing" could only be expressed as "skip". A hook line cannot reasonably do OS/arch detection and a download; something committed has to | `cmd/mxcli/init_hook.go` (`bootstrapScriptTemplate`, `writeBootstrapScript`, `sessionStartHookCommand`, `sessionStartHookMarkers`) | Emit a committed `.claude/bootstrap-mxcli.sh` that resolves OS/arch, fetches the binary when absent (`MXCLI_TAG` to pin), then runs the setup; the hook becomes `sh .claude/bootstrap-mxcli.sh \|\| true`. **Changing the hook command breaks dedupe**, which matched on the old command string — so `addSessionStartHook` now recognises *any* known marker and **rewrites the entry in place**, migrating an old project instead of leaving it with two hooks that both run. **Generalisable**: a guard whose condition is something you deliberately do not commit is a silent no-op waiting for a fresh clone — make the guard able to satisfy itself. Verified by reproducing the reap: moved `./mxcli` out of the project, ran the hook command verbatim, watched it re-download (88 MB, new mtime) and finish with "Setup complete … database ready". Tests `TestAddSessionStartHook_MigratesLegacyCommand`, `TestEnsureSessionStartHook_WritesFile`. mxcli-todo #2 |
| `mxcli exec -p app.mpr - <<'EOF' … EOF` fails with `Error reading file: open -: no such file or directory` — `-` is taken literally as a filename, so MDL cannot be piped or written as a heredoc and every ad-hoc script needs a temp file first | `exec` (and `check`) called `os.ReadFile(path)` directly, with no case for the conventional stdin spelling | `cmd/mxcli/mdlsource.go` (new `readMDLSource`, `mdlSourceLabel`), `cmd/mxcli/cmd_exec.go`, `cmd/mxcli/cmd_check.go` | One helper both commands share, so `check` gained the same spelling rather than only the reported one; `check` reports the source as `<stdin>` instead of a bare `-`. Verified live: a heredoc through `exec` and a pipe through `check` both run. Tests `cmd/mxcli/mdlsource_test.go`. mxcli-todo #5 |
| `mxcli syntax` documents spellings the parser rejects, so an agent following the reference writes MDL that fails — `TEXTBOX … (Binds: Attr)` ("'Binds:' is no longer supported, use 'Attribute:' instead") and `DataSource: MICROFLOW Module.MF()` (a zero-arg microflow datasource takes NO parens, unlike RETRIEVE/CALL) | Nothing checks the `syntax` corpus against the parser. `make check-skill-mdl` validates MDL blocks in the skills and the docs site, but the `Syntax`/`Example` strings in `cmd/mxcli/syntax/*.go` are not covered, so a retired spelling can sit there indefinitely | `cmd/mxcli/syntax/features_page.go` (10 × `Binds:` → `Attribute:`, the datasource parens), `cmd/mxcli/syntax/retired_spellings_test.go` (new guard) | Fix the text **and** pin it: a table-driven test fails if a retired spelling reappears in any topic's Syntax or Example. It is a spelling guard rather than a parse — the snippets are fragments (a DATAVIEW body, a property line) that do not stand alone as statements, so they cannot just be fed to the parser. Proven by reintroducing `Binds:` and watching the test name the topic and field. **A third claim in the same report did not reproduce**: `CONTAINER (OnClick: SHOW_PAGE M.P(Param: $currentObject))` parses fine on current main, so only the two verified ones were changed. mxcli-todo #8 |
| `mxcli test … --local` (or any other `StartLocalApp` caller) fails with MxBuild's `the project file path should be an absolute path`, followed by a page of Windows sample requests, whenever `-p` is given a **relative** path | `ServeServer.Build` forwarded `ProjectFilePath` verbatim. `mxcli run` had learned to absolutize at the CLI layer (findings #17), but that fix lived in `cmd_run.go`, not in the code that talks to MxBuild — so the next caller re-hit it | `cmd/mxcli/docker/mxserve.go` (`ServeServer.Build`) | Absolutize `req.ProjectFilePath` in `Build` itself, the single place that talks to MxBuild, so no future caller can miss it; also resolve `LocalAppOptions.ProjectPath` in `applyDefaults` so `DeployDir` and the runtime log path are not derived from a relative value. Test by pointing a `ServeServer` at an `httptest` fake and asserting on the request body — the CLI-layer fix cannot be tested that way, which is part of why it did not generalise |
| `mxcli test --attach` fails with `reload_model failed: Authentication failed.` — after the test microflows have already been injected into the project | The M2EE admin API and the test endpoint are **different secrets**. `attach` built its `RuntimeController` with `M2EEOptions{Token: hs.Token}` — the endpoint token — instead of the runtime's admin password | `cmd/mxcli/testrunner/runner_attach.go` (`attach`), `cmd/mxcli/testrunner/handshake.go` (`Handshake`) | Carry `AdminPass` in the handshake alongside `Token` and pass that to `M2EEOptions`. The hosting `run --local` publishes it via `docker.LocalAppInfo` (the resolved value, not the package default, so a `--admin-pass` override still works). Whenever one process drives another's M2EE API, check which credential is being passed — `defaultLocalAdminPass` and any app-level token are unrelated |
| `alter page … set Editable = [expr]` (or `set Visible`) writes a project Studio Pro refuses to open: `StorageLoadException: Conditional editability settings has an invalid value '' for property Attribute`. `mxcli check` ✓ and `mx check` ✓ — neither inspects the stored value. The identical settings written by `create page` load fine | The ALTER path builds the `Forms$Conditional{Visibility,Editability}Settings` node by hand and wrote `Attribute: null`. `Attribute` is a **BY_NAME** `AttributeIdentifier`, so its unset value is the empty string, not null — exactly what the CREATE path already encodes via `codec.RegisterTypeDefaults(..., EmptyStringFields: []string{"Attribute"})`, whose comment records this same StorageLoadException from #627. Only the hand-built ALTER node missed it | `mdl/backend/pagemutator/mutator.go` (`setWidgetConditionalSettingMut`) | Write `{Key: "Attribute", Value: ""}`, not `nil`. **General rule: when one path hand-builds BSON that another path builds through the codec, diff the two encodings rather than eyeballing the hand-built one** — `mxcli bson dump --type page --object M.P` on a CREATE-authored and an ALTER-authored widget makes the divergence a one-line diff (key sets and values were otherwise identical). `SourceVariable` stays `nil`: it is BY_ID, where null *is* the absent value, so "null is wrong" is per-field, not a blanket rule. Test `TestSetWidgetConditionalSetting_AttributeIsEmptyString`; repro `mdl-examples/bug-tests/851-alter-page-conditional-attribute.mdl`. Issue #851 |
| A widget conditional using a function whose name is also an MDL lexer keyword — `visible: [trim($currentObject/Slug) != '']`, `[length(…) > 0]`, `empty`/`count`/`find` — **silently drops the whole property**; `mxcli check` ✓, `mx check` ✓, and the widget renders unconditionally visible. `toUpperCase`/`isMatch`/`contains` in the same position work | `xpathFunctionName` (MDLPage.g4) enumerated only `IDENTIFIER \| HYPHENATED_ID \| NOT \| TRUE \| FALSE \| CONTAINS`, so `trim(` never matched `xpathFunctionCall`. The enclosing `[...]` then failed to parse as an `xpathConstraint` and matched the generic `propertyValueV3` alternative instead, so the visitor set `Visible` (an array) rather than `VisibleIf`, and the builder's `else if pages.StaticVisibleExpression(...)` — which reads only bool/string — never fired | `mdl/grammar/domains/MDLPage.g4` (`xpathFunctionName`) + `mdl/executor/validate_widgets.go` (`validateConsumableConditional`) | Define `xpathFunctionName : xpathWord \| NOT` — `xpathWord` is a negated token set, so it self-maintains as the lexer gains keywords; an enumerated list reacquires this bug with the next promoted function name. Safe because `xpathFunctionCall` requires a following LPAREN and no `xpathStepValue` may be followed by one, so bare `empty` still parses as a path word. `NOT` is spelled out (xpathWord excludes it). **Also add the general guard**: MDL-WIDGET19 errors when `Visible`/`Editable` holds a value that is neither routed to `VisibleIf`/`EditableIf` nor a bool/string — that is the residue signature of any conditional the visitor could not build, so the next one fails loudly instead of vanishing. `make grammar` regenerates the parser (not committed). **Verify in a browser, not at `mx check`** — a dropped property is still a valid model, so `mx check` reports 0 errors before AND after; the symptom only exists at render time (see `verify-in-runtime.md`). The repro script carries a `Bug852.Verify` page for this: `Slug` is three spaces, so `trim()` changes the outcome and a dropped `Visible` renders (Mendix defaults to visible). Pre-fix all 5 markers render; post-fix only the 3 that should. **One rule, two contexts**: `xpathConstraint` serves both `Visible:`/`Editable:` (a Mendix *client expression* — trim/length/toUpperCase/find) and a datasource `where` (real *XPath* — contains/starts-with/ends-with/string-length/not, `length()` = list length, aggregates Java-only, and `empty`/`NULL` are KEYWORDS not calls). The sets differ, so the grammar must not enumerate either; mxbuild adjudicates. Regression-test the XPath side when touching this rule — `[Name = empty]`, `[Name = NULL]`, `not()`, `contains()`, `starts-with()`, `string-length()` all still parse and `mx check` clean. Tests `TestConditionalVisibility_KeywordFunctionNames`, `TestValidateStaticWidget_UnconsumableConditional`; repro `mdl-examples/bug-tests/852-conditional-keyword-functions.mdl`. Issue #852 |
| `download file $Doc;` is accepted by `mxcli check` and `mxcli exec` ("Created microflow") but the activity lands with **no action at all** — `describe` renders `-- Empty action` and `mx check` fails `[CE0008] "No action defined."`. Same for `download file $Doc show in browser;` | `microflowActionToGen` (the modelsdk write path) had no `*microflows.DownloadFileAction` case, so it hit `default: return nil` and the enclosing ActionActivity was serialized with a nil Action. Grammar, visitor, flow builder, read path and DESCRIBE formatter were all already in place, so the statement passed every stage that reports anything and vanished at the one that does not | `mdl/backend/modelsdk/microflow_write.go` (`microflowActionToGen`) | Add the case, setting `FileDocumentVariableName`, `ShowFileInBrowser` and `ErrorHandlingType` (Rollback default). **The storage key is `ShowFileInBrowser`, not `ShowInBrowser`** — the gen setter binds the right one; legacy's `parseDownloadFileAction` reads the wrong key. **Test at the round trip, not the reader**: a reader-only test starts from BSON the writer never had to produce, so `TestActionFromGen_DownloadFile` was green throughout. `roundTripMicroflow` (model→gen→codec→model) is the harness; assert the ActionActivity's `Action` is non-nil, which is the CE0008 shape itself. This is the same silent-drop mechanism as the `microflowObjectToGen` default branch (#791) — when auditing, diff the write switch's cases against `sdk/mpr/writer_microflow_actions.go`. Test `TestMicroflowRoundTrip_DownloadFile`; repro `mdl-examples/bug-tests/850-download-file-action.mdl`. Issue #850 |
Expand Down
1 change: 1 addition & 0 deletions .claude/skills/mendix/run-local.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,7 @@ Launch `run --local` as the **sole** command in its invocation (don't chain a tr
| `--screenshot-path` / `--screenshot-url` | `.mxcli/run-local.png` / app root | Screenshot output / page (URL or `/path`) |
| `--screenshot-user` / `--screenshot-password` | — | Log in once, reuse session (pages behind login) |
| `--runtime-log` | `.mxcli/runtime.log` | Runtime log file: JVM stdout/stderr **and** the application log (microflow `LOG` output + server stack traces, via an attached file log subscriber). `-` disables. |
| `--test-endpoint` | off | Host mxcli's token-guarded test endpoint so `mxcli test … --attach` can run a suite against this app with no boot of its own. Installed **before** the boot (the handler registers from after-startup), your own after-startup microflow is chained not displaced, and both are removed on exit. See `test-microflows.md`. |
| `--debug` | off | Enable the microflow debugger at boot + start a session, so `mxcli debug break/paused/…` works from another terminal (see `debug-microflows.md`). No breakpoints = no behaviour change; disabled on shutdown. |
| `--debug-pass` | `mxdebug` | Debugger password when `--debug` is set |
| `--metrics` | off | Register a Prometheus meter registry at boot; the runtime serves metrics at `http://127.0.0.1:<admin-port>/prometheus` |
Expand Down
120 changes: 116 additions & 4 deletions .claude/skills/mendix/test-microflows.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,20 +121,132 @@ mxcli test tests/ -p app.mpr --verbose

## How It Works

The test runner uses the **after-startup microflow** pattern:
There are two mechanisms. `--local` uses the **test endpoint**; Docker uses the
older **after-startup microflow** pattern.

### `--local`: the test endpoint

1. Parses test files and extracts test blocks with annotations
2. Records the project's current after-startup microflow, and whether an `MxTest`
module already exists
3. Generates **one `MxTest.Test_<id>` microflow per test**, plus a Java action
that registers an HTTP endpoint, and points after-startup at a microflow whose
only job is to call it — **no test runs during startup**
4. Builds and boots the app once
5. Invokes each test by name over HTTP; each returns its own verdict in the
response
6. Restores the original after-startup setting and removes everything generated
7. Outputs results (console, JUnit XML)

Two consequences worth knowing when reading a failing run:

- **A test that throws fails only itself.** It is reported as `ERROR` with the
root-cause message, and the next test still runs. Under the after-startup
mechanism an uncaught error ends the whole flow — and because that flow *is*
the startup action, it also fails the boot.
- **Results are returned, not scraped**, so a test cannot be lost to log
buffering or a runtime that stopped echoing to the console.

Each test is a separate microflow with its own variable scope, so `$result` in
one test never collides with `$result` in another.

#### `--watch`: keep the runtime warm

```bash
mxcli test tests/ -p app.mpr --local --watch
```

The first run pays the cold boot; after that the runtime and the build server
stay up, and the suite re-runs on every change — to a test file **or** to the
project's model. Measured on an 11.13.0 app:

| | |
|---|---|
| First run (cold boot) | ~30s |
| Edit a test → verdict on screen | **~2s** |
| Edit a microflow → verdict on screen | **~2s** |
| The tests themselves | 20–70ms |

Editing a microflow and seeing straight away whether it still passes is the loop
this exists for. Ctrl-C stops watching and restores the project — the shutdown
prints `project restored` when it has.

Adding, editing and deleting tests all work mid-session: the suite is re-parsed
on every change, and a deleted test's microflow is dropped rather than left
behind reporting a stale pass.

`--watch` requires `--local`. The Docker and `--legacy-runner` paths can only
re-run tests by restarting, which is the thing being avoided.

#### `--attach`: no boot at all

If you already have the app running, tests can skip the boot entirely. The dev
loop has to opt into hosting the endpoint, because the handler is registered by
the after-startup microflow and so cannot be added to an app that is already up:

```bash
# terminal 1 — the app you are working in
mxcli run --local --test-endpoint -p app.mpr

# terminal 2 — runs in ~2s, no boot, repeatable
mxcli test tests/ -p app.mpr --attach
mxcli test tests/ -p app.mpr --attach --watch # ...and re-run on every change
```

The hosting app chains your project's own after-startup microflow rather than
displacing it, so it still boots normally. The endpoint and the handshake file
(`.mxcli/test-endpoint.json`, mode 0600) are removed when the app stops.

Three things to know before reaching for it:

- **Tests run against the running app's database**, not a scratch one, so they
can leave data behind in the app you are looking at. `--local` uses a separate
`<project>_test` database; `--attach` does not.
- **An attach only owns its own test microflows.** The endpoint and the
after-startup setting belong to the app hosting them, and cleanup never
touches them.
- **A change needing a runtime restart is refused** — a new entity or
association. That runtime belongs to the other process. Restart it, or drop
`--attach`.

| | Boot | Database | Owns the runtime |
|---|---|---|---|
| `--local` | ~30s each run | `<project>_test` | yes |
| `--local --watch` | ~30s once, then ~2s | `<project>_test` | yes |
| `--attach` | none | the running app's | no |

#### Security of the endpoint

It executes microflows under a system context, so it is gated four ways:

| Guard | Behaviour |
|---|---|
| No `MXCLI_TEST_TOKEN` in the runtime's environment | The handler is **not registered at all** (404) |
| Missing or wrong `X-MxTest-Token` header | 401, compared in constant time |
| Non-loopback caller | 403 |
| `mf` outside `MxTest.Test_*` | 403 — it is not a general microflow-invocation API |

The token is generated per run and reaches the runtime through its **environment**,
never written into the project. Combined with fail-closed registration, that means
a project which kept the `MxTest` module through a failed cleanup exposes nothing
when deployed anywhere else.

### Docker: the after-startup microflow

1. Parses test files and extracts test blocks with annotations
2. Records the project's current after-startup microflow, and whether an `MxTest`
module already exists
3. Generates a `MxTest.TestRunner` microflow with assertion logic and points
after-startup at it
4. Builds the project and restarts the runtime (Docker, or local with `--local`)
3. Generates a single `MxTest.TestRunner` microflow containing every test, and
points after-startup at it
4. Builds the project and restarts the container
5. Captures structured `MXTEST:` log lines for pass/fail
6. Restores the original after-startup setting and removes the generated runner —
the whole `MxTest` module when the runner created it, otherwise just the
`TestRunner` microflow
7. Outputs results (console, JUnit XML)

### Both mechanisms

The project's **Security Level is not modified**. The after-startup microflow runs
in an administrative context and is not subject to it, and forcing it off breaks
projects whose published REST/OData services use custom authentication. If a
Expand Down
Loading
Loading