feat(messagedefs): authorable message definition collections - #380
Merged
Conversation
…llections First slice of #272's implementation (proposal in docs/11-proposals/PROPOSAL_authorable_message_definitions.md). Parses only — nothing reaches storage yet. A message definition is the source for 74 of the 327 mappings in the demo corpus (22.6%), and was the only one of a mapping's four schema sources a script could not create. Unlike an XML schema or a WSDL it holds nothing external: every element names an entity, an attribute or an association. Statements: create [or modify] message definition collection M.Name [folder '...'] ( ... ) drop | describe message definition collection M.Name show message definition collections [in M] alter message definition collection M.Name add|drop|rename definition ... alter message definition M.Collection.Definition add|drop|set member ... Two lexer notes, since a new keyword is how scripts break. MEMBER is new (only MEMBERS existed); ANTLR prefers the longest match so `members` still lexes as MEMBERS, and MEMBER is added to identifierOrKeyword so an attribute called `Member` still parses. COLLECTION already accepted singular and plural, so `show message definition collections` needs no new token. An association member spells out its target entity (Assoc/Module.Entity, the shape mappings already use). That is load-bearing rather than decorative: the stored MaxOccurs tracks the DIRECTION of traversal and not the association's type — measured, all 927 resolvable associations in the corpus are `Reference` yet 526 store 1 and 401 store -1 — so naming the target is what makes the direction explicit instead of something a reader has to work out. Refs #272 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Second slice of #272. The document can now be written; nothing calls it yet (the executor is next). model.MessageDefinitionCollection gains the metadata a CREATE OR MODIFY must preserve (Documentation/Excluded/ExportLevel) and its elements gain OriginalName plus, on an association node, the TARGET entity — both needed to rebuild or describe a node, since the stored MaxOccurs depends on the direction of traversal and cannot be recovered from the association alone. The modelsdk reader now carries all of them. The codec writer fills in everything derived, which is nearly the whole document. Measured across the corpus with no exceptions: MinOccurs 0, Nillable true, IsDefaultType false, MaxLength/FractionDigits/TotalDigits -1, all four text fields empty, ElementType from the node's kind, PrimitiveType Unknown for an object, Path from the position in the tree, MaxOccurs -1 on a definition root (56/56) and 1 on a value (3697/3697). The one it does NOT derive is an association's MaxOccurs, which the executor must resolve from the traversal direction. Authoring is modelsdk-only, as rules, menus and layouts are: the legacy writer has no serializer for the document and building one would duplicate a shape the codec already gets right. Reading works on both engines, and the legacy backend refuses with the established message. Refs #272 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ro document
ako/TestApp's OrderMessageDefinitions is hand-authored in Studio Pro rather
than shipped in a marketplace module, which makes it the right oracle for a
write path. Round-tripping it — read into the semantic model, re-encode,
compare against the STORED BYTES — found three things wrong, none of which the
corpus census had shown:
1. Path is a chain of ORIGINAL names, not exposed ones, and an ASSOCIATION
contributes TWO segments — its own name, then the target entity's:
Order|OrderLine_Order|OrderLine|Amount
^root ^association ^entity ^attribute
Confirmed afterwards at 4,707 of 4,707 elements across TestApp and the nine
demo apps.
2. The typed-array marker on MessageDefinitions and Children is 2; the codec
defaults to 3. Measured at 37 definition lists and 575 child lists, all 2.
3. Every exposed element serializes Children even when empty — a leaf
attribute stores the bare [2]. Same MandatoryLists rule as a rule document's
Flows and a custom handler's ParameterMappings.
Example is now carried rather than hardcoded empty. The corpus said empty in
4,686 of 4,686, which is why the proposal scoped it out — but TestApp's
hand-authored definition sets one, so it is author-set and rare rather than
unused, and hardcoding it would silently drop the one that exists.
The fixture also happens to carry the control the direction rule needs: the
SAME association, Mappings.Order_Customer, appears in both definitions with a
different MaxOccurs each time (1 from Order, -1 from Customer). Nothing
synthetic would have been as convincing.
Both round-trip controls verified: reverting the path rule reports 14 diffs,
reverting the marker reports 1.
Refs #272
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Fourth slice of #272. The feature works end to end. describe -> exec against ako/TestApp's hand-authored collection reproduces Studio Pro's document exactly: the executor reports "Unchanged", meaning canon.Reconcile found the rebuilt content semantically equal and elided the write. mxbuild 11.14.0 reports 0 errors on a project holding an mxcli-authored collection and an ALTERed one. The executor's job is resolution, and two resolutions need the domain model. Both were wrong first and were caught by the round trip: MaxOccurs on an association follows the DIRECTION of traversal, not the association's type. Holder is the FROM entity -> 1; holder is the TO entity -> -1. An association that connects the two entities in NEITHER direction is REFUSED rather than defaulted: a wrong cardinality exposes a list as a single object and builds cleanly, so refusing is the only honest answer. PrimitiveType is MAPPED, not passed through. Measured across 3,372 exposed attributes: Long -> Integer, AutoNumber -> Integer, Enumeration -> String, everything else identity. A pass-through gets 279 elements wrong, and the round trip caught it on TestApp's ProductId (a Long stored as Integer). Inherited attributes resolve to the entity that DECLARES them, reusing DeclaringMemberRef — 398 of 3,697 exposed attributes in the corpus are inherited, and qualifying one against the entity that merely uses it is CE1613. `example '...'` is new syntax, added because describe emitting nothing for it made describe -> exec lossy. Rare (1 of 4,707 elements) but real, and silently dropping an authored value is the failure this project keeps guarding against. ALTER edits the stored document rather than rebuilding it, so definitions the statement does not mention are never round-tripped through the describer. Dropping or renaming a definition a mapping still references is refused, naming the mappings. Refs #272 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Fifth and final slice of #272. mdl-examples/doctype-tests/40-message-definition-examples.mdl runs clean end to end and mxbuild reports 0 errors, including an import mapping bound to the authored definition. It also demonstrates the same association traversed both ways, which is the property most likely to be got wrong. Writing it caught one real bug: after a CREATE that resolves a folder, the cached hierarchy predates that folder, so a later CREATE OR MODIFY did not find the collection and wrote a DUPLICATE (CE0122). The update branch gets the invalidation free from applyDocumentFolder; the create branch has to say it. JSON structures avoid this only because their create path happens to call applyDocumentFolder first. The skill section moved to reference/message-definitions.md — the body crossed the 700-line bound — leaving a pointer with the two facts a reader needs before they follow it. The proposal is marked implemented, with two scope decisions corrected against measurement rather than left as written: inherited attributes are IN (398 of 3,697 exposed attributes, 10.8%), and Example is IN (author-set, rare, and silently dropped if hardcoded). It also records the five derivations that only a hand-authored document revealed. Refs #272 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Resolving findings/mdl-executor.jsonl by hand: `merge=union` resurrected a record main had DELETED. main removed a 2026-09-01 navigation finding (consolidated into docs-wiki/bug-patterns/), and union never deletes — so the merged shard had 506 records where main has 504 plus this branch's one. Resolved as main's shard verbatim plus the new record: 505. Second time this has bitten (see the merge in #369). Union is right for concurrent appends to a file of independent records, which is what .gitattributes documents, and wrong the moment either side deletes or rewrites — and `make check-findings` counts records without noticing they came back from the dead, so the check is on line counts against main, not on the tool. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CI's build-and-test failed on PR #380: --- FAIL: TestMxCheck_DoctypeScripts/40-message-definition-examples.mdl/legacy Execution error: failed to create message definition collection: creating a message definition collection requires the modelsdk engine The doctype round trip runs every example through exec + mx check on BOTH engines, and message definition authoring is modelsdk-only by design — the legacy writer has no serializer for the document, and building one would duplicate a shape the codec already gets right, including a typed-array marker of 2 and an empty-but-present Children list, neither of which is the codec's default. engineSkipList is the established mechanism for exactly this; menus, rules, layouts, navigation profiles and languages all have an entry for the same reason. This adds the sixth. Worth recording why local verification missed it: the test is behind //go:build integration, so `go test ./...` never compiles it. `make vet` now vets the integration tag set (from the #319 fix), which type-checks but does not RUN anything. Running it needs mxbuild, which is available locally — `MX_BINARY=~/.mxcli/mxbuild/11.13.0/modeler/mx go test -tags integration ./mdl/executor/ -run TestMxCheck_DoctypeScripts/<file>` — and that is what verified this fix: legacy SKIPs with the reason, modelsdk PASSes at 0 errors. Refs #272 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #272. Implements PROPOSAL_authorable_message_definitions.md (#371, #375).
A mapping binds to one of four schema sources, and MDL could create one. Message
definitions are 74 of the 327 mappings in the demo corpus (22.6%) and were
the only other one a script could bring into existence — an XML schema holds an
imported
.xsdand a web service holds a WSDL, but a message definition holdsnothing external: it is a selection over the domain model.
Plus
DROP,DESCRIBE(re-executable),SHOW— there was no listing at all —and targeted
ALTERfor both definitions and members.The evidence this is right
describe→execagainst ako/TestApp's hand-authored collection reproducesStudio Pro's document exactly. The executor prints
Unchanged, meaningcanon.Reconcilefound the rebuilt content semantically equal and elided thewrite. mxbuild 11.13/11.14: 0 errors, including mappings in both directions
bound to an authored definition.
One hand-authored document beat a 4,686-element census
The proposal was measured over 36 marketplace-module collections. Round-tripping
one hand-authored document (thank you) found five things that census had not:
Pathis a chain of ORIGINAL names, not exposed ones, and anassociation contributes two segments —
Order|OrderLine_Order|OrderLine|Amount.Confirmed afterwards at 4,707/4,707 once we knew to look.
Childreneven when empty — the bare[2],same
MandatoryListsrule as a rule document'sFlows.PrimitiveTypeis mapped, not passed through:Long → Integer,AutoNumber → Integer,Enumeration → String. A pass-through gets 279corpus elements wrong; TestApp caught it on a
Long.Exampleis author-set — empty in 4,686/4,686 of the corpus, set inTestApp. Hardcoding it empty would silently drop the one that exists, so
example '...'is now syntax.The round-trip test is what finds these, and the baseline matters: compare
against the stored bytes, not a re-encoding of the decoded original — a
lazily-decoded element that was never marked dirty encodes as an empty document,
so that baseline passes by comparing nothing to nothing.
The derivation most likely to be got wrong
MaxOccurson an association follows the direction of traversal, not theassociation's type — all 927 resolvable associations in the corpus are
Reference, yet 526 store1and 401 store-1:1-1An association connecting the two entities in neither direction is
refused, not defaulted — a wrong cardinality exposes a list as a single
object and builds cleanly, so there is no downstream check to catch it.
TestApp carries the natural control:
Mappings.Order_Customerappears in bothits definitions with a different
MaxOccurseach time. Pinned asTestFixtureCarriesTheDirectionControl.Two proposal decisions corrected by measurement
(10.8%). The stored
Attributenames the DECLARING entity, whichDeclaringMemberRefalready resolves for mappings.Exampleis IN — see above.Not guessed
Studio Pro pluralises a repeating element's exposed name. mxcli defaults to
the entity's own name and lets
as 'Orders'say otherwise — reproducing Englishinflection needs
-y → -iesand an already-plural detector, and a name theauthor writes beats one a heuristic guesses. Same conclusion as array-item
naming in #272's first half.
A bug the example caught
After a
CREATEthat resolves a folder, the cached hierarchy predates thatfolder, so a later
create or modifydid not find the collection and wrote aduplicate (
CE0122). I checked whether it was pre-existing: JSON structuresdo not hit it, but only because their create path happens to call
applyDocumentFolderfirst. The create branch now invalidates explicitly.Scope
Import and export mappings, and nothing else — confirmed three ways:
generated/metamodel(11.6 snapshot) declaresmessageDefinitionon exactlythose two types;
modelsdk/gen(newer) declares the ref only inimportmappings/exportmappings/mappings, withbusinesseventsandkafkadeclaring none; and across the corpus + TestApp, of 24 collections the only
documents naming one are 8 import and 8 export mappings.
Authoring is modelsdk-only, as rules, menus and layouts are; legacy refuses with
the established message. Reading works on both engines.
Verification
reports 14 diffs, reverting the marker reports 1.
MaxOccurs = 1, want -1; reverting the type map givesLong, want Integer.mdl-examples/doctype-tests/40-message-definition-examples.mdlruns clean andbuilds at 0 errors.
Full suite,
go vet,gofmt,make check-mdl(463 PASS) andmake check-skill-mdlare green.Reviewer notes
MEMBER(onlyMEMBERSexisted) andEXAMPLE.ANTLR prefers the longest match so
membersstill lexes asMEMBERS, andboth are added to
identifierOrKeywordso attributes calledMemberorExamplestill parse.COLLECTIONalready accepted singular and plural.TestNewRegistry_HandlerCountSnapshotandTestEveryShowAlternativeProducesAStatementboth caught missing wiring duringthis work — they are doing their job. The equivalent guard does not exist for
CREATE/ALTER/DROP; worth adding separately, since it will likely flag
pre-existing gaps that should not ride along here.
mainneeded a hand-resolvedfindings/mdl-executor.jsonl:merge=unionresurrected a record main had deleted. Second time(see fix(mappings): refuse to rewrite a SOAP-sourced mapping instead of dropping it #369); the merge commit explains it.
🤖 Generated with Claude Code