PRD: Structural validation for edited MPS model XML

[sergej-koscejev]

Problem Statement

LLMs can edit JetBrains MPS model persistence directly, but after an edit they need a fast way to build confidence that the edited MPS model XML is still structurally usable. Today mops helps inspect and manipulate persistence artifacts, but it does not provide a validation run that reports whether edited targets still have well-formed XML, valid IDs, valid registry/import references, and model-wide local reference consistency.

MPS-backed semantic model checking would provide deeper feedback, but it depends on MPS, project state, loaded languages, and possibly preexisting project issues. The first useful slice should therefore provide always-available structural validation in Go, with a report shape that can later include semantic model checking findings.

Solution

Add mops validate as the model validation entry point. The first implementation layer is structural validation over explicit validation targets. It accepts standalone MPS model XML, file-per-root model folders, and single root files in an incomplete-validation mode.

The validation run checks all requested targets, emits all validation findings, and decides the overall status after the full report is produced. Human-readable output remains terse, while --json provides the stable validation report contract for LLM consumption.

The first slice validates XML and persistence structure, node ID syntax and uniqueness, registry/import index usage, local node references, external reference shape, and basic containment shape. It explicitly does not perform semantic model checking, baseline comparison, external target lookup, language-level role/concept validation, or Git changed-file discovery.

User Stories

1. As an LLM editing MPS model XML, I want to validate an edited target, so that I can tell whether my XML edit left the model structurally usable.
2. As an LLM editing a standalone model, I want mops validate to accept a standalone target, so that I can validate the exact file I changed.
3. As an LLM editing file-per-root persistence, I want mops validate to accept a file-per-root model folder, so that checks that require whole-model scope can see every direct root file.
4. As an LLM editing a single root file, I want validation to still check what it can, so that I can get useful feedback before I know the whole model folder.
5. As an LLM editing a single root file, I want the report to identify incomplete validation, so that I do not mistake root-local checks for whole-model validation.
6. As an LLM making several edits, I want one validation run to check every requested target, so that I receive a complete repair list instead of stopping at the first failure.
7. As an LLM consuming validation output, I want stable finding codes, so that I can react to findings without parsing human message text.
8. As a human reviewing validation output, I want concise messages, so that I can understand what needs attention without reading raw JSON.
9. As an automation consumer, I want --json, so that a coding agent can parse severity, code, target, file location, layer, and source reliably.
10. As an LLM editing XML, I want malformed XML to be reported, so that I know the model persistence cannot be parsed.
11. As an LLM editing model metadata, I want unsupported persistence versions to be reported, so that I do not assume checks designed for version 9 apply to another format.
12. As an LLM editing node IDs, I want invalid regular node IDs to be reported, so that non-canonical or malformed IDs are not introduced.
13. As an LLM editing node IDs, I want 0 to be accepted as a regular node ID, so that the validator matches the agreed persistence rules.
14. As an LLM editing node IDs, I want longer regular IDs with leading zeroes to be rejected, so that non-canonical IDs such as zero-padded values are caught.
15. As an LLM editing foreign node IDs, I want bare ~ and arbitrary ~-prefixed IDs to be accepted as opaque foreign node IDs, so that valid foreign IDs are not rejected.
16. As an LLM editing references, I want ^ to be rejected everywhere, so that dynamic reference markers that should not appear in real models are caught.
17. As an LLM editing a file-per-root model folder, I want duplicate node IDs detected across all direct root files, so that model-wide node identity remains valid.
18. As an LLM editing local references, I want local node references checked against the whole model target, so that cross-root local references are handled correctly.
19. As an LLM editing external references, I want import aliases checked against the model imports, so that broken import indices are caught locally.
20. As an LLM editing external references, I want external target node IDs checked for syntax, so that malformed external targets are caught without requiring the imported model.
21. As an LLM editing registry-backed attributes, I want concept and role indices checked against the model registry, so that stale or invented compact indices are caught.
22. As an LLM editing containment, I want basic containment shape checked, so that roots, child nodes, properties, and references remain in legal persistence positions.
23. As a CLI user, I want exit code 1 when blocking errors exist, so that shell automation can fail the task when validation fails.
24. As a CLI user, I want exit code 0 when only non-blocking findings exist, so that incomplete or informational findings do not automatically fail the workflow.
25. As a CLI user, I want exit code 2 for command misuse, so that CLI syntax errors remain distinct from model validation errors.
26. As a future implementer of semantic model checking, I want the validation report to already include layer and source fields, so that MPS-backed findings can be added without redesigning the report contract.

Implementation Decisions

• Add a new validate command instead of extending existing inspection or ID-generation workflows.
• Treat model validation as a layered concept: structural validation is the first always-available layer; semantic model checking is a later layer.
• Accept explicit validation targets only. Do not infer targets from Git changes or whole-project discovery in the first slice.
• Model a validation run as a combined report over all requested targets.
• Keep human output terse and provide --json as the stable machine-readable contract.
• Use validation findings with stable severity, code, message, target, concrete file location when available, validation layer, and finding source.
• Use error, warning, and info severities from the start, even if the first implementation mostly produces errors.
• Fail the validation run if any error-severity finding exists.
• Treat a file-per-root model folder as one model target made from .model metadata and direct root files.
• Treat a single root file as incomplete validation: useful local checks run, but model-wide checks requiring sibling roots are not asserted as complete.
• Validate regular node IDs with MPS Java-friendly base64 rules, allowing exactly 0 while rejecting longer leading-zero values.
• Accept foreign node IDs as opaque ~-prefixed IDs, including bare ~.
• Reject custom or extended node ID families outside regular and foreign IDs.
• Reject the dynamic reference marker everywhere for this project's structural validation contract.
• Enforce model-wide node ID uniqueness for complete model targets.
• Resolve local node references against the whole model target, not the containing file.
• Validate external references by import-index existence and target-ID syntax only.
• Validate registry-backed concept and role indices against the target's own registry only.
• Validate containment shape as persistence-level structure only; do not validate concept-specific cardinality.
• Keep baseline comparison separate from the first structural validation slice.
• Design the report shape so later semantic model checking can add findings with a different layer/source.
• Likely modules to build or modify: CLI command dispatch, validation target resolution, structural validation scanner, node ID classifier, finding/report model, human report renderer, JSON report renderer.
• The structural validator should be a deep module with a simple target-in/report-out interface that can be tested independently from CLI argument parsing.

Testing Decisions

• Test external behavior rather than internal parser implementation details.
• Add command-level tests for mops validate, exit codes, human output, JSON output, and CLI misuse.
• Add focused structural validation tests for standalone targets, file-per-root folders, and incomplete single-root validation.
• Add tests for node ID classification: regular IDs, 0, leading-zero rejection, foreign IDs including bare ~, malformed IDs, and forbidden ^.
• Add tests for duplicate node IDs across direct root files in a file-per-root model folder.
• Add tests for local references resolving across roots in the same model target.
• Add tests for external reference import-index validation and target-ID syntax validation without requiring the external model.
• Add tests for registry index duplicates and missing concept/role indices.
• Add tests for containment shape: root node role handling, child node role requirements, and property/reference placement.
• Add tests that a validation run continues after one target has findings and reports findings from later targets.
• Reuse the existing Go testing style: package-level tests for focused modules and command-level tests for CLI behavior.
• Keep tests deterministic with inline fixture XML rather than depending on real MPS installations.

Out of Scope

• Semantic model checking through MPS.
• Starting or discovering an MPS instance.
• Talking to an already-open MPS project.
• Baseline comparison of before/after validation reports.
• Git-aware changed-target discovery.
• Whole-project validation discovery.
• Checking whether external target nodes actually exist.
• Checking whether loaded languages define the referenced concepts or roles.
• Checking concept-specific child cardinality, scope rules, or user-defined model checker rules.
• Suggested fixes in validation findings.
• Automatic rewriting or repair of invalid MPS model XML.

Further Notes

The first slice should stay intentionally local and deterministic. It should give an LLM immediate confidence about persistence-level mistakes without depending on MPS runtime state. The report contract should be stable enough that later semantic model checking can add richer findings without changing how agents consume structural findings.