Does bulk API validate against languages?

[enikao]

In a bulk API request, does the repository check the request for consistency w.r.t. the language?

This relates to:

• classifier of node exists
• if node is an annotation: node's classifier can annotate parent's classifier
• if node's classifier is not an annotation: node can exist as child in its parent's classifier
• if node is used as a partition: node's classifier is marked as partition
• if node is not used as a partition: node's classifier is not marked as partition
• nodes mentioned in annotations are annotations
• metapointers of properties are part of the node's classifier
• property values adhere to its classifier type
• all required properties are set
• metapointers of containments are part of the node's classifier
• nodes mentioned in containments are not annotations
• nodes mentioned in containments fit the metapointer of the containment
• all containments adhere to their cardinality (required ones are present, only multiple ones have more than one child)
• metapointers of references are part of the node's classifier
• nodes mentioned in references fit the metapointer of the reference
• all references adhere to their cardinality

Pro:

• All participants can assume valid models (after initial validation)

Con:

• The repository has to check the validity on every request. This might be expensive (we have to care for all concept inheritance and polymorphism)
• We can never change a language without changing its version (useful during development)
• If there is an issue with the model, we need a "backdoor" to the repository to fix it
• For required children (including transitive) and references, we can only store the model once we've created the whole required subtree
• Parser-based clients can only store valid ASTs, or have to use extremely leniently designed languages. Especially for interactive editors, the AST is invalid most of the time.

[enikao]

What happens with repositories that do support invalid models (e.g. MPS)? Do they just omit the invalid parts? They would get lost as soon as the model is stored back.

[dslmeinte]

I think a repo that's aware of the languages involved could optionally do validation w.r.t. those languages (if prompted to do so), but must not have other behavior attached to that. That would make clients needlessly complicated and prohibit language evolution.

[joswarmer]

Just for clarification, we have three options:

Option A: Bulk API never checks against languages
Simple solution, but depends heavily on every client being 100% correct

Option B: Bulk API always checks against languages
More complex repository logic, but always a guaranteed correct model, no dependencies on all client being 100% correct.

Option C: Bulk API optionally checks against languages
This should be a setting defined by the repository because it is of little added value if only some of the requests are check as the repository might have incorrect models anyway and client need to take the same precautions as in option A

I am in favour of option B, for the following reasons

• looking from the client perspective, each chunk I get has the guarantee that it conforms to the language(s). Otherwise there are barely any guarantees and clients need to be able to handle random collections of nodes.
• as Niko already mentioned, if we allow incorrect chunks, we need to specify exactly what the repository will do for each incorrectness. That will make the specification overly complex.

[joswarmer]

Feedback on the Cons mentioned:

• The repository has to check the validity on every request. This might be expensive (we have to care for all concept inheritance and polymorphism)
  • Could be expensive, but by pre-processing the language(s), this can be optimised a lot
• We can never change a language without changing its version (useful during development)
  • Never change language without changing its version is something that we already require and what we definitely want
  • In a development environment you should always be able to use backdoors, but I think that we are not defining a development environment (and we shouldn't).
  • It is perfectly ok to have a 'development' mode for a repository where you are able to do things normally not allowed.
• If there is an issue with the model, we need a "backdoor" to the repository to fix it
  • Indeed, but I do not think that backdoor is something that we should define in the specification.
• For required children (including transitive) and references, we can only store the model once we've created the whole required subtree
  • This raises the question to what extent we check against the language, I can imagine that required children not need to be checked.
• Parser-based clients can only store valid ASTs, or have to use extremely leniently designed languages. Especially for interactive editors, the AST is invalid most of the time.
  • Parser based clients will store the text of the model anyway (because that is the actual source), so nothing is lost. And it probably is not very useful to store incorrect (and usually meaningless) information in a model-repository.

[enikao]

looking from the client perspective, each chunk I get has the guarantee that it conforms to the language(s). Otherwise there are barely any guarantees and clients need to be able to handle random collections of nodes.

I think this highly depends on the client: A regular editor or generator wants a valid model. But a validator or language migrator can (and wants) to deal with an invalid model.

We can never change a language without changing its version (useful during development)

Never change language without changing its version is something that we already require and what we definitely want

Where?

In a development environment you should always be able to use backdoors, but I think that we are not defining a development environment (and we shouldn't).

I disagree. We definitely want to enable language workbenches.

It is perfectly ok to have a 'development' mode for a repository where you are able to do things normally not allowed.

Interesting idea, similar to Meinte's optional validation.
The question is what the repository returns to a validating client if the repo contains invalid models (similar to my question above).

If there is an issue with the model, we need a "backdoor" to the repository to fix it

Indeed, but I do not think that backdoor is something that we should define in the specification.

I think that's a relatively common use case for language workbenches.

For required children (including transitive) and references, we can only store the model once we've created the whole required subtree

This raises the question to what extent we check against the language, I can imagine that required children not need to be checked.

Then we get into "half-valid" models. Does the client gain that much from this?

Parser-based clients can only store valid ASTs, or have to use extremely leniently designed languages. Especially for interactive editors, the AST is invalid most of the time.

Parser based clients will store the text of the model anyway (because that is the actual source), so nothing is lost. And it probably is not very useful to store incorrect (and usually meaningless) information in a model-repository.

I was thinking of something like Xtext: We have a primary AST, but in the background (for LW: Annotations) we store the original text. If we want direct compatibility with any text-based system, I think that's needed (see also #62).

[enikao]

If we find a good way to deal with delivering invalid models to validating clients, I'd prefer option C.
Otherwise, I think we disable many use cases if we enforce validation, so I'd go for option A.

[enikao]

MPS, EMF, and Modelix all allow models that violate their language. Parsers go to great lengths to create partially invalid ASTs (error recovery).
I think we would be incompatible with most existing systems if we enforced valid models.

[enikao]

Regarding the "backdoor" to a repository, I see the difference between using low-level MPS API to fix a model, or editing the .mps file with a text editor (similar for EMF and .xmi files).
If we can only fix the model by editing the file, there's really no other way than going through the backdoor. But that's rarely needed. In most cases, the low-level API suffices.

[joswarmer]

As we think allowing erroneous models might be important, I think we should come up with use cases and see which ones we can or cannot support.

[dslmeinte]

I strongly feel that the API shouldn't make much difference between valid and invalid models – insofar we even have a clear definition of that! The client shouldn't need to use a “backdoor”/low-level API to fix things, especially not if issues come to light during regular operation of the API.

(Just stating this is a guiding principle of sorts, not to say anyone is proposing to go the other direction.)

[joswarmer]

I can imagine that we allow certain kinds of invalid models, like e.g. partial models where e.g. a mandatory child is missing. It is hard to imagine invalid models in general, as clients then need to be able to handle random trees with random content. It seems that this effectively requires every client to be fully generic. Aren't we missing the point of having metamodels if we allow this?

Alternative guiding principle: we allow partial models, where the part of the AST that is there conforms to the metamodel. More explicitly I am thinking of something like:

• allow missing mandatory children, missing property values, etc.
• If we find a good way to attach incorrect "user input" (not an AST, but probably just text) we can support that using e.g. annotations
• we do not allow the use on MetaPointers that do not resolve to a node in the used language definitions.
• we only allow features of a node (properties, containments, references) if they are defined as feature of its classifier.

[joswarmer]

Use Case 1: Incorrect parsed text
The user edits a text, which is parsed with one or more errors. He wants to store this in a LionWeb repository

• Scenario 1: the user saves the file as text and the next time he wants to edit, he opens the text file.
  In this scenario there is no direct need to store invalid text in the repository as the original text is always used. We might or might not allow the part of the text that parses into a partial AST to be stored.
• Scenario 2: the user stores everything in the repository and when he wants to edit it he retrieves the AST from the repository, which is then unparsed into text that the user can edit.
  In this scenario it is useful to store any incorrect text in the repository, otherwise it will be lost. We can store the partial AST, and attach the incorrect text to it. The main requirement is that we can put this text at the original position in the unparsed AST.
• Scenario 3 extends scenario 2: After storing the incorrect model, another user retrieves the model to edit it in a projectional editor.
  Ideally we should be able to project the incorrect text inside the projection of the (partial) AST at the original position. So the location where this text is attached should not only be useable for unparsing , but also for projections.

[enikao]

Use case 2: Editor with hard-coded domain classes

An editor (graphical / form-based / projectional) uses generated classes (derived from LW language) to represent the model. It has no way to handle nodes of unknown classifiers, or unknown features, as they don't appear in the generated classes.