Repo API: Do we need model partitions?

[enikao]

Model partitions are similar to

• packages in Java
• models in MPS
• resources in EMF

Use cases

Model partitions could be useful for:

• Access control (Repo API: Access control #27) checking: a partition might be the atomic unit to set permissions.
  If we allowed permissions on every node, we have the same issue as e.g. ACLs in NTFS
• Locking (Repo API: Locking #30 ): a partition might be the atomic unit to lock parts of the model.
• Loading to web: a partition could be a unit that makes sense to load at once into a web editor, if loading the whole model was too expensive.
• private/public index: a partition might declare private and public symbols; only the public symbols would be entered in a global index.
• Checking whether a repo client can deal with this part of the model.
  Example: Web editor checks whether it knows all the concepts in this part of the model and can display them.
• User perspective: I have a collection of nodes, they comprise my model
• Language engineer: I have a collection of languages that comprise my DSL
• Exporting models automatically (agnostic of specific model/language): When do I need to re-export a bunch of nodes (resulting in one file) because contents changed?
• Re-importing models from external sources: Either overwrite or merge
• Unit for code generation
• Consistency: All nodes in one partition use the same version of a language
• Unit to declare dependencies
• Unit of modifying transaction

Assumptions

• We don't want "something else" besides nodes in LIonWeb
  --> partitions are "nodes with something extra"
• To be compatible with most existing systems, we need to allow several root nodes
• A (logical) LIonWeb repository stands on its own, i.e. it does not directly interact with external entities (might change in the future: Repository federation #23). A concrete LIonWeb repo implementation might host several repositories, but that's considered an implementation detail.

Considerations

• Think about this as a scoping problem
• Do we need a “workspace”/”repository” concept (in sense of MPS repository as logical root node of everything else)?
  --> probably not more than we already have with our repository idea

Arguments

Pro:

• Could benefit performance (e.g. smaller global index, efficient permission lookup)
• Reduce implementation complexity (e.g. don't need to reimplement NTFS ACLs)
• Clear conceptual demarcation line for language engineer (e.g. I only have to check reference proxies if the target might be outside the partition)
• Some kind of organization is needed everywhere. If we don’t standardize, everybody will come up with their own.

Con:

• Dealing with something else than nodes complicates APIs
  Possible solution:
  A partition is bound to a node in the model (e.g. by a special flag in the node's concept, or a special implemented ConceptInterface.) Thus we can never ask for a partition directly, but only for the node that happens to be hosted in its own partition.

• Might introduce issues in combination with versioning

Alternatives on partitions

Option A: No partitions

Some nodes happen to have no parent

Pro:

• Flexible

Con:

• No common understanding between LIonWeb components about use cases listed above

Option B: Non-nested partitions

Only partitions can be without parent

Pro:

• Easy to understand
• No ambiguities

Con:

• Cannot use partition concept instances inside other nodes (unless rootable means can be root, not must be root)
• Might limit flexibility: If two nodes need to be locked independently (as example for a use case), they must be in different roots -- even if semantically they should be in the same root. Workaround: add a reference.

Option C: Nested partitions

Subtrees of partitions can be their own partition

Pro:

• Flexible
• Use cases still applicable

Con:

• Complex interaction of partitions
• Might be hard to support by a repo implementation

How to mark partitions

Alpha: In metamodel as flag of Concept

For each concept, the language designer decides whether it's a partition concept.
A concept flagged as partition can only be used as partition.

Beta: In instance

Any instance can be marked as a partition.
If we chose option B above, then not having a parent node implies that node to be a partition. Otherwise, we can explicitly set an instance to be a partition.

Decision: Option B with variant alpha: Non-nested partitions with flag on concept for partitions

Rationale: Some kind of partition concept is useful, and non-nested partitions add only a reasonable overhead.

Consequences

We need to collect more experience with partitions. For now, we start with these assumptions:

• Partitions are the default unit of loading models into clients (not enforced, clients might request subparts) Bulk API atomicity #75
• Partitions are the default unit of locking Repo API: Locking #30
• Partitions are the default unit of access control Repo API: Access control #27
• Partitions explicitly contribute to a global name index that's visible from all partitions. Non-exported symbols are only visible inside the partition Establish global symbol index #136
• Partitions can be assumed as unit of automatic, generic export mechanisms after any change inside the partition.
• We postpone assumptions about partitions being the unit of automatic import (lack of experience)
• We postpone assumptions about partitions being the unit of derived models (lack of experience)
• Partitions are NOT the default unit of generation (useful unit of generation is too language specific)
• Partitions are NOT the default unit of dependencies (useful unit of dependencies is too language specific)
• One partition CAN contain nodes of the same language in different versions
• Changes to nodes CAN happen at other granularity than partitions (i.e. partitions are NOT unit of modifying transaction)
• A partition is always the root of a node tree. So a partition is similar to an MPS model, or an EMF resource.
• A repository consists of a set of partitions.

Details on concept flag

New property in M3: Concept gets new property partition: boolean. Default value is false, value is not inherited.

Example:

• Concept A { partition: true }
• Concept B extends A {}
• Concept C extends A { partition: false }
• Concept D extends A { partition: true }

Then

• A creates a partition
• B does not create a partition, it can be used as child of another node
• C does not create a partition, it can be used as child of another node
• D creates a partition

Alternative on concept flag: Marker interface

We ship an interface like IPartition in our standard library, and any concept that implements that interface is a partition.
Drawback: Cannot "unset" partition flag in this way.

Alternative on concept flag: Annotation

Might be useful in future once we have annotations.
We don't want to wait for them for introducing this flag.

[joswarmer]

In Freon, and before in Mendix, and various other environments (Eclipse, Microsoft Modeling Tools) I have used model partitions (I currently call them model units) extensively. The private/public notion and the much smaller index allows to have e.g. all data for code completion to be locally available in the client. We had some really large models at Mendix, where it does work ok.

A model partition in Freon is just another Node, with some restriction on its properties. I don't see these restrictions as essential, so it could just be a node. The main thing is that a concept can be marked as a Model Part in the meta-model of a language.

In Freon Model Parts are similar to root nodes in MPS.

In Freon we also have a Model Node which is the root of the model. Model Nodes can only have Model Parts as children and a Model Node is the only node that does not have a parent. We do not nest Model Parts, did not need it until now, not sure how useful that will be.

An additional pro:

• supports information hiding in modeling languages.

All properties inside a Node can be defined either be public or private. If a property is public, the nodes inside the property are visible/referrable from other model parts. This allows the language designer to hide internal details of a model part and make sure they cannot be referred to from other model parts. This makes it possible to replace a model part by another one.

We call the visible nodes of a model part the model part interface, which is just basically a subgraph of the full model part.

[ftomassetti]

I find these partitions useful conceptually: as a user I like to have some way to organize elements. I also think these divisions are useful for performance, scoping, encapsulation, and possibly to handle permissions (if we decided to support those)

[dslmeinte]

I find them useful as well. One drawback of the idea of a model being composed of units is that it a priori forces the organization. Can we generalize that idea in a meaningful and useful way, e.g. through defining custom partitions. Each partition could be a collection of roots, and the idea of an interface of a partition probably works the same way.

[joswarmer]

The definition of which types of modelunits there are in a language is an important concern for the language engineer defining the meta-model of the language. Just like e.g. Java defines classes and interfaces as part of the language definition and defines which subnodes of classes and interfaces are visible/referable.

A custom partition as a collection of roots is something quite different as far as I understand.

[joswarmer]

We should separate two different wways of organising models.

1. Model units as defined by the language designer, these are parts of the language definition.
2. User defined grouping of models/sub-models, like folders or packages, defined on the fly by the users of the language.

The Model Units as I used them and describe them are of type 1.
Type 2 grouping might be useful as well, but I consider that a different issue.

[dslmeinte]

Yeah, the problem with custom grouping is that what might be interesting/necessary to expose might depend on that grouping. So, if you provide the mechanism of model units, you at least can definitively figure out what to expose. For custom groupings, you might not even be able to?

[joswarmer]

You might use a means of grouping model units in (virtual) folders, semantically meaning nothing.
Putting individual nodes in folder is something for which don't really see the consequences, not sure it is even useful.

[dslmeinte]

I'm thinking along the lines of designating a bunch of roots as an “area of interest”, but I guess that's in practice probably pretty similar to a bunch of roots.

[enikao]

Updated original comment to reflect decisions on 2023-06-02

[enikao]

Open questions

• Bulk API to ask repository about all its partitions?
• Name: partition / model / model root / chunk / ?
• Confirm concept flag details as described above

[enikao]

We might revisit how to mark partition Concepts once Annotations are available (#13 )

[enikao]

How to specify Partition Concepts in MPS?

Assume we have a language in MPS, and want to export it to LIonWeb. How do we specify the value of the partition flag?

Candidates:

• Assume root concept == partition
• Provide a helper API / export model to specify the flag.
  This allows to add partition flag to languages we cannot change.
• Define a NodeAttribute for MPS ConceptDeclaration for the flag

[enikao]

Decision via Slack vote: Name "Partition"