Make LionCore M3 and builtins language definitions have version-dependent IDs

[dslmeinte]

ID could be something like -id-2024-1-Concept, (or -id-2024_1-Concept) etc.

Try this out in the reference implementations.

Also look at: IDs + keys of built-ins are defined, but only keys for LionCore elements — maybe want to align that a bit.

[enikao]

We do NOT have specified the ids of M3 elements for exactly that reason: Several versions of the same language MUST have different ids (#240). The ids of M3 elements in our tests happen to be similar to their keys, but that's only convenience for testing.

We only use language node ids to refer from other language elements (e.g. the type of a link points to the id of the classifier) #131.
Reason: They should be references like any other.
This is not a problem for languages: Languages are immutable (any changes to them are out-of-scope, #213). If we want to adjust our language in any way, it's a new language, with new ids, that might depend on a newer version of another language -- the target language having different ids again.

The only exception are built-in elements: they are used on all metalevels, and thus are also referenced by id on the same meta-level (all of M1, M2, M3).

Besides, we DO NOT want to give node ids any meaning or structure.

[enikao]

I found a use case for stable M3 ids: If we have annotations on language elements, the definition of that annotation refers to an M3 type:

annotation IntPropertyRequiredValueRange
  annotates M3.Property
  property minValue: int
  property maxValue: int

annotates is a regular reference, thus it uses the target's id, and not its key.

[joswarmer]

Indeed, but this is an annotation at the meta level (it annotates M3) and if the M3 changes into a new version shouldn't the annotation be updated?

[enikao]

Discussion from meeting at 2024-11-22:

Assume we have this language:

// meaning: MyLang is a language, defined by its meta-pointer to M3 concept with key langkey and version 2023.1
// The language itself has id 42, key abc, and version 1
MyLang <<Language key=langkey version=2023.1>>  id=42 key=abc version=1
  concept MyConcept
    property MyProp
          // meaning: MyProp is of type boolean, addressed by boolean's id which is id-builtins-2023_1_boolean
          // we refer to the built-in boolean type because both this language and builtin are on M2,
          // so the reference is not crossing meta-levels -- it's just a normal reference
	  type = {id: id-builtins-2023_1_boolean, resolveInfo: boolean}
    property OtherProp
          // meaning: OtherProp is of type x, the StructuredDataType from OtherLang. Similar to the previous property,
          // we refer to the property's type by id (regular reference on the same meta-level).
	  type = 5

OtherLang
  StructuredDataType x id=5 key=xxx

// This assumes the following builtin:
Builtins_v1<<Language>> id=id-builtins-2023_1  key=key-builtins-2023_1 version=2023.1
  Boolean<<PrimitiveDataType>> id=id-builtins-2023_1_boolean
  JSON<<PrimitiveDataType>> id=id-builtins-2023_1_json

----

// if we had the same language MyLang in a LW-2024.1 environment, it would need to look like:
MyLang <<Language key=langkey version=2024.1>>  id=42 key=abc version=1
  concept MyConcept
    property MyProp
	  type = id-builtins-2024_1_boolean

Builtins_v2<<Language>> id=id-builtins-2024_1  key=key-builtins version=2024.1
  Boolean<<PrimitiveDataType>> id=id-builtins_boolean

Builtins_v1 and Builtins_v2 MUST be different, because they have different children (JSON is missing in Builtins_v2).
Thus, the MUST have different node ids. Thus, all their children MUST have different node ids (in each version).

But then languages that use builtin types have to point to different node ids without any actual change.

NOTE: The same applies to Lionweb M3 language, e.g. if a link targets Node.

Possible solutions

A) Treat references to builtins special: resolve references to buitins via key, everything else via id

Pro:

• don't need to change languages when migrating to new LW version
  Con:
• Language nodes are special nodes
  Can't use regular tools (e.g. node comparer) to process them

B) Have LW-version-specific node ids for builtins

Pro:

• Unique node ids
• Language elements stay regular nodes
• Can treat different builtin versions similar to different other language versions

Con:

• Languages change when migrating to new LW version

C) All builtins versions share the same node id

Pro:

• don't need to change languages when migrating to new LW version
  Con:
• breaks fundamental node id uniqueness assumptions
  Builtins are kind of special often anyways

D) reference targets are described by either an id or a meta-pointer

Pro:

• Unique resolution possible

Con:

• M2 nodes look different from M1 nodes

node {
      "id": "ddd",
      "classifier": {
        "language": "myLanguage",
        "version": "2",
        "key": "myConceptId"
      },
      "properties": [],
      "containments": [],
      "references": [
        {
          "reference": {
            "language": "myLanguage",
            "version": "2",
            "key": "singleReferenceId"
          },
          "targets": [
            {
              "resolveInfo": "some name",
              // This is a reference to a regular node
              "reference": "dee"
            }
          ],
          "targets": [
            {
              "resolveInfo": "some name",
              // This is a reference to a language node
              "reference":  {
                "language": "builtins",
                // "version": "2023.1",
                "key": "boolean"
              }
            }
          ]
        }
	  ]
}

E) Don't store LW-version-specific builtins reference target node ids, only store resolveInfo

(Alternatively: Also store target node id, but also MUST resolve by name via resolveInfo if target id is unknown or doesn't fit current LW version)

Pro:

• No new node type
• Uses regular, recommended meaning of resolveInfo

Con:

• builtins become a bit more magical
• resolveInfo MUST be set
• resolveInfo MUST be standardized

F) Any time a MyLang is used in a new LW version context, we have to create a new version of MyLang that uses the new Builtins

Pro:

• Clear-cut

Con:

• Have to migrate unchanged languages
• Have to migrate all languages you're involved with
  i.e. cannot mix languages of different LW versions

Decision: E

Rationale: Least special, no fundamentally new constructs, sufficiently unambiguous.

Open Questions:

• Does the resolveInfo contain e.g. Boolean or LionWeb.Builtins.Boolean? What if MyLang contains its own DataType Boolean? By usual scope resolution rules, the "closer" name would win.
  Decision: Use LionWeb.Builtins.Boolean

• Does the resolveInfo contain Boolean (the name) or key-boolean (the key)?
  Decision: Neither, see above

• What if we actually do know the target node id, but it's in the wrong LionWeb version (example: Chunk is in version 2024.1, but referenced boolean lives in version 2023.1)?
  NOTE: This only is relevant if there's a mismatch between target node id LW version and chunk LW version.

  • Variant aa: Always use target node id if resolvable
    Pro: Same handling as everywhere else
    Con: We can have mixed LW versions in same chunk
  • Variant bb: Always use resolveInfo, ignore target node id (only for builtins/M3)
    Pro: No mixture of LW versions possible
    Con: Different from usual handling (resolvable target node id takes precedence)
  • Variant cc: Fail with error
    Pro: Unambiguous
    Con: ?

Decision: Variant cc

[joswarmer]

"But then languages that use builtin types have to point to different node ids without any actual change."

Moving to a newer version of LionWeb is an actual change, isn't it? So making some (minor) changes to a language and/or model to migrate to a new version seem quite normal. Why is this assumed to be a problem in this issue?

Freon uses name-based references for this very reason, you can have new versions of nodes (not just builtins, but all) without breaking things. So I think E is a good solution

The discussion above is important in a much broader context: e.g. if I use a model library and have references to nodes in this library, then the same problems occurs when moving to a newer version of the library, since node id's might have changed. That is why the resolveInfo is always important. If the referred id does not exist, we can use the resolveInfo to rebind the reference to another node, usually the new version of the 'same' node.

[joswarmer]

The resolveInfo should contain the name, not the key. The reason is that when the resolveInfo is not enough to find the node, we can only show the resolveInfo to the user, and that should be a readable name that the user can recognise. Keys are internal names that the user never sees.

[enikao]

Consequence: Assume we ship a language as nodes and assume upwards and downwards compatibility with future/previous LionWeb versions. Then, we SHOULD serialize all references to M3/builtins with only resolveInfo filled, not target node id.

[dslmeinte]

I assume this change will be part of LionWeb version 2024.1, right?

[enikao]

"But then languages that use builtin types have to point to different node ids without any actual change."

Moving to a newer version of LionWeb is an actual change, isn't it? So making some (minor) changes to a language and/or model to migrate to a new version seem quite normal. Why is this assumed to be a problem in this issue?

Freon uses name-based references for this very reason, you can have new versions of nodes (not just builtins, but all) without breaking things. So I think E is a good solution

The discussion above is important in a much broader context: e.g. if I use a model library and have references to nodes in this library, then the same problems occurs when moving to a newer version of the library, since node id's might have changed. That is why the resolveInfo is always important. If the referred id does not exist, we can use the resolveInfo to rebind the reference to another node, usually the new version of the 'same' node.

I think the difference is who initiates the change: Changes to a model library are done by the language designer, thus they can also apply all relevant follow-up changes.

Moving a repository (implementation) to a new LionWeb version is done by the repository (or its admin). So the repo knows about this change, and can handle follow-up changes of that change -- i.e. changed builtins etc.

According to #322, a repository can chose not to support older and/or mixed LionWeb versions. Then it needs to provide some way of migrating contained languages -- but this must be implementation-specific for several reasons:

• LW versions don't need to be backwards compatible (We strive for compatibility between different LionWeb versions, but don't guarantee it #323); what if a language in the repo used JSON datatype?
• The repo would need to create new versions of each language, thus assign a new version number. But according to What does Language.version mean semantically? #130, we (in general) don't know how to "increment" any language version.