Multiple Languages from different Projects

[mehmetsalgar]

Hi everybody,

to let the reasoning of discussion become more clear for you, I am coming from Eclipse XText world, which is nice of course because most the concepts of the Langium are really similar.

I am interested multi dependant languages feature of the Langium as explained here..

Multi Dependant Language

My dilemma is, this article and from what I can see from the examples in the repositories, I have to implement all the languages in one single project. My wish is to bring languages as dependencies to a new Langium project.

We are able to this in Eclipse XText as declaring an OSGI dependency the referenced language, I don't see such an example or documentation in Langium, is it possible at all...

The reason I am asking, a company that I am involved has departments that are creating grammars/languages under Eclipse XText projects and this languages can be referenced other departments grammars/languages, which is not big deal as referencing those as OSGI dependencies.

Now I am evaluating that Langium can be an alternative to Eclipse XText or not but I can't figure out this possibility exist or not? It is not practical for the company that I mentioned to place all grammar/languages in one single project because it shear size of the end project would reach.

Did anybody else tried something like this?

[spoenemann]

We explicitly decided against the option of combining languages dynamically. When you have multiple languages referencing each other in Langium, you have to build them all in one project.

The reason for this decision is that we wanted to avoid the additional complexity involved with dynamic linking of languages. In Xtext, that's only possible because the underlying EMF holds complete meta-models of the ASTs in memory, enabling complex reflection at runtime. We don't have that level of reflection in Langium and we're not planning to add it as of now.

[msujew]

@spoenemann and I just met up to see how one would approach this:

1. While Langium does not enable simple reuse of language artifacts, you should be able to import any language defined in any dependency. I've created Enable grammar imports from node_modules #1376 to make this a bit easier, but it's already possible. You can use the langium-cli and should be able to reuse the services defined in your imported language.
2. You can in actually reuse generated language artifacts by exposing the generated service module and importing that in your project. However, that prevents you from actually using any parser rules from the imported language - you can only create references to the objects in the imported language.
3. You mentioned "sub languages". Do you mean inherited languages with overriden rules? Langium does not support that feature (yet), but you should be able to accomplish something similar using parameterized rules.

Note that we are always open to contributions to improve things. However, adopters appreciate the simplicity of Langium and we don't intend on making things more complicated for them. Anyone who has ever worked with Xtext can attest that the framework is the definition of non-trivial when it comes to configuration and packaging.

[mehmetsalgar]

When I am more experienced with Langium, I am open for contribution but I would need little bit time for it :)

You mentioned "sub languages"

what I mean there the way you structure multiple language under one project as you demonstrated in Multiple Dependant Language example.

I guess I have to give little bit more context about what we did with XText and what I would try to achieve with Langium.

In our Software Development projects, we are using XText DSL and Model, to create with XTend, Angular Single Page Applications, REST Microservices and JPA persistence layers.

Now we have multiple departments like Customer Acquisition, for this department, they have DSL that is focused in Customer Acquisition (which has a DSL for Private Persons and Business Persons which sub languages I mentioned), then we have a department focusing on Display Product Options, its DSL focusing on that topic, a Payments Department which has a DSL for that topic (off course there are other departments but lets start with those).

We also have other departments are using DSL as puzzle parts to build there Business Case in their DSL. So their DSL should be to reference the DSL of the Customer Acquisition, Product Options, Payment, etc and combine those and additional elements from their DSL and Generators to create their Angular, REST, JPA code....to have element of reusability for different Business Cases.

Now this works with XText/XTend actually it is not a must to change to Langium but as you said and it is also my first impression, Langium seems to be much more lightweight so I am just checking, we can implement the same scenario or not?

[spoenemann]

We believe it is possible to implement such a complex scenario with Langium, but it will likely involve some custom code to wire the independent languages together (e.g. to properly support the AstReflection service).

@mehmetsalgar you're welcome to get in touch with me directly if you think your company would have funding for a support contract: https://www.typefox.io/language-engineering/

[mehmetsalgar]

Hi,

first of all let me say I have no problem with the idea of a support contract but (there is always a but :)), as I previously mentioned, we have everything working in Eclipse XText / XTend.

My motivation is just to look is it possible to realise something similar with Langium, while our user complains how heavy Eclipse IDE is and my first impression of Langium and Visual Studio Code how light it feels but to prove to the company the transformation can work, I have to build a PoC.

If I go and say, we need a support contract for a PoC, well that would not be recipe for success, but with
the input I got here, I will try to understand how AstReflection System works, try to manipulate it so it can help me with PoC, but I am not a TypeScript expert so that would probably take some time, then we will see how it goes.

[spoenemann]

Got it. Feel free to ask detailed questions in this forum.

I think the wiring of multiple languages is something that needs to be solved only once, but after that you'll really have a more light-weight solution compared to Eclipse/Xtext/EMF.

[goto40]

I wonder if this is related to #1329? At least the question stated above is exactly what I miss in Langium. My use case is restricted to different languages (from different projects) referencing each other. This restricted use case might be much simpler than the general case.

[goto40]

I continued my experiment.

Following your suggestion (as far as I understood), I defined a custom CommonAstReflection and 'wired' it as follows:

export const CommonSharedModule: Module<LangiumSharedServices, LangiumGeneratedSharedCoreServices> = {
    AstReflection: () => new CommonAstReflection()
};

However, it seems that the generated ast.ts contains a line

export const reflection = new HelloAstReflection();

With this my custom AstReflection class CommonAstReflection seems not to be used in all cases...

Did I miss some detail? I would appreciate a hint how to proceed.

[msujew]

The export const reflection is only used by the generated is<type> methods. The scoping mechanism, etc. should all use the CommonAstReflection. Is that not the case for you?

[goto40]

Ah. I tested it with exactly the is<type>. You perfectly explained the effect.

I just wondered how to link my placeholder Ast-type in my referencing language (PersonFronOtherModel) with the concrete type Person from the referenced language.

//...
interface PersonFromOtherModel {
    name: string
}
Greeting:
    'Hello' person=[PersonFromOtherModel:ID]'!';
//...

I added Person to my custom Ast reflection in the getAllTypes() method and modified the isSubtype() method. Note, I did not customize scoping...

//...
    override isInstance(node: unknown, type: string): boolean {
        const result = super.isInstance(node, type);
        console.log(`isInstance(%o, ${type}) -> %o`, node, result);
        return result;
    }
    override isSubtype(subtype: string, supertype: string): boolean {
        let result = super.isSubtype(subtype, supertype);
        if (subtype===Person && supertype===PersonFromOtherModel) result=true;
        console.log(`isSubtype(${subtype}, ${supertype}) -> %o`, result);
        return result;
    }
    override getAllTypes(): string[] {
        const result = super.getAllTypes();
        result.push(Person);
        console.log('getAllTypes() -> %o', result);
        return result;
    }
//...

Correction: It works after everything is loaded. On startup I get linker errors, but when editing the model I resolves everything correctly! :-)

Now I need to figure out, where my async problem is (I need to wait for all models to be loaded...)

[goto40]

Do I need to tell the referenced model to notify some instance/service?
The effect is that after opening the workspace the reference is not found - but when editing something (a space somewhere) everything is fine...

After opening the workspace

After entering some white-space

[spoenemann]

🤔 looks like some debugging is needed to find the cause