Scope Provider Issue

[vasilache-radu]

Hi, guys! This is the first time I ask a question on this forum. Until now I was able to find some threads about the issue I was facing, but this time it seems like nobody had this problem.

My goal was to create a custom scope computation and scope provider to filter the scope of a cross-reference, with respect to a condition I am setting. What I am trying to say is that in my DSL code, when I want to reference a node through a symbol (grammar allows it) but there are 2 nodes that have the same name/ symbol, I will choose the one that respects a certain condition. I know that one of the solutions would be using qualified names, but I don't want that.

The problem appeared here:
My idea was to access the list/ map that contains the scope of a certain context (ReferenceInfo) and delete all the AstNodeDescription associated with that symbol, whose node doesn't respect my condition. I want to say that the scope that I want to filter will be the globalScope, meaning the one that is made during the indexing phase.

For this reason I am only trying to override the globalScope. MapScope being used as the class for the scope, I realize that the AstNodeDescriptions are saved in a Map(key, value) data structure and the constructor assigns elements to this.elements using this.elements.set(name, element) . This method has the property that if the name is already found in the map, it will override the value saved for that key/name. The name represents the name of the node of the respective AstNodeDescription, which is the same as the symbol we look for in the DSL code. Because of this, the global scope will not contain both nodes with the same symbol, but only the one that was saved the second (which overrides the first one).
As a consequence, not only can't I filter the scope as I want, but also the whole global scope is not calculated "correctly".

Question:
I spent a lot of time on this issue, and it seems like the creators of Langium didn't take into consideration this case, as the data structure that is at the center of this system doesn't support by nature 2 AstNodeDescription having the same name/symbol. Besides that I also couldn't find in any of the examples on the internet an implementation of the custom scope provider doing exactly this. Is this really the case or I am missing something fundamentally? In case it can be implemented, please explain me in detail the steps ( in the documentation I couldn't find anything related to my problem).

Custom Scope Computation + Provider:

export class MyScopeComputation extends DefaultScopeComputation {

    qualifiedNameProvider: QualifiedNameProvider;

    constructor(services: Dt5Services){
        super(services);
        this.qualifiedNameProvider= services.references.QualifiedNameProvider;
    }

    /**
     * Export all functions using their fully qualified name
     */
    override async computeExports(document: LangiumDocument): Promise<AstNodeDescription[]> {
        const exportedDescriptions: AstNodeDescription[] = [];
        for (const childNode of streamAllContents(document.parseResult.value)) {
            if (isStateCategory(childNode) || isStateValue(childNode) || isNameSpaceDefinition(childNode) || isComponentType(childNode)
            || isColor(childNode) || isName(childNode) || isDisplayDefinition(childNode)) {
                let name = this.nameProvider.getName(childNode);
                if(name){
                    const fullyQualifiedName = this.qualifiedNameProvider.getQualifiedName(childNode, name);
                    /**
                     * Checking the qualified names on the console
                     */
                    exportedDescriptions.push(this.descriptions.createDescription(childNode, fullyQualifiedName, document));
                }    
            }
        }
        return exportedDescriptions;
    }

    override async computeLocalScopes(document: LangiumDocument, cancelToken = CancellationToken.None): Promise<PrecomputedScopes> {
        const model = document.parseResult.value as Model;
        const scopes = new MultiMap<AstNode, AstNodeDescription>();
        await this.processContainer(model, scopes, document, cancelToken);
        return scopes;
    }

    protected async processContainer(container: Model, scopes: PrecomputedScopes, document: LangiumDocument, cancelToken: CancellationToken): Promise<AstNodeDescription[]> {
        const localDescriptions: AstNodeDescription[] = [];
        for (const element of container.elements) {
            await interruptAndCheck(cancelToken);
            if (!isStateCategory(element) && !isStateValue(element) && !isColor(element) && !isDisplayDefinition(element) && 'name' in element) {
                const description = this.descriptions.createDescription(element, element.name as string, document);
                localDescriptions.push(description);
            } else if (isStateCategory(element) || isStateValue(element) || isColor(element) || isDisplayDefinition(element)) {
                const nestedDescriptions = await this.processContainer(element, scopes, document, cancelToken);
                for (const description of nestedDescriptions) {
                    // Add qualified names to the container
                    const qualified = this.createQualifiedDescription(element, description, document);
                    localDescriptions.push(qualified);
                }
            }
        }
        scopes.addAll(container, localDescriptions);
        return localDescriptions;
    }

    protected createQualifiedDescription(pack: StateCategory | StateValue | Color | DisplayDefinition, description: AstNodeDescription, document: LangiumDocument): AstNodeDescription {
        const name = this.qualifiedNameProvider.getQualifiedName(pack, description.name);
        // eslint-disable-next-line @typescript-eslint/no-non-null-assertion
        return this.descriptions.createDescription(description.node!, name, document);
    }
}
    
export class MyScopeProvider extends DefaultScopeProvider {

    constructor(services: Dt5Services){
        super(services);
    }
    
    protected override getGlobalScope(referenceType: string, context: ReferenceInfo): Scope{
        let globalScope  = new MapScope(this.indexManager.allElements(referenceType));
        if(isModelInstanceDefault(context.container)){
            if(context.property=='category'){
                const modelInstanceDefault = context.container;
                const modelInstance = modelInstanceDefault.$container;
                const componentType = modelInstance.type.ref;

                let filteredGlobalScope: MapScope = new MapScope([]);
                filteredGlobalScope.elements.clear();
                /**
                 * element = <name, AstNodeDescription>
                 */
                for(const element of globalScope.elements){
                    const name = element[0];
                    const desc = element[1];
                    const nodeOfDesc = desc.node;

                    if(isStateCategory(nodeOfDesc) && nodeOfDesc.$container!==componentType){
                        //globalScope.elements.delete(name);
                        //I think this deletes actually the other node, that we want to keep, because the name is the same.
                    }else{
                        filteredGlobalScope.elements.set(name, desc); 
                    }              
                }
                return filteredGlobalScope;
            }
            else{   // context.property == 'stateValue'
                return globalScope;
            }
        }
        return globalScope;
    }
}

[msujew]

Hey @vasilache-radu, thanks for the question,

I'm not sure why you're doing this:

let globalScope  = new MapScope(this.indexManager.allElements(referenceType));

In your global scope computation. Your issue seems to be that you're losing information in the MapScope, which is expected. You don't need to base your scope implementation on the MapScope though. It's just a very specific implementation of the more general Scope interface. You can implement your getGlobalScope implementation on this.indexManager.allElements(referenceType) directly, instead of going through the MapScope that is being produced, which, again, is a lossy operation.

Note that we also have a StreamScope implementation available that doesn't lose any information. MapScope just has the benefit that it's the fastest scope possible when combined with caching for the default global scope implementation. That's why we ship it as the default implementation.

[vasilache-radu]

Hey @msujew, thank you very much for your fast reply.

You are right about the misuse of MapScope. I don't know why I tried only with MapScope and not StreamScope.
I changed the code as you suggested, but I get the same error, I am wondering now whether indexManager is indexing all the elements, even the ones with the same name.

In the following code I am trying to filter the descriptions, whose nodes do respect the conditions nodeOfDesc.&container == componentType (for the stateCategory case) or nodeOfDesc?.$container?.$container == componentType (for the stateValue case).

The 2 nodes that have the same symbol are in 2 different files, and both files are different from the file where the cross-reference takes place. It is interesting that when deleting one of the files and then recreating it immediately, the symbol in the other file gets referenced. This makes me think that there might be another overriding process somewhere (maybe in the indexing).
What do you think about this?

     protected override getGlobalScope(referenceType: string, context: ReferenceInfo): Scope{
            let globalScope = new StreamScope(this.indexManager.allElements(referenceType));
            if(isModelInstanceDefault(context.container)){
            
                const modelInstanceDefault = context.container;
                const modelInstance = modelInstanceDefault.$container;
                const componentType = modelInstance.type.ref;
            
                if(context.property == 'category'){
                    let elements =  globalScope.elements.filter(desc =>{
                       const nodeOfDesc = desc.node;
                        if(isStateCategory(nodeOfDesc)){
                            if(nodeOfDesc?.$container === componentType)
                                return true
                            else
                                return false
                        }else return true
                    })
    
                    return new StreamScope(elements);
                    
                }else if(context.property == 'stateValue'){
                    let elements = globalScope.elements.filter(function(desc): boolean{
                        const nodeOfDesc = desc.node;
                        if(isStateValue(nodeOfDesc)){
                            if(nodeOfDesc?.$container?.$container == componentType)
                                return true;
                            else return false;
                        } else return true;
                    })
    
                    return new StreamScope(elements);
                }
            }
            return globalScope;
        }

[msujew]

It is interesting that when deleting one of the files and then recreating it immediately, the symbol in the other file gets referenced. This makes me think that there might be another overriding process somewhere (maybe in the indexing).

So that generally makes sense. The index is stored in the order in which files are processed. In case a file is removed and re-added it gets moved to the end of the index, therefore possibly making other elements have higher "priority" in the linking process. Generally, linking becomes non-deterministic once you have multiple elements in the same scope with the same name.

I changed the code as you suggested, but I get the same error, I am wondering now whether indexManager is indexing all the elements, even the ones with the same name.

Yes, that is exactly what the IndexManager is supposed to do. It stores all exported elements of the workspace in an index. Note that the AstNodeDescription#node property you're attempting to use is usually undefined (and only very rarely defined, see here). Filtering through that doesn't really work. You can load the node like done here though to get the node back.

[vasilache-radu]

Hei @msujew,

After a lot of time and effort, it finally worked. Your both answers were very helpful, but I would make one suggestion.
In your last answer you suggested to load the node as the Linker does. I initially tried to make an instance of the Linker in the scope provider and then load the node something like Linker.loadAstNode(...), which didn't work, I guess because the Linker was called in a previous phase than it was supposed to. Maybe this is a newbie approach, but probably there are other beginners who thought about trying it and they will get disappointed. What I did and it worked, was to create a new loadAstNode method in MyScopeProvider class, which was exactly the same as the one in the linker and use it.
It isn't 100% clear to me why it works, but it works. I hope it helps.