Token scoping?

[larsbaunwall]

Hi all

Thanks for starting what looks like a great and way more accessible approach to language engineering.

I am new to this discipline (so trying to learn as much as I can), and I am currently fiddling with a DSL for domain-driven design documentation - inspired by the xtext-based ContextMapper project.

I am trying to construct a language where I can describe different attributes of a model element - like e.g. this snippet (redacted):

BoundedContext:
    'BoundedContext' name=ID ('in' (inDomain+=[Domain]) ("," inDomain+=[Domain])*)?
    '{'
        ('description' ('is')? description=STRING)?
        ('classifiers' '{'
            ('role' ('is')? roleClassifier=[Role])?
        '}')?
    '}'
;

Role:
    'Role' name=ID
;

I am not allowed to use the keyword role here (in the inner scope), because it is used for the definition of Role.

The parser generates two identical tokens - which doesn't work runtime:

const RoleKeyword = createToken({ name: 'RoleKeyword', pattern: /Role/, longer_alt: ID });
const RoleKeyword = createToken({ name: 'RoleKeyword', pattern: /role/, longer_alt: ID });

How can I limit the scope of the keyword, so I can use it nested with different meaning?

Or maybe I am approaching this completely wrong?

[msujew]

Hi @larsbaunwall, glad to hear you're interested in working with Langium :)

Looks like you encountered one of the many bugs of version 0.1.0. This should definitely work, but when we wrote the initial parser generator we didn't think about that use case (same keyword wird different capitalization). For now, you should probably just change both of the role keywords to use the same capitalization.

With #169 we completely changed how the parser works (it is now completely in memory). For this, we also changed how token generation works and this use case won't be a problem anymore. So you didn't do anything wrong; This is definitely a bug in Langium. We are close to releasing a 0.2, which will contain a ton of new features (workspace indexing, cross-file cross-references, hovering, support for unit testing languages, a watch mode for the langium-cli etc.). You won't have to live with that workaround for long.

[larsbaunwall]

Thanks for a quick and thorough answer. Much appreciated - and looking forward to trying out ver. 0.2 🙂