Rework token single inheritance into multiple classifications/ categories

[bd82]

Currently Token inheritance is limited to single inheritance.

createToken({name: "TrueLiteral", pattern :  /true/, parent : LiteralToken})

There is no reason to limit the inheritance to single inheritance as the diamond problem
can no longer exist here as the tokens are no longer instances of their classes
but are instead simple JS Objects with properties.

Instead the inheritance can be modeled as traits in Scala or multiple interfaces in Java.
This actually has common valid use cases:

Consider the above example of the "TrueLiteral" token. In an ECMAScript grammar
one may want to mark it as both:

• A "LiteralToken" for simplifying the grammar of atomic expression.
  • https://www.ecma-international.org/ecma-262/5.1/index.html#sec-7.8
• An Identifier Token as it behaves similarly to reservedKeywords.
  • https://www.ecma-international.org/ecma-262/5.1/index.html#sec-7.6.1

createToken({name: "TrueLiteral", pattern :  /true/, traits : [LiteralToken, Identifier]})

[bd82]

Breaking Changes

CreateToken API

• The parent property no longer exists in the createToken API.
  • Use categories instead:

         const Keyword = createToken({name: "Keyword", pattern: Lexer.NA })
         const While = createToken({
               name: "While",
               pattern: /while/,
               categories: Keyword
        })

• The categories (previously parent) property no longer causes static properties to be mixed-in, the inheritance model is only used for classifications.
  • Instead explicitly mix-in static properties yourself:

        const Identifier = createToken({/* ... */}) 
        // static property "longer_alt" will not be mixed-in to TokenTypes "extending" it using
        // the categories property, the inheritance is only used for classifications.
        const Keyword = createToken({name: "Keyword", pattern: Lexer.NA, longer_alt: Identifier })
        const While = createToken({name: "While", pattern: /while/, categories: Keyword})
        
        // Before
        console.log(While.LONGER_ALT)  // "Identifier"
    
        // Now
        console.log(While.LONGER_ALT)  // "undefined"
    
       // Solution 1 - Be explicit:
       const While = createToken({
            name: "While", 
            pattern: /while/, 
            categories: Keyword, 
            longer_alt: Identifier
      })
    
      // Solution 2: Create a utility function
     function createKeywordToken(options) {
        options.categories = Keyword
        options.longer_alt = Identifier
        return createToken(options)
     }

Tokens as Classes

• The Token class no longer exists.

       // Before
       class Keyword extends Token {
          static PATTERN = Lexer.NA
       } 
  
      // Now
      class Keyword {
          static PATTERN = Lexer.NA
       } 

• The inheritance(prototype) chain is no longer automatically understood as parent/categories

  • Explicitly specify the categories instead:

        Class Keyword {
           static PATTERN = Lexer.NA
        }
    
        Class While {
           static PATTERN = /while/
           // explicitly specifying the categories (previously parent)
           static categories = keyword 
        }

[bd82]

Categories property

This replaces the deprecated parent property.
It can either be a single TokenType or an array of TokenTypes

interface ITokenConfig {
    categories?: TokenType | TokenType[]
}

This means multiple classifications are supported.
For example: the "true" literal in ECMA5 can be classified as both a Reserved Keyword
and a Literal.