New "createToken" helper functions.

[bd82]

The Old "extendToken" helpers were not suitable for handling multiple optional arguments.
So a config object is now in the new API.

// instead of:
extendToken("Integer", /\d+/, Literal)
// the new API is:
createToken(name: "Integer", pattern: /\d+/, parent: Literal)

The old APIs are still present but are considered deprecated and will probably be removed at (or before)
version 1.0.0

[bd82]

open question:
Use "pattern" or "PATTERN" (low/upper case) for the property names?

The problem is that when defining Tokens ESNext classes, the Chevrotain convention was to use
upper case names:

class Integer extends Token {
  static PATTERN = /\d+/
  static LABEL = "A whole number"
}

but when using helper functions with the config object it seems more idiomatic to use low case property names.

or perhaps we should(can?) be idiot proof for these case issues?

[bd82]

Use "pattern" or "PATTERN" (low/upper case) for the property names?

Being 100% tolerant to API mistakes in this case will just add unneeded complexity.
The common case is a user transitioning from ES5 "createToken" syntax to
ESNext classes with static properties.

In that case the static property names will have to be renamed to uppercase (pattern --> PATTERN).
There already exists a validation that checks for missing PATTERN when initializing a Lexer.
This means that the most common mistake will be caught and properly displayed to the user.

I do not believe additional validations are warranted at this time.

[bd82]

reopening to remind myself to modify the playground examples.