Skip to content

Commit

Permalink
v4.3.3
Browse files Browse the repository at this point in the history
  • Loading branch information
bd82 committed Apr 12, 2019
1 parent dc9102f commit a2ff37a
Show file tree
Hide file tree
Showing 28 changed files with 87 additions and 87 deletions.
4 changes: 2 additions & 2 deletions examples/custom_apis/package.json
@@ -1,11 +1,11 @@
{
"name": "chevrotain_examples_custom_apis",
"version": "4.3.2",
"version": "4.3.3",
"scripts": {
"test": "mocha \"!(node_modules)/**/*_spec.js\""
},
"dependencies": {
"chevrotain": "^4.3.2"
"chevrotain": "^4.3.3"
},
"private": true
}
4 changes: 2 additions & 2 deletions examples/grammars/package.json
@@ -1,11 +1,11 @@
{
"name": "chevrotain_examples_grammars",
"version": "4.3.2",
"version": "4.3.3",
"scripts": {
"test": "mocha \"!(node_modules)/**/*spec.js\""
},
"dependencies": {
"chevrotain": "^4.3.2"
"chevrotain": "^4.3.3"
},
"devDependencies": {
"acorn": "^6.0.3",
Expand Down
4 changes: 2 additions & 2 deletions examples/implementation_languages/package.json
@@ -1,14 +1,14 @@
{
"name": "chevrotain_examples_implementation_languages",
"version": "4.3.2",
"version": "4.3.3",
"scripts": {
"build": "npm-run-all build:ts build:coffee",
"build:ts": "tsc ./typescript/typescript_json.ts --types \" \"",
"build:coffee": "coffee -c ./coffeescript/coffeescript_json.coffee",
"test": "mocha \"*spec.js\""
},
"dependencies": {
"chevrotain": "^4.3.2"
"chevrotain": "^4.3.3"
},
"devDependencies": {
"coffee-script": "^1.11.1",
Expand Down
4 changes: 2 additions & 2 deletions examples/lexer/package.json
@@ -1,11 +1,11 @@
{
"name": "chevrotain_examples_lexer",
"version": "4.3.2",
"version": "4.3.3",
"scripts": {
"test": "mocha \"!(node_modules)/**/*spec.js\""
},
"dependencies": {
"chevrotain": "^4.3.2",
"chevrotain": "^4.3.3",
"lodash": "~4.17.10"
},
"devDependencies": {
Expand Down
4 changes: 2 additions & 2 deletions examples/parser/package.json
@@ -1,6 +1,6 @@
{
"name": "chevrotain_examples_parser",
"version": "4.3.2",
"version": "4.3.3",
"scripts": {
"build": "npm-run-all build:minification build:webpack build:serialized-grammar",
"build:minification": "grunt --gruntfile minification/gruntfile.js",
Expand All @@ -9,7 +9,7 @@
"test": "mocha \"!(node_modules)/**/*spec.js\""
},
"dependencies": {
"chevrotain": "^4.3.2",
"chevrotain": "^4.3.3",
"lodash": "~4.17.10"
},
"devDependencies": {
Expand Down
4 changes: 2 additions & 2 deletions examples/tutorial/package.json
@@ -1,11 +1,11 @@
{
"name": "chevrotain_examples_tutorial",
"version": "4.3.2",
"version": "4.3.3",
"scripts": {
"test": "mocha \"step*/*spec.js\""
},
"dependencies": {
"chevrotain": "^4.3.2"
"chevrotain": "^4.3.3"
},
"devDependencies": {
"chai": "^4.1.2",
Expand Down
2 changes: 1 addition & 1 deletion lerna.json
Expand Up @@ -2,5 +2,5 @@
"packages": ["packages/*", "examples/*"],
"useWorkspaces": true,
"npmClient": "yarn",
"version": "4.3.2"
"version": "4.3.3"
}
2 changes: 1 addition & 1 deletion packages/chevrotain/docs/FAQ.md
Expand Up @@ -102,7 +102,7 @@ See [related documentation](https://github.com/SAP/chevrotain/blob/master/exampl

The first time a Chevrotain parser is initialized additional validations and computations are performed.
Some of these can take a very long time under certain edge cases. Specifically the detection of ambiguous alternatives
when the parser uses a larger than the default [maxLookahead](https://sap.github.io/chevrotain/documentation/4_3_2/interfaces/iparserconfig.html#maxlookahead)
when the parser uses a larger than the default [maxLookahead](https://sap.github.io/chevrotain/documentation/4_3_3/interfaces/iparserconfig.html#maxlookahead)
and there are many (thousands) of ambiguous paths.

To resolve this try reducing the maxLookahead and inspect the ambiguity errors to fix
Expand Down
2 changes: 1 addition & 1 deletion packages/chevrotain/docs/changes/CHANGELOG.md
@@ -1,4 +1,4 @@
## X.Y.Z (INSERT_DATE_HERE)
## 4.3.3 (4-12-2019)

#### Bug Fixes

Expand Down
12 changes: 6 additions & 6 deletions packages/chevrotain/docs/features/custom_errors.md
Expand Up @@ -3,8 +3,8 @@
Chevrotain allows users to customize both the parser and lexer error messages.
This can be accomplished by implementing the following interfaces:

- [IParserErrorMessageProvider](https://sap.github.io/chevrotain/documentation/4_3_2/interfaces/iparsererrormessageprovider.html)
- [ILexerErrorMessageProvider](https://sap.github.io/chevrotain/documentation/4_3_2/interfaces/ilexererrormessageprovider.html)
- [IParserErrorMessageProvider](https://sap.github.io/chevrotain/documentation/4_3_3/interfaces/iparsererrormessageprovider.html)
- [ILexerErrorMessageProvider](https://sap.github.io/chevrotain/documentation/4_3_3/interfaces/ilexererrormessageprovider.html)

See executable examples:

Expand All @@ -25,7 +25,7 @@ $.RULE("myStatement", () => {

The **ERR_MSG** config property is available for the following DSL methods:

- [CONSUME](https://sap.github.io/chevrotain/documentation/4_3_2/classes/parser.html#consume)
- [OR](https://sap.github.io/chevrotain/documentation/4_3_2/classes/parser.html#or)
- [AT_LEAST_ONE](https://sap.github.io/chevrotain/documentation/4_3_2/classes/parser.html#at_least_one)
- [AT_LEAST_ONE_SEP](https://sap.github.io/chevrotain/documentation/4_3_2/classes/parser.html#at_least_one_sep)
- [CONSUME](https://sap.github.io/chevrotain/documentation/4_3_3/classes/parser.html#consume)
- [OR](https://sap.github.io/chevrotain/documentation/4_3_3/classes/parser.html#or)
- [AT_LEAST_ONE](https://sap.github.io/chevrotain/documentation/4_3_3/classes/parser.html#at_least_one)
- [AT_LEAST_ONE_SEP](https://sap.github.io/chevrotain/documentation/4_3_3/classes/parser.html#at_least_one_sep)
4 changes: 2 additions & 2 deletions packages/chevrotain/docs/features/grammar_inheritance.md
Expand Up @@ -4,8 +4,8 @@ Chevrotain supports Grammar Inheritance, This is useful to represent multiple va
for example a grammar for ECMAScript 6 **extends** an ECMAScript 5.1 grammar.

Chevrotain Grammars are JavaScript classes, so Grammar inheritance is simpliy JavaScript inheritance
with the replacement of the [**RULE**](https://sap.github.io/chevrotain/documentation/4_3_2/classes/parser.html#rule)
DSL method with [**OVERRIDE_RULE**](https://sap.github.io/chevrotain/documentation/4_3_2/classes/parser.html#override_rule) method when needed.
with the replacement of the [**RULE**](https://sap.github.io/chevrotain/documentation/4_3_3/classes/parser.html#rule)
DSL method with [**OVERRIDE_RULE**](https://sap.github.io/chevrotain/documentation/4_3_3/classes/parser.html#override_rule) method when needed.

See [executable example](https://github.com/SAP/chevrotain/tree/master/examples/parser/inheritance)
for further details.
2 changes: 1 addition & 1 deletion packages/chevrotain/docs/features/position_tracking.md
Expand Up @@ -8,7 +8,7 @@ token_skipping
- start and end lines.
- start and end columns.

The level of position information tracking can be reduced by using the [**positionTracking**](https://sap.github.io/chevrotain/documentation/4_3_2/interfaces/ilexerconfig.html#positiontracking) lexer config option.
The level of position information tracking can be reduced by using the [**positionTracking**](https://sap.github.io/chevrotain/documentation/4_3_3/interfaces/ilexerconfig.html#positiontracking) lexer config option.
For example:

```javascript
Expand Down
2 changes: 1 addition & 1 deletion packages/chevrotain/docs/features/token_grouping.md
Expand Up @@ -3,7 +3,7 @@
Chevrotain lexers support grouping Tokens Types **separately** from the main token vector in the lexing result.
This is often useful to **collect** a specific set of Token Types for later processing, for example to collect comments tokens.

To group a Token Type simply specify the [**group**](https://sap.github.io/chevrotain/documentation/4_3_2/interfaces/itokenconfig.html#group) property in its configuration.
To group a Token Type simply specify the [**group**](https://sap.github.io/chevrotain/documentation/4_3_3/interfaces/itokenconfig.html#group) property in its configuration.
For example:

```javascript
Expand Down
2 changes: 1 addition & 1 deletion packages/chevrotain/docs/guide/concrete_syntax_tree.md
Expand Up @@ -27,7 +27,7 @@ There are two major differences.

How to enable CST output?

This feature is controlled by the **outputCst** flag of the parser [configuration object](https://sap.github.io/chevrotain/documentation/4_3_2/interfaces/iparserconfig.html).
This feature is controlled by the **outputCst** flag of the parser [configuration object](https://sap.github.io/chevrotain/documentation/4_3_3/interfaces/iparserconfig.html).

- Since 4.0 CST output is **enabled by default**.

Expand Down
40 changes: 20 additions & 20 deletions packages/chevrotain/docs/guide/custom_apis.md
Expand Up @@ -17,16 +17,16 @@ Skip to [runnable examples](https://github.com/SAP/chevrotain/tree/master/exampl

The structure of the GAST is made up of the following classes:

- [Rule](https://sap.github.io/chevrotain/documentation/4_3_2/classes/rule.html)
- [Terminal](https://sap.github.io/chevrotain/documentation/4_3_2/classes/terminal.html)
- [NonTerminal](https://sap.github.io/chevrotain/documentation/4_3_2/classes/nonterminal.html)
- [Alternation](https://sap.github.io/chevrotain/documentation/4_3_2/classes/alternation.html)
- [Option](https://sap.github.io/chevrotain/documentation/4_3_2/classes/option.html)
- [Repetition](https://sap.github.io/chevrotain/documentation/4_3_2/classes/repetition.html)
- [RepetitionWithSeparator](https://sap.github.io/chevrotain/documentation/4_3_2/classes/repetitionwithseparator.html)
- [RepetitionMandatory](https://sap.github.io/chevrotain/documentation/4_3_2/classes/repetitionmandatory.html)
- [RepetitionMandatoryWithSeparator](https://sap.github.io/chevrotain/documentation/4_3_2/classes/repetitionmandatorywithseparator.html)
- [Flat](https://sap.github.io/chevrotain/documentation/4_3_2/classes/flat.html) (sequence)
- [Rule](https://sap.github.io/chevrotain/documentation/4_3_3/classes/rule.html)
- [Terminal](https://sap.github.io/chevrotain/documentation/4_3_3/classes/terminal.html)
- [NonTerminal](https://sap.github.io/chevrotain/documentation/4_3_3/classes/nonterminal.html)
- [Alternation](https://sap.github.io/chevrotain/documentation/4_3_3/classes/alternation.html)
- [Option](https://sap.github.io/chevrotain/documentation/4_3_3/classes/option.html)
- [Repetition](https://sap.github.io/chevrotain/documentation/4_3_3/classes/repetition.html)
- [RepetitionWithSeparator](https://sap.github.io/chevrotain/documentation/4_3_3/classes/repetitionwithseparator.html)
- [RepetitionMandatory](https://sap.github.io/chevrotain/documentation/4_3_3/classes/repetitionmandatory.html)
- [RepetitionMandatoryWithSeparator](https://sap.github.io/chevrotain/documentation/4_3_3/classes/repetitionmandatorywithseparator.html)
- [Flat](https://sap.github.io/chevrotain/documentation/4_3_3/classes/flat.html) (sequence)

For example to define a grammar rule for a fully qualified name:

Expand Down Expand Up @@ -94,7 +94,7 @@ Important to note that:

Chevrotain exposes three functions for Grammar(GAST) Validation purposes:

1. [**resolveGrammar**](https://sap.github.io/chevrotain/documentation/4_3_2/globals.html#resolvegrammar)
1. [**resolveGrammar**](https://sap.github.io/chevrotain/documentation/4_3_3/globals.html#resolvegrammar)
NonTerminals are often referenced by their name as cyclic references will make
a direct object reference impossible, for example with right recursion:

Expand All @@ -107,25 +107,25 @@ Chevrotain exposes three functions for Grammar(GAST) Validation purposes:

For any reference that cannot be resolved an error object will be outputted,
this object will contain an error message which can be customized by providing
a custom implementation of [IGrammarResolverErrorMessageProvider](https://sap.github.io/chevrotain/documentation/4_3_2/interfaces/igrammarresolvererrormessageprovider.html).
The [default implementation](https://sap.github.io/chevrotain/documentation/4_3_2/globals.html#defaultgrammarresolvererrorprovider) also exported as part of the public API.
a custom implementation of [IGrammarResolverErrorMessageProvider](https://sap.github.io/chevrotain/documentation/4_3_3/interfaces/igrammarresolvererrormessageprovider.html).
The [default implementation](https://sap.github.io/chevrotain/documentation/4_3_3/globals.html#defaultgrammarresolvererrorprovider) also exported as part of the public API.

1) [**validateGrammar**](https://sap.github.io/chevrotain/documentation/4_3_2/globals.html#validategrammar)
1) [**validateGrammar**](https://sap.github.io/chevrotain/documentation/4_3_3/globals.html#validategrammar)
Chevrotain expose a set of checks on the grammar structure that it is **highly recommended** to execute.
These checks will detect ambiguous alternatives, left recursion, conflicting Terminals & NonTerminal names and more...

validateGrammar is side effect free and like resolveGrammar will return an array of error objects.
The error messages in these objects can once again be customized by providing a [IGrammarResolverErrorMessageProvider](https://sap.github.io/chevrotain/documentation/4_3_2/interfaces/igrammarvalidatorerrormessageprovider.html)
optionally based on the [default implementation](https://sap.github.io/chevrotain/documentation/4_3_2/globals.html#defaultgrammarvalidatorerrorprovider)
The error messages in these objects can once again be customized by providing a [IGrammarResolverErrorMessageProvider](https://sap.github.io/chevrotain/documentation/4_3_3/interfaces/igrammarvalidatorerrormessageprovider.html)
optionally based on the [default implementation](https://sap.github.io/chevrotain/documentation/4_3_3/globals.html#defaultgrammarvalidatorerrorprovider)

1) [**assignOccurrenceIndices**](https://sap.github.io/chevrotain/documentation/4_3_2/globals.html#assignoccurrenceindices)
1) [**assignOccurrenceIndices**](https://sap.github.io/chevrotain/documentation/4_3_3/globals.html#assignoccurrenceindices)
Chevrotain has certain constraint on the "shape" of the generated code. The relevant one in this case is the [unique numerical suffixes](https://sap.github.io/chevrotain/docs/FAQ.html#NUMERICAL_SUFFIXES) for the DSL methods.
The assignOccurrenceIndices function will take care of this requirement by mutating the idx property on the GAST classes accordingly.

### A Note on Custom Error Messages

As mentioned earlier **validateGrammar** **assignOccurrenceIndices** optionally accept
a [errMsgProvider](https://sap.github.io/chevrotain/documentation/4_3_2/globals.html#validategrammar) option.
a [errMsgProvider](https://sap.github.io/chevrotain/documentation/4_3_3/globals.html#validategrammar) option.
Note that all custom error message builders receive a GAST instance as input. What this means is that
the GAST classes created by the implementor of the custom API can be augmented with additional information to enable
producing better error messages. e.g:
Expand All @@ -140,14 +140,14 @@ producing better error messages. e.g:

There are two APIs for code generation and execution.

- [**generateParserModule**](https://sap.github.io/chevrotain/documentation/4_3_2/globals.html#generateparsermodule)
- [**generateParserModule**](https://sap.github.io/chevrotain/documentation/4_3_3/globals.html#generateparsermodule)
This will generate the string literal of a [UMD](https://github.com/umdjs/umd) module.
This UMD pattern is consumable in all standard JS runtimes & module implementations.
The approach is to generate the string literal and write it to a file for later consumption.
However for development purposes or under certain runtimes it is possible to skip the file creation
For example see the [require-from-string](https://github.com/floatdrop/require-from-string)

- [**generateParserFactory**](https://sap.github.io/chevrotain/documentation/4_3_2/globals.html#generateparserfactory)
- [**generateParserFactory**](https://sap.github.io/chevrotain/documentation/4_3_3/globals.html#generateparserfactory)
This API skips string literal and directly evals (new Function(...)) the code and returns
a factory that can be used to create Parser instances.

Expand Down
6 changes: 3 additions & 3 deletions packages/chevrotain/docs/guide/custom_token_patterns.md
Expand Up @@ -31,7 +31,7 @@ function. But with a small constraint.
- A custom pattern should behave as though the RegExp [sticky flag](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/RegExp/sticky) has been set.
This means that attempted matches must begin at the offset argument, **not** at the start of the input.

The basic syntax for supplying a custom pattern is defined by the [ICustomPattern](https://sap.github.io/chevrotain/documentation/4_3_2/interfaces/icustompattern.html) interface.
The basic syntax for supplying a custom pattern is defined by the [ICustomPattern](https://sap.github.io/chevrotain/documentation/4_3_3/interfaces/icustompattern.html) interface.
Example:

```javascript
Expand Down Expand Up @@ -59,7 +59,7 @@ createToken({
pattern: { exec: matchInteger },

// Optional properrty that will enable optimizations in the lexer
// See: https://sap.github.io/chevrotain/documentation/4_3_2/interfaces/itokenconfig.html#start_chars_hint
// See: https://sap.github.io/chevrotain/documentation/4_3_3/interfaces/itokenconfig.html#start_chars_hint
start_chars_hint: ["1", "2", "3", "4", "5", "6", "7", "8", "9"]
})
```
Expand Down Expand Up @@ -96,6 +96,6 @@ function matchInteger(text, offset, matchedTokens, groups) {

A larger and non contrived example can seen here: [Lexing Python like indentation using Chevrotain](https://github.com/SAP/chevrotain/blob/master/examples/lexer/python_indentation/python_indentation.js).

It is important to note that The matchedTokens and groups arguments match the token and groups properties of the tokenize output ([ILexingResult](https://sap.github.io/chevrotain/documentation/4_3_2/interfaces/ilexingresult.html)).
It is important to note that The matchedTokens and groups arguments match the token and groups properties of the tokenize output ([ILexingResult](https://sap.github.io/chevrotain/documentation/4_3_3/interfaces/ilexingresult.html)).
These arguments are the current state of the lexing result so even if the lexer has performed error recovery any tokens found
in those arguments are still guaranteed to be in the final result.
Expand Up @@ -19,7 +19,7 @@ library by @tabatkins.

## Instructions

Chevrotain provides the [**createSyntaxDiagramsCode**](https://sap.github.io/chevrotain/documentation/4_3_2/globals.html#createsyntaxdiagramscode) API to generate the **html source code**
Chevrotain provides the [**createSyntaxDiagramsCode**](https://sap.github.io/chevrotain/documentation/4_3_3/globals.html#createsyntaxdiagramscode) API to generate the **html source code**
of syntax diagrams. This html source code can then be used by an end user in either node.js or a browser:

1. By writing it directly to the disk in a pure node.js runtime scenario.
Expand Down
4 changes: 2 additions & 2 deletions packages/chevrotain/docs/guide/performance.md
Expand Up @@ -30,7 +30,7 @@ many computations required for its execution, This cache is kept on the instance
So creating a new Parser instance for each input would lose all advantages of this cache.

Note that this means that if your parser "carries" additional state, that state should also be reset.
Simply override the Parser's [reset](https://sap.github.io/chevrotain/documentation/4_3_2/classes/parser.html#reset) method
Simply override the Parser's [reset](https://sap.github.io/chevrotain/documentation/4_3_3/classes/parser.html#reset) method
to accomplish that.

## Lexer Optimizations
Expand Down Expand Up @@ -186,7 +186,7 @@ It is important to note that:
These are only required if you are trying to squeeze every tiny bit of performance out of your parser.

1. **Reduce the amount of Token position tracking** the lexer performs.
See The [ILexerConfig.positionTracking](https://sap.github.io/chevrotain/documentation/4_3_2/interfaces/ilexerconfig.html) property.
See The [ILexerConfig.positionTracking](https://sap.github.io/chevrotain/documentation/4_3_3/interfaces/ilexerconfig.html) property.

2. **Avoid creating parsing rules which only parse a single Terminal.**

Expand Down

0 comments on commit a2ff37a

Please sign in to comment.