Correction to the ASI generation #18
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Drop the extraneous auto_semi rule, as it turns out that rule is **never** hit due to it being effectively replicates the unmatched production condition (through auto_semi then error). - However, on the case where the SEMI is produced through the error handler after an error condition that is actually an error, if the production stack is just that freshly generated SEMI, it will be immediately consumed and used by the empty_statement production rule, then the previously erroring token is yielded again, restarting the cycle thus that's the real cause behind the infinite loop as reported in rspivak/slimit#29. - However, if the auto generated SEMI tokens are correctly marked, and update the existing rules to make use of that token, the immediate consumption of the token cannot happen if an empty statement does NOT include the auto generated SEMI token (which has been done by creating a new AUTOSEMI token type), the condition which the immediate consumption of a token to result in the error token can no longer occur. - Removed the seen before token hack. - Moved all the conditions for which AUTOSEMI token is generated to the lexer auto_semi method. - This also completely eliminates the incorrect production of statements that only has SEMIs but not the ASI equivalent, such as the case with ``if`` or ``for`` statements as per 7.9.2 of the ECMA-262 5.1 spec, which is reported in rspivak/slimit#101.
- Also include a reference to the previous fix.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Turns out the production rules in the parser relating to ASI (with
auto_semi) were implemented incorrectly. The correct way is to create a specific token for this purpose and update those production rules to consume this token, so that no incorrect production from the confusion of this generated token versus the actual tokens can occur. Through this the actual generation relating to this can be cleaned up and the hack that stops the infinite loop for the ASI generation may be removed. Also this means that the incorrect production of statements that require a semi-colon will no longer occur (e.g.(function(){if (true)});should now result in a parsing error).This fixes rspivak/slimit#29 for real and rspivak/slimit#101.