alex 3.2.7 fails to parse definitions which can pass alex 3.2.6 #197
Comments
|
OK I deprecated 3.2.7 and made it unbuildable. We'll need some new test cases for this. |
|
@andreasabel Assuming this isn't a crazy bootstrapping issue, it looks like you made all the parser changes between 3.2.6 and 3.2.7. Would have any time to help debug this? |
|
@Ericson2314 : I started to have a look. Then I hit #195. |
|
Shrunk example: Phew, finally found the magic incantations to make bisection work: Result: d6653f3 is the first bad commit
|
|
Running info on the happy grammar, The problem is these lower rules don't have I think remembering to double up NUM and CHAR is hard, and all the more so when this case we would need to "unparse" the number, yet also preserve things like This makes makes we think we should not have a |
|
Er, @Ericson2314, I am already working on this. I just lack the repo rights to assign this to me so that you would know about it. |
In different contexts within Alex's surface syntax, something like "2340898" might be a string of characters or a number. The contexts are are only distinguished at the grammar level, not the token level, so this more or less (we could very layer-violation-y tricks) precludes lexing entire number literals. Instead of a number token, we have a digit token. This we treat as "sub-token", making a `DIGIT | CHAR` non-terminal we use everywhere we want to parse a character. For number literals, we just parse a non-empty string of numbers, and the left recursion makes the `* 10` elegant. Fixes #197
In different contexts within Alex's surface syntax, something like "2340898" might be a string of characters or a number. The contexts are are only distinguished at the grammar level, not the token level, so this more or less (we could very layer-violation-y tricks) precludes lexing entire number literals. Instead of a number token, we have a digit token. This we treat as "sub-token", making a `DIGIT | CHAR` non-terminal we use everywhere we want to parse a character. For number literals, we just parse a non-empty string of numbers, and the left recursion makes the `* 10` elegant. Fixes #197
Yes this sounds good. |
In different contexts within Alex's surface syntax, something like "2340898" might be a string of characters or a number. The contexts are are only distinguished at the grammar level, not the token level, so this more or less (we could very layer-violation-y tricks) precludes lexing entire number literals. Instead of a number token, we have a digit token. This we treat as "sub-token", making a `DIGIT | CHAR` non-terminal we use everywhere we want to parse a character. For number literals, we just parse a non-empty string of numbers, and the left recursion makes the `* 10` elegant. Fixes haskell#197
Oh sorry! That's awkward. I should have watched the time zones more carefully, I just assumed you stopped at the bisect. Yes, let's get you those perms. |
…ssions. In issue haskell#141, multiplicity annotations in regexes where extended to the general, multi-digit case {nnn,mmm}. However, lexing numeric literals broke parsing of regexes like: 32|64 [01-89] The solution here is to only lex numeric literals in a special lexer state called `multiplicity` which is entered by the parser when parsing multiplicity braces {nnn,mmm}. This restores alex' handling of digits as characters in the non-multiplicity situations.
|
I should have stated in text that I am looking into a fix... PR #202 seems to solve the problem in a satisfactory way, by restricting the parsing of numerals to inside multiplicity expressions |
|
@andreasabel BTW I earlier sent you an email taking the git email address. Is that a fine email address? I want to reach out to @simonmar more directly to get to permissions. |
I found the message. I am often slow with email, looking into my github notifications more often than my email. Thanks for your efforts! |
…ssions. In issue haskell#141, multiplicity annotations in regexes where extended to the general, multi-digit case {nnn,mmm}. However, lexing numeric literals broke parsing of regexes like: 32|64 [01-89] The solution here is to only lex numeric literals in a special lexer state called `multiplicity` which is entered by the parser when parsing multiplicity braces {nnn,mmm}. This restores alex' handling of digits as characters in the non-multiplicity situations.
…#202) In issue #141, multiplicity annotations in regexes where extended to the general, multi-digit case {nnn,mmm}. However, lexing numeric literals broke parsing of regexes like: 32|64 [01-89] The solution here is to only lex numeric literals in a special lexer state called `multiplicity` which is entered by the parser when parsing multiplicity braces {nnn,mmm}. This restores alex' handling of digits as characters in the non-multiplicity situations.
|
@Commelina and others, please try https://hackage.haskell.org/package/alex-3.2.7.1 |
I tried the latest alex 3.2.7 on a common-used library
language-c-0.9.0.1. However, it failed to parseLex.x:It seems that the problem happened here:
Then I rolled my alex back to version 3.2.6 then everything worked. I am not sure if it is a bug.
The text was updated successfully, but these errors were encountered: