Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

7bit switch doesn't work but 8bit and unicode does #756

Closed
vinhtq115 opened this issue Apr 26, 2020 · 3 comments · Fixed by #758
Closed

7bit switch doesn't work but 8bit and unicode does #756

vinhtq115 opened this issue Apr 26, 2020 · 3 comments · Fixed by #758
Assignees
Labels
bug
Projects
Milestone

Comments

@vinhtq115
Copy link

@vinhtq115 vinhtq115 commented Apr 26, 2020

I'm trying to generate a specification for a language using 7bit switch but JFlex won't generate anything. It always says:

Reading "vc.flex"
0 errors, 0 warnings.

Generation aborted.

When I use 8bit or unicode switch, it works fine.

Reading "vc.flex"
Constructing NFA : 276 states in NFA
Converting NFA to DFA :
...............................................................................
.......................................................
138 states before minimization, 121 states in minimized DFA
Writing code to "Scanner.java"
0 errors, 0 warnings.

Generation finished successfully.

I'm using JFlex 1.8.1.

@lsf37
Copy link
Member

@lsf37 lsf37 commented Apr 26, 2020

Hi @vinhtq115, thanks for the report. Could you append the lex spec or a shortened version of it that has the same symptom?

@vinhtq115
Copy link
Author

@vinhtq115 vinhtq115 commented Apr 26, 2020

Hi @vinhtq115, thanks for the report. Could you append the lex spec or a shortened version of it that has the same symptom?

Here you go:

vc.flex.txt
(Please remove txt extension)

@lsf37
Copy link
Member

@lsf37 lsf37 commented Apr 27, 2020

Thanks for that, I can reproduce the problem and confirm this is a bug. Will look into it.

For now, I'd recommend using %unicode (or nothing, since unicode is default) instead. I don't know if that is the case for your setting, but often, even if correct input is only 7 or 8 bit, incorrect input could still contain other characters. If that happens in the 7 or 8 bit scanner, you'll get an exception that you need to handle outside the scanner, whereas in the unicode scanner, you can write a default rule for unmatched characters that deal with the error case inside the scanner spec.

@lsf37 lsf37 added the bug label Apr 27, 2020
@lsf37 lsf37 self-assigned this Apr 27, 2020
lsf37 added a commit that referenced this issue May 3, 2020
Currently fails in scanner generation (#756)
lsf37 added a commit that referenced this issue May 3, 2020
Previously, unicode char class initialisation was skipped for `%7bit`
scanners, which after a previous refactor meant that initialisation was
skipped completely.

Fixes #756
lsf37 added a commit that referenced this issue May 3, 2020
Previously, unicode char class initialisation was skipped for `%7bit`
scanners, which after a previous refactor meant that initialisation was
skipped completely.

Fixes #756
@lsf37 lsf37 closed this as completed in #758 May 3, 2020
lsf37 added a commit that referenced this issue May 3, 2020
Currently fails in scanner generation (#756)
lsf37 added a commit that referenced this issue May 3, 2020
Previously, unicode char class initialisation was skipped for `%7bit`
scanners, which after a previous refactor meant that initialisation was
skipped completely.

Fixes #756
regisd pushed a commit that referenced this issue May 3, 2020
commit bb37e0d
Author:     Gerwin Klein <lsf@jflex.de>
AuthorDate: Sun May 3 10:39:20 2020 +0800
Commit:     Gerwin Klein <lsf37@doclsf.de>
CommitDate: Sun May 3 11:49:55 2020 +0800

    initialise char classes on all paths

    Previously, unicode char class initialisation was skipped for `%7bit`
    scanners, which after a previous refactor meant that initialisation was
    skipped completely.

    Fixes #756

Updated from target/jflex-parent-1.8.2-SNAPSHOT-sources.jar
@regisd regisd added this to Open bugs in JFlex core via automation Dec 18, 2020
@regisd regisd added this to the 1.9.0 milestone Dec 18, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug
Projects
JFlex core
  
Open bugs
Development

Successfully merging a pull request may close this issue.

3 participants