Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unexpected exception encountered in JFlex: Not normalised type = STAR #888

Closed
AmyD97 opened this issue Jan 31, 2021 · 10 comments · Fixed by #996
Closed

Unexpected exception encountered in JFlex: Not normalised type = STAR #888

AmyD97 opened this issue Jan 31, 2021 · 10 comments · Fixed by #996
Labels
bug Not working as intended
Milestone

Comments

@AmyD97
Copy link

AmyD97 commented Jan 31, 2021

I received this exception while trying to run JFlex to create a Scheme lexical analyzer for my Programming Languages course. I have never done this before, and I had never used JFlex before so this is probably a mistake I made. I am still following the instructions to try to get some help. Any help would be highly appreciated. Thanks in advance!

"Reading "Scheme.jflex"

Unexpected exception encountered. This indicates a bug in JFlex.
Please consider filing an issue at http://github.com/jflex-de/jflex/issues/new

Not normalised type = STAR
content :
type = PRIMCLASS
content :
{ [' ']['"']['('-')']['']['^']['|'] }
jflex.exceptions.CharClassException: Not normalised type = STAR
content :
type = PRIMCLASS
content :
{ [' ']['"']['('-')']['']['^']['|'] }
at jflex.core.RegExp.checkPrimClass(RegExp.java:242)
at jflex.core.RegExp.normalise(RegExp.java:323)
at jflex.core.RegExp.normalise(RegExp.java:353)
at jflex.core.RegExps.normalise(RegExps.java:293)
at jflex.core.LexParse$CUP$LexParse$actions.CUP$LexParse$do_action_part00000000(LexParse.java:1029)
at jflex.core.LexParse$CUP$LexParse$actions.CUP$LexParse$do_action(LexParse.java:2257)
at jflex.core.LexParse.do_action(LexParse.java:598)
at java_cup.runtime.lr_parser.parse(lr_parser.java:699)
at jflex.generator.LexGenerator.generate(LexGenerator.java:74)
at jflex.Main.generate(Main.java:320)
at jflex.Main.main(Main.java:336)"

@regisd
Copy link
Member

regisd commented Jan 31, 2021

Thanks for the report. Could you share a minimal flex spec that reproduces the issue?

@AmyD97
Copy link
Author

AmyD97 commented Jan 31, 2021 via email

@regisd
Copy link
Member

regisd commented Jan 31, 2021

In particular, it's a bit surprising to see an empty interval [''].
I still think JFlex should output a more descriptive error message when this happens.

@AmyD97
Copy link
Author

AmyD97 commented Jan 31, 2021 via email

@AmyD97
Copy link
Author

AmyD97 commented Jan 31, 2021 via email

@regisd
Copy link
Member

regisd commented Jan 31, 2021

Yes, it would be helpful. Unfortunately, the attachment by email was not handled. I think you can attach a file in the web UI simply by dragging in the comment section.

@AmyD97
Copy link
Author

AmyD97 commented Jan 31, 2021 via email

@AmyD97
Copy link
Author

AmyD97 commented Jan 31, 2021

Again this file has multiple errors.

Scheme.jflex.txt

@lsf37
Copy link
Member

lsf37 commented Nov 10, 2022

This does still look like a potential bug, because even for strange specs JFlex shouldn't just throw an exception. For easier discussion later, here the file again:

package grammar;
import java.io.*;

%class SchemeLexicalAnalyzer

%%

Letter			=	[A-Za-z]
Digit			=	[0-9]
Initial			=	[ {Letter} | "!" | "$" | "%" | "&" | "*" | "/" | ":" | "<" | "=" | ">" | "?" | "~" | "_" | "^" ]
Subsequent		=	[ {Initial} | {Digit} | "." | "+" | "-" ]
Identifier		=	{Initial} [{Subsequent}* | "+" | "-" | "..."]
Boolean			=	[ #t | #f ]
/*
Num2			=	[0-1]["#b"]
Num8
Num10			=	{Digit}*
Num16			
Number			=	[ {Num2} | {Num8} | {Num10} | {Num16} ] 
*/
Number 			= 	[{Digit}]
StringChar		= 	"\"" | ("\\") | [^"\\]
String			=	{StringChar}*
Whitespace		= 	[\ |\n|\t]
Character		=	[{Whitespace} | {StringChar}]
Constant		= 	[ {Boolean} | {Number} | {Character} | {String} ]
Keyword			=	[ "and" | "begin" | "case" | "cond" | "define" | "delay" | "if" | "lambda" | "let" | "let*" | "let-syntax" | "letrec-syntax" | "or" | "quasiquote" | "quote" | "set!" ]


%%


{Identifier}	{ return System.out.printf("IDENTIFIER %s", yytext()); }
{Boolean}		{ return System.out.printf("BOOLEAN %s", yytext()); }
{Number}		{ return System.out.printf("NUMBER %s", yytext()); }
{String}		{ return System.out.printf("STRING %s", yytext()); }
{Whitespace}	{  } // do nothing
{Character}		{ return System.out.printf("CHARACTER %s", yytext()); }
{Constant}		{ return System.out.printf("CONSTANT %s", yytext()); }
{Keyword}		{ return System.out.printf("KEYWORD %s", yytext()); }
"("				{ return System.out.printf("LEFTPARENTHESIS %s", yytext()); }
")" 			{ return System.out.printf("RIGHTPARETHESIS %s", yytext()); }
"." 			{ return System.out.printf("DOT %s", yytext()); }
"#(" 			{ return System.out.printf("OPENVECTOR %s", yytext()); }
"'" 			{ return System.out.printf("QUOTE %s", yytext()); }
"`" 			{ return System.out.printf("QUOTATION %s", yytext()); }

@lsf37 lsf37 added the bug Not working as intended label Nov 10, 2022
@lsf37 lsf37 added this to the 1.9.0 milestone Dec 30, 2022
@lsf37
Copy link
Member

lsf37 commented Jan 1, 2023

While there are some other problems in the spec, we should at least get a useful error message, and in fact most of the variations here would actually make sense. Here are some minimised specs that lead to an exception that could/should produce valid code:

%%
M = a
%%
[ {M} ]	 {  }
%%
M = "a"
%%
[ {M} ]	 {  }
%%
M = [a] | [b]
%%
[ {M} ]	 {  }

lsf37 added a commit that referenced this issue Jan 1, 2023
Because we allow macros in the syntax for char class contents, we can
get arbitrary regular expressions inside, not just char class content
expressions.

This commit turns the previous "cannot happen" check for such a case
into proper error reporting.

Fixes #888 and #939
lsf37 added a commit that referenced this issue Jan 1, 2023
Because we allow macros in the syntax for char class contents, we can
get arbitrary regular expressions inside, not just char class content
expressions.

This commit turns the previous "cannot happen" check for such a case
into proper error reporting.

Fixes #888 and #939
lsf37 added a commit that referenced this issue Jan 1, 2023
Because we allow macros in the syntax for char class contents, we can
get arbitrary regular expressions inside, not just char class content
expressions.

This commit turns the previous "cannot happen" check for such a case
into proper error reporting.

Fixes #888 and #939
lsf37 added a commit that referenced this issue Jan 1, 2023
lsf37 added a commit that referenced this issue Jan 1, 2023
Because we allow macros in the syntax for char class contents, we can
get arbitrary regular expressions inside, not just char class content
expressions.

This commit turns the previous "cannot happen" check for such a case
into proper error reporting.

Fixes #888 and #939
lsf37 added a commit that referenced this issue Jan 1, 2023
Signed-off-by: Gerwin Klein <lsf@jflex.de>
lsf37 added a commit that referenced this issue Jan 1, 2023
Because we allow macros in the syntax for char class contents, we can
get arbitrary regular expressions inside, not just char class content
expressions.

This commit turns the previous "cannot happen" check for such a case
into proper error reporting.

Fixes #888 and #939

Signed-off-by: Gerwin Klein <lsf@jflex.de>
lsf37 added a commit that referenced this issue Jan 1, 2023
lsf37 added a commit that referenced this issue Jan 1, 2023
Because we allow macros in the syntax for char class contents, we can
get arbitrary regular expressions inside, not just char class content
expressions.

This commit turns the previous "cannot happen" check for such a case
into proper error reporting.

Fixes #888 and #939
lsf37 added a commit that referenced this issue Jan 1, 2023
@lsf37 lsf37 closed this as completed in 321d931 Jan 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Not working as intended
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants