Using explicit rule for empty production seems to cause parse fail in some cases.

[SimonSntPeter]

Mostly copied from https://groups.google.com/g/antlr-discussion/c/AnG5BA-OrTw.

(github won't allow preview or text alterations (markdown?) so slight cleanup and hope for the best)

Grammar is at end. It's hugely minimised from an original sql-like language.

Essentially a 'select' statement takes an optional scaffold part,
followed by an optional 'with' statement,
followed by the 'select' statement itself.
These have been reduced down to single lexemes here for simplicity.

Optionality is done with an "empty_production" rule, which I prefer the explicitness of over just '... | ;'. It should not matter.

So with this grammar, with this:

scaffold
select
;

It parses ok, but comment out the 'scaffold':

--scaffold
select
;

and you get:

line 2:0 mismatched input 'select' expecting {'with', 'scaffold'}
Stack overflow.

Scaffold is optional, so why the problem?

Odder yet (to me), if you add a statement with a scaffold above that statement without a scaffold (which failed previously), it then succeeds overall:

scaffold
select
;

--scaffold <<< this failed before
select
;

Which parses ok.

From playing about it seems the optionality of the 'with' statement interferes with the optionality of the 'scaffold' statement, but that's just an impression.
AIUI they should not interfere because there's no actual ambiguity in the grammar.

I'm using the AntlrVSIX plugin for visual studio, version 8.3, which reports the Antlr parser version as 4.9.

grammar LDB;

start_parse returns [LDBitems ldbis] :
siX = ldb_items
EOF
;

ldb_items :
(
sisX += select_statement
SEMICOLON
) +
;

select_statement :
sns = opt_set_name_scaffold
wctec = opt_with_CTEs_clause
qe = query_expression
;

opt_set_name_scaffold :
SCAFFOLD
|
empty_production
;

opt_with_CTEs_clause :
WITH
|
empty_production
;

query_expression :
SELECT
;

empty_production : ;

SELECT : 'select' ;
WITH : 'with' ;
SCAFFOLD : 'scaffold' ;

SEMICOLON : ';' ;

fragment WUnl : ( '\r' ? ) '\n' ;

SLCOMMENT : ( '--' .*? WUnl ) -> skip ;

fragment ALLWSes : [ \t\r\n]+ ;

SKIPWS : ALLWSes -> skip ;

[SimonSntPeter]

Per digging by Eric Vergnaud this was tracked down to a difference between the runtime and compile time environments, essentially the Antlr4.Runtime dll was version 4.8 whereas it should have been 4.9.2 because that's what the visual studio plugin was using.
Downloading the Antlr4.Runtime dll version 4.9.2 and dropping that in the right directory seems to have fixed the problem.
Why this discrepancy has happened is not quite clear yet, will add that info and close the issue shortly.

[SimonSntPeter]

With the help of Ken Domino this was tracked down further to a .csproj file referring to an old version of the Antlr runtime (4.8.0), something like

The .csproj had migrated with the project when an OS and VS2019 reinstall was done. I wasn't aware it needed updating.

Ken Domino advised on how to fix this (see thread) with a package update so this now is

This now works.

Perhaps it would be possible for this kind of issue to be detected by having the generated code call the runtime for a version and loudly bail if it's not matching?

[ericvergnaud]

The code already does that. I think the core issue was fixed between 4.7 and 4.9. Envoyé de mon iPhone

…

Le 3 juin 2021 à 16:28, SimonSntPeter ***@***.***> a écrit :  With the help of Ken Domino this was tracked down further to a .csproj file referring to an old version of the Antlr runtime (4.8.0), something like The .csproj had migrated with the project when an OS and VS2019 reinstall was done. I wasn't aware it needed updating. Ken Domino advised on how to fix this (see thread) with a package update so this now is This now works. Perhaps it would be possible for this kind of issue to be detected by having the generated code call the runtime for a version and loudly bail if it's not matching? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[SimonSntPeter]

My install was clearly discrepant between the 2 components, version 4.8 runtime and 4.9 for the build stuff, but no alert of any kind was visible to me.
I still have the old 4.8 DLL lying around, I can try and replicate if you wish. If not, feel free to close.

jan