No viable alternative can be incorrectly thrown for start rules without explicit EOF

[sharwell]

If the start rule does not contain an explicit EOF transition, ParserATNSimulator.adaptivePredict may fail to return a viable alternative.

Grammar:

start : ID | ID INT ID;
ID : [a-z]+;
INT : [0-9]+;
WS : [ \t]+ -> skip;

Input: x 1

[sharwell]

I think the key insight here is we can create a "pseudo-rule" during ATN deserialization with the following form, where rule1, rule2, and rule3 are all the rules in the grammar. This can be reduced a bit by excluding rules from the set which end with an explicit EOF.

During SLL prediction, a configuration which steps out of the decision rule will have at least one configuration which ends up in the wildcard loop in the implicit follow rule.

Special care must be taken for the following:

1. Only choose a configuration in the implicit follow rule when no other alternatives are viable.
2. When all viable configurations are in the implicit follow rule, choose the alternative corresponding to the last alternative to enter the implicit follow rule.

[sharwell]

When the parser is configured to fall back to LL prediction in the event of SLL conflict, the "Special care" conditions listed above are implicitly handled by the full-context parsing algorithm.

[sathya2311]

I looked into the source code and found that the above is happening because of new error handling strategy introduced in ANTLR 4 as part of org.antlr.v4.runtime.DefaultErrorStrategy class.
Also the javadoc of the 'sync()' methods suggests it may have performance overhead and can be overridden. After overriding the sync() method EOF exception is not occurring and parser terminates as soon as it doesn't find a corresponding match. I think for people who wants to migrate to ANTLRv4 and doesn't need this feature then this is the best option.

[sharwell]

I believe you may have misinterpreted the problem described above. To make sure I understand what you are saying, here are the two parts to your comment that led me to this conclusion.

I looked into the source code and found that the above is happening because of new error handling strategy introduced in ANTLR 4 as part of DefaultErrorStrategy class.

By the time the error handler identifies a problem and attempts to report and/or recover from it, the prediction mechanism has already failed to return a correct prediction for a previous input.

parser terminates as soon as it doesn't find a corresponding match

This issue describes a case where the parser should match the input, but fails to do so. Suppressing the error message but failing to find a match would still be the wrong result.

[sathya2311]

@sharwell, We used Antlr3 for file parsing and java objects created out of it. Parsing is done partially so that one object is created at a time. Example grammar is as below

file : metadata record* endsummary

Object is created once a record is identified. After upgrading to Antlr4, it doesnt stop after finding one a record rather continues till EOF and prints error for every record from second instance of it. The error is thrown from DefaultErrorStrategy.sync() method.

I posted it because others may find it useful as well. Do you see any issue with this approach?

[sharwell]

Do you have a copy of the grammar and example input demonstrating the problem?

[Arpit2506]

Not able to pass double datatype getting error:
found: '40.715', expected: '}}'