CSharp and Java producing different error messages for expected tokens

[kaby76]

For:

• Grammar is grammars-v4/csharp.
• Antlr4.10.1 Java and CSharp targets.
• Uses the standard console error listener.
• You can use trgen to generate for either CSharp or Java target on Windows or Linux.

Output:

• We're getting different lookahead sets for what to expect next for "mismatched input ':' expecting" between Java and CSharp.
• See [CSharp] Parsing source with syntax error causes ClassCastExceptin grammars-v4#2612 (comment)

Expected output:

• There should not be any diff in the error message between targets.

[kaby76]

This needs to be solved since the build compares outputs--and soon the .tree files--across targets.

[ericvergnaud]

This could be solved by converting the set to a naturally ordered list.

[kaby76]

This could be solved by converting the set to a naturally ordered list.

I think there's more to it because the LA sets are not the same at all between CSharp and Java targets.

# remastered for the CSharp target, testing the Java target.
diff --git a/csharp/examples/issue-2612.txt.errors b/csharp/examples/issue-2612.txt.errors
index 3c03c4a0..6832b6ca 100644
--- a/csharp/examples/issue-2612.txt.errors
+++ b/csharp/examples/issue-2612.txt.errors
@@ -1 +1 @@
-line 4:17 mismatched input ':' expecting ';'
+line 4:17 mismatched input ':' expecting {'add', 'alias', '__arglist', 'ascending', 'async', 'await', 'by', 'descending', 'dynamic', 'equals', 'from', 'get', 'group', 'into', 'join', 'let', 'nameof', 'on', 'orderby', 'partial', 'remove', 'select', 'set', 'unmanaged', 'var', 'when', 'where', 'yield', IDENTIFIER, '[', '*', '?'}
Test failed.

[ericvergnaud]

That bug is strange because many tests in the test suite check for errors...
Can you add it to the 'ParserErrors' test suite ? Then we can check it against all targets ?

[kaby76]

This is an old bug, but I am working out why there are differences because someone is trying to use grammars-v4/csharp (CSharp target) in StackOverflow (https://stackoverflow.com/questions/78424669/linking-visitmember-access-and-visitifstatement-methods-of-antlr4#comment138303496_78424669). (I created the desc.xml to describe what ports work for a grammar, and to try to prevent people from using bad ports. CSharp is not listed, but people ignore that anyway.)

The CSharp code and Java code error recovery are not similar.

antlr4/runtime/CSharp/src/DefaultErrorStrategy.cs

Lines 573 to 574 in 380ce4b

	// even that didn't work; must throw the exception
	throw new InputMismatchException(recognizer);

antlr4/runtime/Java/src/org/antlr/v4/runtime/DefaultErrorStrategy.java

Lines 480 to 488 in 380ce4b

	// even that didn't work; must throw the exception
	InputMismatchException e;
	if (nextTokensContext == null) {
	e = new InputMismatchException(recognizer);
	} else {
	e = new InputMismatchException(recognizer, nextTokensState, nextTokensContext);
	}
	throw e;

"nextTokensContext" is not null, so Java does something completely different from CSharp.

In fact, the code is not similar across targets.

antlr4/runtime/Cpp/runtime/src/DefaultErrorStrategy.cpp

Lines 209 to 210 in 380ce4b

	// Even that didn't work; must throw the exception.
	throw InputMismatchException(recognizer);

antlr4/runtime/Dart/lib/src/error/src/error_strategy.dart

Lines 546 to 558 in 380ce4b

	// even that didn't work; must throw the exception
	InputMismatchException e;
	if (nextTokensContext == null) {
	e = InputMismatchException(recognizer);
	} else {
	e = InputMismatchException(
	recognizer,
	nextTokensState!,
	nextTokensContext,
	);
	}
	throw e;

antlr4/runtime/Go/antlr/v4/error_strategy.go

Lines 364 to 365 in 380ce4b

	// even that didn't work must panic the exception
	recognizer.SetError(NewInputMisMatchException(recognizer))

antlr4/runtime/JavaScript/src/antlr4/error/DefaultErrorStrategy.js

Lines 424 to 425 in 380ce4b

	// even that didn't work; must throw the exception
	throw new InputMismatchException(recognizer);

antlr4/runtime/Python3/src/antlr4/error/ErrorStrategy.py

Lines 406 to 407 in 380ce4b

	# even that didn't work; must throw the exception
	raise InputMismatchException(recognizer)

It seems the change in Java was for #1922 0803c74 but it wasn't applied across targets and they were available. https://github.com/antlr/antlr4/tree/0803c74eb255fe5e6fe5a6f11d879198d305e715/runtime . Sloppy coding.