Add visitor tests for Java runtime API

[sharwell]

This is a re-implementation of #1566 targeting the Java runtime API specifically. Other targets may follow with their own equivalent implementations.

[parrt]

@antlr/antlr-targets note for testing api libs i'm leaning towards "wild wild west" whereby each target, that owns everything under their runtime dir, could use the test rigs / libs (analogous to junit) to mimic the tests done by ref impl. E.g., java in the runtime-testsuite now has runtime-testsuite/test/org/antlr/v4/test/runtime/java/api/TestExpectedTokens.java and of course what Sam just added.

[hanjoes]

@parrt
Does it make sense to add a "testAPI/testLib" method to BaseRuntimeTest that handles Descriptors for API test? It seems better for target like Swift since we can use the same mechanism for testing API as Parser/Lexer and naturally makes the API tests a part of the "mvn test". Although we could do it only for Swift, but it feels quite general.

[sharwell]

@hanjoes My recommendation would be to create a unit test project in the target language, and run the tests there instead of under Maven. In the TypeScript and optimized C# target, this strategy led to a 20x improvement in the time it takes to run the complete test suite. Currently each JUnit test creates, compiles, and runs a Swift project. With the strategy I've moved towards in my targets, each JUnit test generates code from the grammar and emits a copy of the test (input, expected output, expected errors, etc.) in the target language. After all JUnit tests are run, I have a complete copy of the runtime test suite in TypeScript or C# form. I then compile everything at once and then run it under mocha (TypeScript) or MSTest (C#) to execute and verify the results.

The new strategy would leave you with a skeleton Swift test project where you could add versions of the API tests without relying on the very slow JUnit approach.

The pull requests for my targets still use the old test template approach, but the strategy turned out to be quite simple to put in place so it may be easy to use a similar one here:

• TypeScript: Mocha runtime tests tunnelvisionlabs/antlr4ts#244
• C#: Rewrite the runtime test suite so it's fast enough to use tunnelvisionlabs/antlr4cs#169

[hanjoes]

@sharwell Make sense, I think we have the similar plan. I was just asking for an entrance from the current JUnit test mechanism that can let us enter the Swift world. And by using that entrance we can still use the current JUnit test rig while run the whole thing as one Swift project. That "entrance" seems quite general so it might make sense to add it to the higher level packages. But for now I will work only under the swift package.

[mike-lischke]

For the C++ runtime I also have a number of unit tests written in a native project (with its test integration in the IDE). These tests target only functionality that are platform specific (like does the implementation of a class work as expected). The only drawback is that these tests are never run as part of the main runtime tests.

[sharwell]

@mike-lischke Is there a way to configure the Travis and/or AppVeyor builds to run those tests, even if they aren't run by "the main runtime"?

[mike-lischke]

Tricky. For instance in the C++ runtime I use an XCode test target, which you cannot run in Travis CI (unless it's a Mac env). We could of course avoid using our IDE's tools and create all tests as plain target language code, but that makes quick runs more difficult (in XCode I just press a hotkey to quickly run all local tests, similar in other IDEs, constantly going back to terminal is a pain).