AVRO-3403: Create and use ANTLR to parse IDL files

[opwvhk]

What is the purpose of the change

Implement AVRO-3403: Migrate JavaCC to ANTLR

The implementation uses a shared ANTLR grammar, without actions or
predicates. This guarantees ANTLR can create usable parsers in any
language it supports. To implement IDL support for other languages, the
Java code to port is the class IdlReader.

The IDL reader using the ANTLR generated parser replaces the JavaCC
parser in the maven plugin and in the tools. The JavaCC parser has been
deprecated, but not removed.

Verifying this change

This change is huge, but consists roughly of these types of changes:

1. A shared grammar; please verify if ANTLR generates valid code for your favourite language (I tested Java & Python)
2. A parser in the IDLReader class in a new Java avro-idl module
3. A lot of tests and some utilities copied from avro-compiler
4. Using the new IDLReader class instead of the (now deprecated) IDL class

Note that reviews become a lot easier by reviewing per commit.

Documentation

• Does this pull request introduce a new feature? (yes / no)
• If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

[opwvhk]

Added a change to the grammar (sorry for the way in which I did...) such that it the generated code can also work on Python.

[martin-g]

I didn't review yet the new IdlReader class and few related classes, mostly because it is big and I have zero experience with ANTLR. I will try do to it soon!

Apart from my comments there are also some CodeQL comments in the diff view which might be addressed.

[github-advanced-security]

Found 14 potential problems in the proposed changes. Check the Files changed tab for more details.

[lmove]

Hi! Our tool BreakBot found that this pull request introduces 3 breaking changes and they appear to impact 5 of your clients. You can find the full BreakBot report in our fork repository: report for PR#1588.

We hope this information is valuable to you, and apologize otherwise. If you're willing to help, we would kindly ask for your help to fill in a 5-minutes survey about the report. Your feedback will help us improve the tool and provide a better experience for users in the future!

[opwvhk]

The BreakBot report marks the @Deprecated annotations as breaking changes.

Set aside that these are not actually breaking changes, they were added on purpose to signal a migration is needed for future compatibility.

[tdegueul]

You are right. We include the impact of @Deprecated elements as it may give a better idea of which clients will be faced with a deprecated API. Thanks for your input @opwvhk!

[martin-g]

I didn't spend too much time (the change list is huge!) but I didn't notice anything bad.

[opwvhk]

I didn't spend too much time (the change list is huge!) but I didn't notice anything bad.

Ah, yes... the tricky part about replacing the IDL parser is that that's unavoidable.

Thank you for taking the time though!

The subsequent PR's (#1589 & #1954) are significantly smaller 😅

[clesaec]

A bit complex, but, as you said, unavoidable :-)
Why not put "Idl.g4" file in src/main/resources of Idl module ?
Need a rebase

[opwvhk]

Why not put "Idl.g4" file in src/main/resources of Idl module ?

Because this is a shared location, which allows the grammar to be reused for e g. a Python implementation.

The reason to migrate to ANTLR is partly because it is supported, but also because it supports multiple languages.

[opwvhk]

After resolving the conflicts, I'd like to merge this one and move on to PR #1589 (making the IDL syntax a full .avsc equivalent). As there's 2 approvals other than mine, are there any objections to merge?