[SPARK-44366][BUILD] Upgrade antlr4 to 4.13.1

[LuciferYang]

What changes were proposed in this pull request?

This pr is aims upgrade antlr4 from 4.9.3 to 4.13.1

Why are the changes needed?

After 4.10, antlr4 is using Java 11 for the source code and the compiled .class files for the ANTLR tool. There are some bug fix and Improvements after 4.9.3:

• Implement caseInsensitive option antlr/antlr4#3399
• avoid java.lang.StackOverflowError antlr/antlr4#1105
• antlr4 java.lang.NullPointerException Antlr 4 4.8 antlr/antlr4#2788
• Generate identical atn simulation output across targets to compare parsing functionality antlr/antlr4#3957
• Java: suppress this-escape warning introduced in JDK 21. antlr/antlr4#4394

The full release notes as follows:

• https://github.com/antlr/antlr4/releases/tag/4.13.1
• https://github.com/antlr/antlr4/releases/tag/4.13.0
• https://github.com/antlr/antlr4/releases/tag/4.12.0
• https://github.com/antlr/antlr4/releases/tag/4.11.1
• https://github.com/antlr/antlr4/releases/tag/4.11.0
• https://github.com/antlr/antlr4/releases/tag/4.10.1
• https://github.com/antlr/antlr4/releases/tag/4.10

Does this PR introduce any user-facing change?

No

How was this patch tested?

Pass GitHub Actions

Was this patch authored or co-authored using generative AI tooling?

No

[LuciferYang]

Test first

[srowen]

Merged to master

[LuciferYang]

Thanks @srowen

[jimsimon-wk]

Any idea on when this might make it into a tagged release?

[srowen]

Looks like this would go out with 4.0 next year

[jimsimon-wk]

I noticed the Spark project uses maven-shade-plugin to relocate some dependencies. Any likelihood that we could provide a minor/patch version that provides shading for the antlr 4.9 dependency? There appears to be some conflicts with Hibernate 6.x.

• Hibernate ORM discussion about upgrading to antlr 4.10
• Hibernate ORM PR to upgrade to antlr 4.10
• Stack Overflow reporting some issues with Spark 3.3.2 + Hibernate 6

[LuciferYang]

I noticed the Spark project uses maven-shade-plugin to relocate some dependencies. Any likelihood that we could provide a minor/patch version that provides shading for the antlr 4.9 dependency? There appears to be some conflicts with Hibernate 6.x.

• Hibernate ORM discussion about upgrading to antlr 4.10
• Hibernate ORM PR to upgrade to antlr 4.10
• Stack Overflow reporting some issues with Spark 3.3.2 + Hibernate 6

Spark 3.5 still uses antlr4 4.9.3.

[weikangchia]

Hi, I am also facing issues with conflicts with Hibernate 6.x and Spark 3.4.1. May I know if there is any solution or workaround I can use?

[LuciferYang]

Hi, I am also facing issues with conflicts with Hibernate 6.x and Spark 3.4.1. May I know if there is any solution or workaround I can use?

This pr is for Spark 4.x version only and it's unrelated to Spark 3.4.1. Due to the Spark 3.x releases needing to support Java 8, it also can't be backported to Spark 3.4.1. If you have other issues to discuss, you can initiate a discussion via the dev mailing list or create a related jira (even your improved patch). Thanks :)