Compiling natty with 3.4 antlr-runtime ? #60

sofra · 2013-10-31T01:35:51Z

Hi,

I wanna wrap natty (awesome lib!!!) in HIve UDF. Unfortunately hive is using antlr-runtime 3.4, so I have no option but to compile natty with antlr plugin antlr 3.4.

Compilation just hangs. I was wandering did you tried to migrate to 3.4 and do you have any hints how to do this (also 0.8 natty can not run in 3.4 antlr enviroment if fails with java.lang.NoSuchMethodError: org.antlr.runtime.debug.DebugEventListener.enterDecision(I)V )

All the best,
Nemanja.

PS: More detail

Running 0.8 natty in 3.4 antlr runtime throws
java.lang.NoSuchMethodError: org.antlr.runtime.debug.DebugEventListener.enterDecision(I)V

My compilation setup
https://github.com/klout/natty/compare/joestelmach:master...master

antlr 3.0.1 (was 2.7.7) works just fine with runtime 3.2
antlr-runtime 3.4 (was 3.2)

Compilation (cmd - mvn clean install -DskipTests -Pnon-debug -X) hangs on:

As a result, alternative(s) 3 were disabled for that input
warning(200): com/joestelmach/natty/generated/DateParser.g:775:3:
Decision can match input such as "IN WHITE_SPACE ONE WHITE_SPACE SECOND" using multiple alternatives: 2, 3

As a result, alternative(s) 3 were disabled for that input

And fails after 55min (3.2 antlr-runtime env compiles in 23sec) with:

org.apache.maven.lifecycle.LifecycleExecutionException: ANTLR caught 2 build errors.
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:719)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalWithLifecycle(DefaultLifecycleExecutor.java:556)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:535)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:387)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:348)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:180)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:328)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:138)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
at org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:60)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
at org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
Caused by: org.apache.maven.plugin.MojoExecutionException: ANTLR caught 2 build errors.
at org.antlr.mojo.antlr3.Antlr3Mojo.execute(Antlr3Mojo.java:384)
at org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:490)
at org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:694)
... 17 more

joestelmach · 2013-10-31T03:10:15Z

I haven't attempted to migrate natty to ANTLR 3.4, but I'd be more than happy to merge in a tested pull request.

sofra · 2013-10-31T21:30:31Z

Thanks for reply :-)

I'll give it a try to make it work with 3.4 ANTLR, but given I did not used antlr before probably slim chance I could figure it out.

It seems almost like new antler is getting into some infinity loop and just hanging on :(

o-nix · 2014-03-11T00:52:44Z

Just faced with the problem using drools-compiler 6.0.1 which requires antlr-runtime 3.5.

zhemaituk · 2014-08-07T09:07:06Z

I was successful with compiling natty with antlr 3.5. Encountered problems:

Build took about 45 minutes (with 3.2 it was 23 seconds on the same machine).
(resolved) Failed DateTest with NPEs in logs.
SearchTest failed with:

junit.framework.ComparisonFailure: 
Expected :sep 13, 2013
Actual   :sepsep 13,sepsep 13, 2013sepsep 13sepsep 13,sepsep 13, 2013sepsep 13,sepsep 13, 2013 sepsep 13,sepsep 13, 2013sepsep 13,sepsep 13, 2013
    at com.joestelmach.natty.SearchTest.test(SearchTest.java:246)

Code changes here: https://github.com/azhemoytuk/natty

clementcm · 2014-08-15T20:55:34Z

azhemoytuk I tried your changes and had the same issue.
The test failed because a signature change in a generated class: BlankDebugEventListener

I fixed it by changing the following in the ParseListener.

Changed from:
public void enterDecision(int d) {
backtracking++;
}

to
public void enterDecision(int d, boolean couldBacktrack) {
backtracking++;
}

The build still took me about an hour tho.

This should solve some dependency conflicts regarding antlr-runtime, since natty currently doesn't work with versions higher than antlr-runtime 3.2. See joestelmach/natty#60 for the related bug report.

jmahonin · 2014-12-09T21:25:07Z

+1 to this, would like to use Natty in a Hadoop/HBase project with hard ANTLR dependencies. The org.graylog2.repackaged version is working for me however.

joestelmach · 2015-02-08T17:22:10Z

The main problem I've had when trying to upgrade ANTLR past 3.2 was the removal of the conversion timeout property, which would cause crazy slow compile times. I've finally been able to rework the grammar to compile quickly with ANTLR 3.3+ (fc0ff4c), but it looks like 3.3 and 3.4+ have their own compatibility issues with natty, so I'm trying to upgrade one step at a time.

joestelmach · 2015-02-09T04:26:38Z

@azhemoytuk : would you mind sending a pull request for your changes in NattyTokenSource? And @clementcm for your changes in ParseListener? I've verified these changes work just fine with the recent grammar changes, and compile time is down to around 10 seconds on my machine. I'd like to make sure you both get marked as a contributor. Thanks!

clementcm · 2015-02-09T17:50:49Z

@joestelmach I am trying to create a pull request to your antlr33 branch. I upgraded to Antlr3.5 with my ParseListener changes. I got ode too large for try statement error when I build with Maven. I don't get this error with eclipse build. Have you ever run into this problem before?

[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] \projects\natty\natty-git\natty\src\main\java\com\joestelmach\natty\generated\DateParser.java:[32370,7] error: code too large for try statement
[ERROR] \projects\natty\natty-git\natty\src\main\java\com\joestelmach\natty\generated\DateParser.java:[35329,9] error: code too large for try statement
[ERROR] \projects\natty\natty-git\natty\src\main\java\com\joestelmach\natty\generated\DateParser.java:[35328,9] error: code too large for try statement
[ERROR] \projects\natty\natty-git\natty\src\main\java\com\joestelmach\natty\generated\DateParser.java:[32369,7] error: code too large for try statement
[ERROR] \projects\natty\natty-git\natty\src\main\java\com\joestelmach\natty\generated\DateParser.java:[35476,2] error: code too large for try statement
[ERROR] \projects\natty\natty-git\natty\src\main\java\com\joestelmach\natty\generated\DateParser.java:[32326,6] error: code too large for try statement
[ERROR] \projects\natty\natty-git\natty\src\main\java\com\joestelmach\natty\generated\DateParser.java:[32321,6] error: code too large for try statement
[ERROR] \projects\natty\natty-git\natty\src\main\java\com\joestelmach\natty\generated\DateParser.java:[32299,43] error: code too large

joestelmach · 2015-02-09T18:09:54Z

I'm seeing this as well with 3.4 and 3.5. If you want to build against 3.3 for now we can get a change in to fix the API differences introduced in 3.3. Then I'll take a look at the code too large issue (which should be unrelated to these changes)

Thanks for the help!

clementcm · 2015-02-09T19:34:33Z

I tired to upgrade Antlr to 3.3 to your antlr33 branch. All test cases failed as follow. Your Master branch does not have this problem.

Feb 09, 2015 2:31:04 PM com.joestelmach.natty.Parser singleParse
SEVERE: Could not parse input
java.lang.NullPointerException
at org.antlr.runtime.BufferedTokenStream.fetch(BufferedTokenStream.java:134)
at org.antlr.runtime.BufferedTokenStream.sync(BufferedTokenStream.java:127)
at org.antlr.runtime.CommonTokenStream.consume(CommonTokenStream.java:67)
at org.antlr.runtime.debug.DebugTokenStream.consume(DebugTokenStream.java:60)
at com.joestelmach.natty.generated.DateParser_NumericRules.int_24_to_31(DateParser_NumericRules.java:8466)
at com.joestelmach.natty.generated.DateParser_NumericRules.int_01_to_31_optional_prefix(DateParser_NumericRules.java:1449)
at com.joestelmach.natty.generated.DateParser.int_01_to_31_optional_prefix(DateParser.java:31837)
at com.joestelmach.natty.generated.DateParser.formal_day_of_month(DateParser.java:12717)
at com.joestelmach.natty.generated.DateParser.formal_date(DateParser.java:11840)
at com.joestelmach.natty.generated.DateParser.date(DateParser.java:2615)
at com.joestelmach.natty.generated.DateParser.synpred4_DateParser_fragment(DateParser.java:31628)
at com.joestelmach.natty.generated.DateParser.synpred4_DateParser(DateParser.java:31879)
at com.joestelmach.natty.generated.DateParser$DFA19.specialStateTransition(DateParser.java:37435)
at org.antlr.runtime.DFA.predict(DFA.java:80)
at com.joestelmach.natty.generated.DateParser.date_time_alternative(DateParser.java:2840)
at com.joestelmach.natty.generated.DateParser.parse(DateParser.java:547)
at com.joestelmach.natty.Parser.singleParse(Parser.java:153)
at com.joestelmach.natty.Parser.parse(Parser.java:77)
at com.joestelmach.natty.AbstractTest.parseCollection(AbstractTest.java:39)
at com.joestelmach.natty.AbstractTest.parseSingleDate(AbstractTest.java:49)
at com.joestelmach.natty.AbstractTest.validateDate(AbstractTest.java:64)
at com.joestelmach.natty.DateTest.testFormal(DateTest.java:29)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.junit.internal.runners.TestMethodRunner.executeMethodBody(TestMethodRunner.java:99)
at org.junit.internal.runners.TestMethodRunner.runUnprotected(TestMethodRunner.java:81)
at org.junit.internal.runners.BeforeAndAfterRunner.runProtected(BeforeAndAfterRunner.java:34)
at org.junit.internal.runners.TestMethodRunner.runMethod(TestMethodRunner.java:75)
at org.junit.internal.runners.TestMethodRunner.run(TestMethodRunner.java:45)
at org.junit.internal.runners.TestClassMethodsRunner.invokeTestMethod(TestClassMethodsRunner.java:71)
at org.junit.internal.runners.TestClassMethodsRunner.run(TestClassMethodsRunner.java:35)
at org.junit.internal.runners.TestClassRunner$1.runUnprotected(TestClassRunner.java:42)
at org.junit.internal.runners.BeforeAndAfterRunner.runProtected(BeforeAndAfterRunner.java:34)
at org.junit.internal.runners.TestClassRunner.run(TestClassRunner.java:52)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)

joestelmach · 2015-02-09T19:51:42Z

You'll need azhemoytuk's changes to get past the NPE's, but feel free to send a pull request with just your changes to ParseListener and I can manage any integration trouble on my end.

clementcm · 2015-02-09T20:49:34Z

Oops forgot to merge his code. Now the pull request is sent.
Could you shear your plan for the next release? We have product to go live soon. It is nice to have your latest release instead of my custom build.

Thanks.

joestelmach · 2015-02-10T05:20:20Z

I just released version 0.10 - give it a spin.

clementcm · 2015-02-10T17:50:22Z

Thanks Joe! It passed all my test cases except one. I'vd opened an issue #108.

zhemaituk · 2015-02-13T13:33:29Z

Thanks Joe, that's the great news!
New version works good for my test cases.

clementcm · 2015-02-13T16:43:32Z

Sweet. Thank you very much Joe!

douglaslyon · 2016-11-04T10:57:22Z

Not sure what I am doing wrong; getting error:
Exception in thread "main" java.lang.NoSuchMethodError: org.antlr.runtime.debug.DebugEventListener.enterDecision(IZ)V
just did a fresh clone, rebuild and used the jar created with:
antlr-3.1.2-jar-with-dependencies.jar
Am I using the wrong version of antlr?

joestelmach · 2016-11-04T14:01:25Z

The project is configured to build easily with Maven. Take a look at the pom.xml file for the proper dependencies, which does indicate you're using the wrong version of ANTLR (3.5.2 is required)

douglaslyon · 2016-11-04T15:55:20Z

Hi Joe,
Yes, the new version of ANTLR fixed it!
Thanks!

Doug

On 11/4/16 10:01 AM, Joe Stelmach wrote:

The project is configured to build easily with Maven. Take a look at the
pom.xml file for the proper dependencies, which does indicate you're
using the wrong version of ANTLR (3.5.2 is required)

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#60 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABaDopv0f-CDFUYgE-gYJsS2KJu6gflLks5q6zq2gaJpZM4BJ5mf.

joestelmach closed this as completed in 2964683 Feb 10, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Compiling natty with 3.4 antlr-runtime ? #60

Compiling natty with 3.4 antlr-runtime ? #60

sofra commented Oct 31, 2013

joestelmach commented Oct 31, 2013

sofra commented Oct 31, 2013

o-nix commented Mar 11, 2014

zhemaituk commented Aug 7, 2014

clementcm commented Aug 15, 2014

jmahonin commented Dec 9, 2014

joestelmach commented Feb 8, 2015

joestelmach commented Feb 9, 2015

clementcm commented Feb 9, 2015

joestelmach commented Feb 9, 2015

clementcm commented Feb 9, 2015

joestelmach commented Feb 9, 2015

clementcm commented Feb 9, 2015

joestelmach commented Feb 10, 2015

clementcm commented Feb 10, 2015

zhemaituk commented Feb 13, 2015

clementcm commented Feb 13, 2015

douglaslyon commented Nov 4, 2016

joestelmach commented Nov 4, 2016

douglaslyon commented Nov 4, 2016

Compiling natty with 3.4 antlr-runtime ? #60

Compiling natty with 3.4 antlr-runtime ? #60

Comments

sofra commented Oct 31, 2013

joestelmach commented Oct 31, 2013

sofra commented Oct 31, 2013

o-nix commented Mar 11, 2014

zhemaituk commented Aug 7, 2014

clementcm commented Aug 15, 2014

jmahonin commented Dec 9, 2014

joestelmach commented Feb 8, 2015

joestelmach commented Feb 9, 2015

clementcm commented Feb 9, 2015

joestelmach commented Feb 9, 2015

clementcm commented Feb 9, 2015

joestelmach commented Feb 9, 2015

clementcm commented Feb 9, 2015

joestelmach commented Feb 10, 2015

clementcm commented Feb 10, 2015

zhemaituk commented Feb 13, 2015

clementcm commented Feb 13, 2015

douglaslyon commented Nov 4, 2016

joestelmach commented Nov 4, 2016

douglaslyon commented Nov 4, 2016