Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Missing jackson.databind in jar? #57

Closed
laxatives opened this issue Jan 22, 2016 · 27 comments
Closed

Missing jackson.databind in jar? #57

laxatives opened this issue Jan 22, 2016 · 27 comments

Comments

@laxatives
Copy link

I'm trying to use this library, but get the error below from the following line:

val file = new File("src/main/resources/GeoLite2-City.mmdb")
val db = new DatabaseReader.Builder(file).withCache(new CHMCache()).build()

yields:

An exception or error caused a run to abort: com.fasterxml.jackson.databind.node.ArrayNode.<init>(Lcom/fasterxml/jackson/databind/node/JsonNodeFactory;Ljava/util/List;)V 
java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.node.ArrayNode.<init>(Lcom/fasterxml/jackson/databind/node/JsonNodeFactory;Ljava/util/List;)V
    at com.maxmind.db.Decoder.decodeArray(Decoder.java:272)
    at com.maxmind.db.Decoder.decodeByType(Decoder.java:156)
    at com.maxmind.db.Decoder.decode(Decoder.java:147)
    at com.maxmind.db.Decoder.decodeMap(Decoder.java:281)
    at com.maxmind.db.Decoder.decodeByType(Decoder.java:154)
    at com.maxmind.db.Decoder.decode(Decoder.java:147)
    at com.maxmind.db.Decoder.decode(Decoder.java:87)
    at com.maxmind.db.Reader.<init>(Reader.java:132)
    at com.maxmind.db.Reader.<init>(Reader.java:116)
    at com.maxmind.geoip2.DatabaseReader.<init>(DatabaseReader.java:39)
    at com.maxmind.geoip2.DatabaseReader.<init>(DatabaseReader.java:27)
    at com.maxmind.geoip2.DatabaseReader$Builder.build(DatabaseReader.java:133)
    at com.sift.etl.processors.IpToLatLong$.<init>(IpToLatLong.scala:22)

Does this indicate the source is missing the jackson.databind.jar? Is it possible I have two conflicting copies of this dependency?
@oschwald
Copy link
Member

I don't understand. What is the error you are receiving? How are you installing this? If you are just loading the JAR, you will need to also load the other dependencies. There is a zip file with all of them on the releases page.

@laxatives
Copy link
Author

I added the following to my pom.xml:

<dependency>
            <groupId>com.maxmind.geoip2</groupId>
            <artifactId>geoip2</artifactId>
            <version>2.6.0</version>
</dependency>

@laxatives
Copy link
Author

Sorry, that error message was a bad copy/paste. Fixed.

@oschwald
Copy link
Member

Is it possible you are pulling in older version of jackson-databind from somewhere else?

@laxatives
Copy link
Author

Probably, although I'm not pulling it in explicitly in my pom. Is there a straightforward way for me to check where else it could be provided? Is there a setting that would allow me to ensure that geoIP is using the correct version of jackson-databind?

@oschwald
Copy link
Member

If you want to figure out the problem, you could look at mvn dependency:tree. If you just want to force it, you could add the following to your pom.xml:

        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-databind</artifactId>
            <version>2.7.0</version>
        </dependency>

I'd suggest the former as if you force the dependency, you'll need to update it manually with new versions of GeoIP2.

@phraktle
Copy link
Contributor

If you check your dependency tree, you'll most likely find a version of jackson older than 2.7.0 pulled in by another library. (For example, I had to add this override for Jersey: com.fasterxml.jackson.jaxrs:jackson-jaxrs-json-provider:2.7.0)

In any case, this is not a geoip issue, just a general dependency management problem.

@laxatives
Copy link
Author

Got it, thanks. I'll close the issue and try to debug this a bit more.

@laxatives
Copy link
Author

Hmm, so I thought I resolved this issue and after the tests successfully built and executed in Intellij, but in production I'm getting the same behavior I described in the initial post. I checked out the dependency tree and there are many instances of jackson-databind, all of them in packages I don't control.

[INFO] +- org.apache.spark:spark-core_2.10:jar:1.3.1:compile
[INFO] |  +- org.json4s:json4s-jackson_2.10:jar:3.2.10:compile
[INFO] |  |  \- (com.fasterxml.jackson.core:jackson-databind:jar:2.3.1:compile - omitted for conflict with 2.4.4)
[INFO] |  +- io.dropwizard.metrics:metrics-json:jar:3.1.0:compile
[INFO] |  |  \- (com.fasterxml.jackson.core:jackson-databind:jar:2.4.2:compile - omitted for conflict with 2.3.1)
[INFO] |  +- (com.fasterxml.jackson.core:jackson-databind:jar:2.4.4:compile - omitted for conflict with 2.7.0)
[INFO] |  +- com.fasterxml.jackson.module:jackson-module-scala_2.10:jar:2.4.4:compile
[INFO] |  |  \- (com.fasterxml.jackson.core:jackson-databind:jar:2.4.4:compile - omitted for duplicate)
[INFO] |  \- org.tachyonproject:tachyon-client:jar:0.5.0:compile
[INFO] |     \- org.tachyonproject:tachyon:jar:0.5.0:compile
[INFO] |        \- (com.fasterxml.jackson.core:jackson-databind:jar:2.3.0:compile - omitted for conflict with 2.4.4)
[INFO] +- org.elasticsearch:elasticsearch-spark_2.10:jar:2.1.2:compile
[INFO] |  \- org.apache.spark:spark-sql_2.10:jar:1.5.1:runtime
[INFO] |     \- (com.fasterxml.jackson.core:jackson-databind:jar:2.4.4:runtime - omitted for duplicate)
[INFO] +- com.maxmind.geoip2:geoip2:jar:2.6.0:compile
[INFO] |  \- com.maxmind.db:maxmind-db:jar:1.2.0:compile
[INFO] |     \- (com.fasterxml.jackson.core:jackson-databind:jar:2.7.0:compile - omitted for conflict with 2.4.4)
[INFO] +- com.fasterxml.jackson.core:jackson-databind:jar:2.7.0:compile
[INFO] \- com.thinkaurelius.titan:titan-core:jar:0.5.3:compile
[INFO]    \- com.tinkerpop.blueprints:blueprints-core:jar:2.5.0:compile
[INFO]       \- com.fasterxml.jackson.datatype:jackson-datatype-json-org:jar:2.2.3:compile
[INFO]          \- (com.fasterxml.jackson.core:jackson-databind:jar:2.2.2:compile - omitted for conflict with 2.7.0)

How can I resolve these conflicts? Should I exclude jackson-databind from each of these dependencies?

@laxatives
Copy link
Author

Hmm, the exclude removed all conflicts in the tree, yet I'm still getting the same error in production (but not running unit tests in intellij).

[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ sift-etl ---
[INFO] com.sift.etl:sift-etl:jar:0.1.6
[INFO] \- com.maxmind.geoip2:geoip2:jar:2.6.0:compile
[INFO]    \- com.maxmind.db:maxmind-db:jar:1.2.0:compile
[INFO]       \- com.fasterxml.jackson.core:jackson-databind:jar:2.7.0:compile

Exception in thread "main" java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.node.ArrayNode.<init>(Lcom/fasterxml/jackson/databind/node/JsonNodeFactory;Ljava/util/List;)V
        at com.maxmind.db.Decoder.decodeArray(Decoder.java:272)

@laxatives
Copy link
Author

Moving issue to https://stackoverflow.com/questions/34958520/nosuchmethodexception-in-mvn-dependency-without-conflicts since it probably isn't a GeoIP issue.

@laxatives laxatives reopened this Jan 26, 2016
@laxatives
Copy link
Author

This probably isn't a GeoIP problem, but I did notice that the shaded jar I am providing to Apache Spark expands the jackson-databding jar provided by GeoIp into its constituent *.class files, instead of the jar itself. Is this normal? There's a bit more detail in the stackoverflow link, but the spark task fails at init while test cases outside of spark succeed.

It seems like this there is something wrong either with the shaded jar or the spark classpath, but I'm not sure how to proceed debugging this issue. Has anyone else used GeoIP2 in a Spark task? I know there was a known issue with jackson-databind due to a very old version being used and the jar was not shaded, causing some issues in the dependency namespace, but that seems to have been patched in recent versions.

@oschwald
Copy link
Member

I don't know much about Apache Spark. You might have better luck asking on the Spark user email list or a similar forum.

@laxatives
Copy link
Author

THis was resolved in the shaded jar by relocating the jackson-databing.

@akashihi
Copy link

akashihi commented Feb 8, 2016

I have same problem and i do not have any other jackson-databind in the classpath.

Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.maxmind.geoip2.DatabaseReader]: Factory method 'countryLookup' threw exception; nested exception is java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.node.ArrayNode.(Lcom/fasterxml/jackson/databind/node/JsonNodeFactory;Ljava/util/List;)V
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:189) ~[spring-beans-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:588) ~[spring-beans-4.2.4.RELEASE.jar:4.2.4.RELEASE]
... 31 common frames omitted
Caused by: java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.node.ArrayNode.(Lcom/fasterxml/jackson/databind/node/JsonNodeFactory;Ljava/util/List;)V
at com.maxmind.db.Decoder.decodeArray(Decoder.java:272) ~[maxmind-db-1.2.0.jar:na]

Artifacts tree is:

[INFO] --- maven-dependency-plugin:2.10:tree (default-cli) @ cms ---
[INFO] org.satgate.filestream:cms:jar:1.0.0
[INFO] +- org.projectlombok:lombok:jar:1.16.6:provided
[INFO] +- com.maxmind.geoip2:geoip2:jar:2.6.0:compile
[INFO] | +- com.maxmind.db:maxmind-db:jar:1.2.0:compile
[INFO] | | - com.fasterxml.jackson.core:jackson-databind:jar:2.6.5:compile
[INFO] | | +- com.fasterxml.jackson.core:jackson-annotations:jar:2.6.5:compile
[INFO] | | - com.fasterxml.jackson.core:jackson-core:jar:2.6.5:compile
[INFO] | - com.google.http-client:google-http-client:jar:1.21.0:compile
[INFO] | +- com.google.code.findbugs:jsr305:jar:1.3.9:compile
[INFO] | - org.apache.httpcomponents:httpclient:jar:4.5.1:compile
[INFO] | +- org.apache.httpcomponents:httpcore:jar:4.4.4:compile
[INFO] | - commons-codec:commons-codec:jar:1.9:compile
[INFO] +- org.springframework.boot:spring-boot-starter-web:jar:1.3.2.RELEASE:compile
[INFO] | +- org.springframework.boot:spring-boot-starter:jar:1.3.2.RELEASE:compile
[INFO] | | +- org.springframework.boot:spring-boot:jar:1.3.2.RELEASE:compile
[INFO] | | +- org.springframework.boot:spring-boot-autoconfigure:jar:1.3.2.RELEASE:compile
[INFO] | | +- org.springframework.boot:spring-boot-starter-logging:jar:1.3.2.RELEASE:compile
[INFO] | | | +- ch.qos.logback:logback-classic:jar:1.1.3:compile
[INFO] | | | | - ch.qos.logback:logback-core:jar:1.1.3:compile
[INFO] | | | +- org.slf4j:jul-to-slf4j:jar:1.7.13:compile
[INFO] | | | - org.slf4j:log4j-over-slf4j:jar:1.7.13:compile
[INFO] | | - org.yaml:snakeyaml:jar:1.16:runtime
[INFO] | +- org.springframework.boot:spring-boot-starter-tomcat:jar:1.3.2.RELEASE:compile
[INFO] | | +- org.apache.tomcat.embed:tomcat-embed-core:jar:8.0.30:compile
[INFO] | | +- org.apache.tomcat.embed:tomcat-embed-el:jar:8.0.30:compile
[INFO] | | +- org.apache.tomcat.embed:tomcat-embed-logging-juli:jar:8.0.30:compile
[INFO] | | - org.apache.tomcat.embed:tomcat-embed-websocket:jar:8.0.30:compile
[INFO] | +- org.springframework.boot:spring-boot-starter-validation:jar:1.3.2.RELEASE:compile
[INFO] | | - org.hibernate:hibernate-validator:jar:5.2.2.Final:compile
[INFO] | | +- javax.validation:validation-api:jar:1.1.0.Final:compile
[INFO] | | +- org.jboss.logging:jboss-logging:jar:3.3.0.Final:compile
[INFO] | | - com.fasterxml:classmate:jar:1.1.0:compile
[INFO] | +- org.springframework:spring-web:jar:4.2.4.RELEASE:compile
[INFO] | | - org.springframework:spring-beans:jar:4.2.4.RELEASE:compile
[INFO] | - org.springframework:spring-webmvc:jar:4.2.4.RELEASE:compile
[INFO] | - org.springframework:spring-expression:jar:4.2.4.RELEASE:compile
[INFO] +- org.springframework.boot:spring-boot-starter-aop:jar:1.3.2.RELEASE:compile
[INFO] | +- org.springframework:spring-aop:jar:4.2.4.RELEASE:compile
[INFO] | | - aopalliance:aopalliance:jar:1.0:compile
[INFO] | - org.aspectj:aspectjweaver:jar:1.8.8:compile
[INFO] +- org.springframework.boot:spring-boot-starter-cache:jar:1.3.2.RELEASE:compile
[INFO] | +- org.springframework:spring-context:jar:4.2.4.RELEASE:compile
[INFO] | - org.springframework:spring-context-support:jar:4.2.4.RELEASE:compile
[INFO] +- org.springframework.boot:spring-boot-starter-data-mongodb:jar:1.3.2.RELEASE:compile
[INFO] | +- org.mongodb:mongo-java-driver:jar:2.13.3:compile
[INFO] | - org.springframework.data:spring-data-mongodb:jar:1.8.2.RELEASE:compile
[INFO] | +- org.springframework:spring-tx:jar:4.2.4.RELEASE:compile
[INFO] | +- org.springframework.data:spring-data-commons:jar:1.11.2.RELEASE:compile
[INFO] | +- org.slf4j:slf4j-api:jar:1.7.13:compile
[INFO] | - org.slf4j:jcl-over-slf4j:jar:1.7.13:compile
[INFO] +- org.springframework.boot:spring-boot-starter-redis:jar:1.3.2.RELEASE:compile
[INFO] | +- org.springframework.data:spring-data-redis:jar:1.6.2.RELEASE:compile
[INFO] | - redis.clients:jedis:jar:2.7.3:compile
[INFO] | - org.apache.commons:commons-pool2:jar:2.4.2:compile
[INFO] +- org.springframework.boot:spring-boot-starter-security:jar:1.3.2.RELEASE:compile
[INFO] | +- org.springframework.security:spring-security-config:jar:4.0.3.RELEASE:compile
[INFO] | | - org.springframework.security:spring-security-core:jar:4.0.3.RELEASE:compile
[INFO] | - org.springframework.security:spring-security-web:jar:4.0.3.RELEASE:compile
[INFO] +- org.springframework.boot:spring-boot-starter-test:jar:1.3.2.RELEASE:test
[INFO] | +- org.mockito:mockito-core:jar:1.10.19:test
[INFO] | | - org.objenesis:objenesis:jar:2.1:test
[INFO] | +- org.hamcrest:hamcrest-core:jar:1.3:compile
[INFO] | +- org.hamcrest:hamcrest-library:jar:1.3:test
[INFO] | +- org.springframework:spring-core:jar:4.2.4.RELEASE:compile
[INFO] | - org.springframework:spring-test:jar:4.2.4.RELEASE:test
[INFO] - junit:junit:jar:4.12:compile

What i could be doing wrong?

@oschwald
Copy link
Member

oschwald commented Feb 8, 2016

@akashihi, it appears you are pulling in jackson-databind 2.6.5 when maxmind-db 1.2.0 requires 2.7.0.

@akashihi
Copy link

akashihi commented Feb 8, 2016

@oschwald maven is so weird :-/

@oschwald
Copy link
Member

As this issue has come up so much, I have specified a hard minimum dependency on 2.7.0. The soft dependency in maxmind-db was not working too well in the case of conflicts with Maven's nearest-first resolution rules.

@navis navis mentioned this issue Mar 24, 2016
Closed
@sonya
Copy link

sonya commented Apr 28, 2016

For people running into this on Spark, the hard minimum dependency in this library may be ignored at runtime because Spark provides its own version (2.4.4 as of Spark 1.6.1). For now I think @laxatives' solution is the way to get around this (though I'm not sure why you only had to shade core). For sbt I used the following

assemblyShadeRules in assembly := Seq(
  ShadeRule.rename("com.fasterxml.jackson.core.**"       -> "shadedjackson.core.@1").inAll,
  ShadeRule.rename("com.fasterxml.jackson.annotation.**" -> "shadedjackson.annotation.@1").inAll,
  ShadeRule.rename("com.fasterxml.jackson.databind.**"   -> "shadedjackson.databind.@1").inAll
)

@oschwald
Copy link
Member

Although I am not familiar with Spark, if you are unable to provide your own Jackson version, you could try using a version of this library that doesn't use 2.7-specific functionality, e.g., 2.5.0.

@oschwald oschwald mentioned this issue Oct 26, 2016
Closed
@deepakas
Copy link

I am having the same problem allocating the correct dependency version while using spark from Zeppelin. Any ideas ?

@oschwald
Copy link
Member

@deepakas, does @sonya's suggestion above work for you? Alternatively, you could use 2.5.0 of geoip2, which doesn't require a recent version of Jackson.

@constantOut
Copy link

While I'm using jackson dependency directly, I was surprised to see different versions of jackson in the outpu of evicted sbt command :

[info]         +- com.fasterxml.jackson.module:jackson-module-scala_2.11:2.9.5 (depends on 2.9.5)
[info]         +- com.fasterxml.jackson.module:jackson-module-paranamer:2.9.5 (depends on 2.9.5)
[info]         +- com.maxmind.geoip2:geoip2:2.12.0                   (depends on 2.9.5)
[info]         +- com.maxmind.db:maxmind-db:1.2.2                    (depends on 2.7.9)

Pay attention that com.maxmind.geoip2:geoip2:2.12.0 and com.maxmind.db:maxmind-db:1.2.2
depend on different versions of com.fasterxml.jackson.core:jackson-databind which I assume to be problematic.

@oschwald
Copy link
Member

@constantOut, is this causing a particular issue? Maven uses nearest-first dependency resolution and you should end up with the newer version.

@constantOut
Copy link

constantOut commented Dec 19, 2018 •
edited

I'm using SBT, this is causing issue when running on spark, however I'm not sure that this is a primary cause of the problem as other people had problem with jackson on Spark.
I'm getting java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.node.ArrayNode That's how I've found this issue in google.

@oschwald
Copy link
Member

oschwald commented Dec 19, 2018 •
edited

java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.node.ArrayNode happens when a version of jackson-databind before 2.7.0 is used, as discussed above. This is unrelated to the differing version numbers in maxmind-db and geoip2. In the full dependency tree that another person above posted, Spark was pulling in jackson-databind 2.4.0, which caused their issue.

@constantOut
Copy link

I'll dig into that, thanks.

@drinkemc drinkemc mentioned this issue Sep 2, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@sonya @oschwald @phraktle @akashihi @deepakas @laxatives @constantOut