New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing jackson.databind in jar? #57

Closed
laxatives opened this Issue Jan 22, 2016 · 22 comments

Comments

Projects
None yet
6 participants
@laxatives

laxatives commented Jan 22, 2016

I'm trying to use this library, but get the error below from the following line:

val file = new File("src/main/resources/GeoLite2-City.mmdb")
val db = new DatabaseReader.Builder(file).withCache(new CHMCache()).build()

yields:

An exception or error caused a run to abort: com.fasterxml.jackson.databind.node.ArrayNode.<init>(Lcom/fasterxml/jackson/databind/node/JsonNodeFactory;Ljava/util/List;)V 
java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.node.ArrayNode.<init>(Lcom/fasterxml/jackson/databind/node/JsonNodeFactory;Ljava/util/List;)V
    at com.maxmind.db.Decoder.decodeArray(Decoder.java:272)
    at com.maxmind.db.Decoder.decodeByType(Decoder.java:156)
    at com.maxmind.db.Decoder.decode(Decoder.java:147)
    at com.maxmind.db.Decoder.decodeMap(Decoder.java:281)
    at com.maxmind.db.Decoder.decodeByType(Decoder.java:154)
    at com.maxmind.db.Decoder.decode(Decoder.java:147)
    at com.maxmind.db.Decoder.decode(Decoder.java:87)
    at com.maxmind.db.Reader.<init>(Reader.java:132)
    at com.maxmind.db.Reader.<init>(Reader.java:116)
    at com.maxmind.geoip2.DatabaseReader.<init>(DatabaseReader.java:39)
    at com.maxmind.geoip2.DatabaseReader.<init>(DatabaseReader.java:27)
    at com.maxmind.geoip2.DatabaseReader$Builder.build(DatabaseReader.java:133)
    at com.sift.etl.processors.IpToLatLong$.<init>(IpToLatLong.scala:22)

Does this indicate the source is missing the jackson.databind.jar? Is it possible I have two conflicting copies of this dependency?

@oschwald

This comment has been minimized.

Member

oschwald commented Jan 22, 2016

I don't understand. What is the error you are receiving? How are you installing this? If you are just loading the JAR, you will need to also load the other dependencies. There is a zip file with all of them on the releases page.

@laxatives

This comment has been minimized.

laxatives commented Jan 22, 2016

I added the following to my pom.xml:

<dependency>
            <groupId>com.maxmind.geoip2</groupId>
            <artifactId>geoip2</artifactId>
            <version>2.6.0</version>
</dependency>
@laxatives

This comment has been minimized.

laxatives commented Jan 22, 2016

Sorry, that error message was a bad copy/paste. Fixed.

@oschwald

This comment has been minimized.

Member

oschwald commented Jan 22, 2016

Is it possible you are pulling in older version of jackson-databind from somewhere else?

@laxatives

This comment has been minimized.

laxatives commented Jan 22, 2016

Probably, although I'm not pulling it in explicitly in my pom. Is there a straightforward way for me to check where else it could be provided? Is there a setting that would allow me to ensure that geoIP is using the correct version of jackson-databind?

@oschwald

This comment has been minimized.

Member

oschwald commented Jan 22, 2016

If you want to figure out the problem, you could look at mvn dependency:tree. If you just want to force it, you could add the following to your pom.xml:

        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-databind</artifactId>
            <version>2.7.0</version>
        </dependency>

I'd suggest the former as if you force the dependency, you'll need to update it manually with new versions of GeoIP2.

@phraktle

This comment has been minimized.

Contributor

phraktle commented Jan 22, 2016

If you check your dependency tree, you'll most likely find a version of jackson older than 2.7.0 pulled in by another library. (For example, I had to add this override for Jersey: com.fasterxml.jackson.jaxrs:jackson-jaxrs-json-provider:2.7.0)

In any case, this is not a geoip issue, just a general dependency management problem.

@laxatives

This comment has been minimized.

laxatives commented Jan 22, 2016

Got it, thanks. I'll close the issue and try to debug this a bit more.

@laxatives laxatives closed this Jan 22, 2016

@laxatives

This comment has been minimized.

laxatives commented Jan 23, 2016

Hmm, so I thought I resolved this issue and after the tests successfully built and executed in Intellij, but in production I'm getting the same behavior I described in the initial post. I checked out the dependency tree and there are many instances of jackson-databind, all of them in packages I don't control.

[INFO] +- org.apache.spark:spark-core_2.10:jar:1.3.1:compile
[INFO] |  +- org.json4s:json4s-jackson_2.10:jar:3.2.10:compile
[INFO] |  |  \- (com.fasterxml.jackson.core:jackson-databind:jar:2.3.1:compile - omitted for conflict with 2.4.4)
[INFO] |  +- io.dropwizard.metrics:metrics-json:jar:3.1.0:compile
[INFO] |  |  \- (com.fasterxml.jackson.core:jackson-databind:jar:2.4.2:compile - omitted for conflict with 2.3.1)
[INFO] |  +- (com.fasterxml.jackson.core:jackson-databind:jar:2.4.4:compile - omitted for conflict with 2.7.0)
[INFO] |  +- com.fasterxml.jackson.module:jackson-module-scala_2.10:jar:2.4.4:compile
[INFO] |  |  \- (com.fasterxml.jackson.core:jackson-databind:jar:2.4.4:compile - omitted for duplicate)
[INFO] |  \- org.tachyonproject:tachyon-client:jar:0.5.0:compile
[INFO] |     \- org.tachyonproject:tachyon:jar:0.5.0:compile
[INFO] |        \- (com.fasterxml.jackson.core:jackson-databind:jar:2.3.0:compile - omitted for conflict with 2.4.4)
[INFO] +- org.elasticsearch:elasticsearch-spark_2.10:jar:2.1.2:compile
[INFO] |  \- org.apache.spark:spark-sql_2.10:jar:1.5.1:runtime
[INFO] |     \- (com.fasterxml.jackson.core:jackson-databind:jar:2.4.4:runtime - omitted for duplicate)
[INFO] +- com.maxmind.geoip2:geoip2:jar:2.6.0:compile
[INFO] |  \- com.maxmind.db:maxmind-db:jar:1.2.0:compile
[INFO] |     \- (com.fasterxml.jackson.core:jackson-databind:jar:2.7.0:compile - omitted for conflict with 2.4.4)
[INFO] +- com.fasterxml.jackson.core:jackson-databind:jar:2.7.0:compile
[INFO] \- com.thinkaurelius.titan:titan-core:jar:0.5.3:compile
[INFO]    \- com.tinkerpop.blueprints:blueprints-core:jar:2.5.0:compile
[INFO]       \- com.fasterxml.jackson.datatype:jackson-datatype-json-org:jar:2.2.3:compile
[INFO]          \- (com.fasterxml.jackson.core:jackson-databind:jar:2.2.2:compile - omitted for conflict with 2.7.0)

How can I resolve these conflicts? Should I exclude jackson-databind from each of these dependencies?

@laxatives

This comment has been minimized.

laxatives commented Jan 23, 2016

Hmm, the exclude removed all conflicts in the tree, yet I'm still getting the same error in production (but not running unit tests in intellij).

[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ sift-etl ---
[INFO] com.sift.etl:sift-etl:jar:0.1.6
[INFO] \- com.maxmind.geoip2:geoip2:jar:2.6.0:compile
[INFO]    \- com.maxmind.db:maxmind-db:jar:1.2.0:compile
[INFO]       \- com.fasterxml.jackson.core:jackson-databind:jar:2.7.0:compile
Exception in thread "main" java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.node.ArrayNode.<init>(Lcom/fasterxml/jackson/databind/node/JsonNodeFactory;Ljava/util/List;)V
        at com.maxmind.db.Decoder.decodeArray(Decoder.java:272)
@laxatives

This comment has been minimized.

laxatives commented Jan 23, 2016

@laxatives laxatives reopened this Jan 26, 2016

@laxatives

This comment has been minimized.

laxatives commented Jan 26, 2016

This probably isn't a GeoIP problem, but I did notice that the shaded jar I am providing to Apache Spark expands the jackson-databding jar provided by GeoIp into its constituent *.class files, instead of the jar itself. Is this normal? There's a bit more detail in the stackoverflow link, but the spark task fails at init while test cases outside of spark succeed.

It seems like this there is something wrong either with the shaded jar or the spark classpath, but I'm not sure how to proceed debugging this issue. Has anyone else used GeoIP2 in a Spark task? I know there was a known issue with jackson-databind due to a very old version being used and the jar was not shaded, causing some issues in the dependency namespace, but that seems to have been patched in recent versions.

@oschwald

This comment has been minimized.

Member

oschwald commented Jan 26, 2016

I don't know much about Apache Spark. You might have better luck asking on the Spark user email list or a similar forum.

@laxatives

This comment has been minimized.

laxatives commented Jan 27, 2016

THis was resolved in the shaded jar by relocating the jackson-databing.

@laxatives laxatives closed this Jan 27, 2016

@akashihi

This comment has been minimized.

akashihi commented Feb 8, 2016

I have same problem and i do not have any other jackson-databind in the classpath.

Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.maxmind.geoip2.DatabaseReader]: Factory method 'countryLookup' threw exception; nested exception is java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.node.ArrayNode.(Lcom/fasterxml/jackson/databind/node/JsonNodeFactory;Ljava/util/List;)V
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:189) ~[spring-beans-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:588) ~[spring-beans-4.2.4.RELEASE.jar:4.2.4.RELEASE]
... 31 common frames omitted
Caused by: java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.node.ArrayNode.(Lcom/fasterxml/jackson/databind/node/JsonNodeFactory;Ljava/util/List;)V
at com.maxmind.db.Decoder.decodeArray(Decoder.java:272) ~[maxmind-db-1.2.0.jar:na]

Artifacts tree is:

[INFO] --- maven-dependency-plugin:2.10:tree (default-cli) @ cms ---
[INFO] org.satgate.filestream:cms:jar:1.0.0
[INFO] +- org.projectlombok:lombok:jar:1.16.6:provided
[INFO] +- com.maxmind.geoip2:geoip2:jar:2.6.0:compile
[INFO] | +- com.maxmind.db:maxmind-db:jar:1.2.0:compile
[INFO] | | - com.fasterxml.jackson.core:jackson-databind:jar:2.6.5:compile
[INFO] | | +- com.fasterxml.jackson.core:jackson-annotations:jar:2.6.5:compile
[INFO] | | - com.fasterxml.jackson.core:jackson-core:jar:2.6.5:compile
[INFO] | - com.google.http-client:google-http-client:jar:1.21.0:compile
[INFO] | +- com.google.code.findbugs:jsr305:jar:1.3.9:compile
[INFO] | - org.apache.httpcomponents:httpclient:jar:4.5.1:compile
[INFO] | +- org.apache.httpcomponents:httpcore:jar:4.4.4:compile
[INFO] | - commons-codec:commons-codec:jar:1.9:compile
[INFO] +- org.springframework.boot:spring-boot-starter-web:jar:1.3.2.RELEASE:compile
[INFO] | +- org.springframework.boot:spring-boot-starter:jar:1.3.2.RELEASE:compile
[INFO] | | +- org.springframework.boot:spring-boot:jar:1.3.2.RELEASE:compile
[INFO] | | +- org.springframework.boot:spring-boot-autoconfigure:jar:1.3.2.RELEASE:compile
[INFO] | | +- org.springframework.boot:spring-boot-starter-logging:jar:1.3.2.RELEASE:compile
[INFO] | | | +- ch.qos.logback:logback-classic:jar:1.1.3:compile
[INFO] | | | | - ch.qos.logback:logback-core:jar:1.1.3:compile
[INFO] | | | +- org.slf4j:jul-to-slf4j:jar:1.7.13:compile
[INFO] | | | - org.slf4j:log4j-over-slf4j:jar:1.7.13:compile
[INFO] | | - org.yaml:snakeyaml:jar:1.16:runtime
[INFO] | +- org.springframework.boot:spring-boot-starter-tomcat:jar:1.3.2.RELEASE:compile
[INFO] | | +- org.apache.tomcat.embed:tomcat-embed-core:jar:8.0.30:compile
[INFO] | | +- org.apache.tomcat.embed:tomcat-embed-el:jar:8.0.30:compile
[INFO] | | +- org.apache.tomcat.embed:tomcat-embed-logging-juli:jar:8.0.30:compile
[INFO] | | - org.apache.tomcat.embed:tomcat-embed-websocket:jar:8.0.30:compile
[INFO] | +- org.springframework.boot:spring-boot-starter-validation:jar:1.3.2.RELEASE:compile
[INFO] | | - org.hibernate:hibernate-validator:jar:5.2.2.Final:compile
[INFO] | | +- javax.validation:validation-api:jar:1.1.0.Final:compile
[INFO] | | +- org.jboss.logging:jboss-logging:jar:3.3.0.Final:compile
[INFO] | | - com.fasterxml:classmate:jar:1.1.0:compile
[INFO] | +- org.springframework:spring-web:jar:4.2.4.RELEASE:compile
[INFO] | | - org.springframework:spring-beans:jar:4.2.4.RELEASE:compile
[INFO] | - org.springframework:spring-webmvc:jar:4.2.4.RELEASE:compile
[INFO] | - org.springframework:spring-expression:jar:4.2.4.RELEASE:compile
[INFO] +- org.springframework.boot:spring-boot-starter-aop:jar:1.3.2.RELEASE:compile
[INFO] | +- org.springframework:spring-aop:jar:4.2.4.RELEASE:compile
[INFO] | | - aopalliance:aopalliance:jar:1.0:compile
[INFO] | - org.aspectj:aspectjweaver:jar:1.8.8:compile
[INFO] +- org.springframework.boot:spring-boot-starter-cache:jar:1.3.2.RELEASE:compile
[INFO] | +- org.springframework:spring-context:jar:4.2.4.RELEASE:compile
[INFO] | - org.springframework:spring-context-support:jar:4.2.4.RELEASE:compile
[INFO] +- org.springframework.boot:spring-boot-starter-data-mongodb:jar:1.3.2.RELEASE:compile
[INFO] | +- org.mongodb:mongo-java-driver:jar:2.13.3:compile
[INFO] | - org.springframework.data:spring-data-mongodb:jar:1.8.2.RELEASE:compile
[INFO] | +- org.springframework:spring-tx:jar:4.2.4.RELEASE:compile
[INFO] | +- org.springframework.data:spring-data-commons:jar:1.11.2.RELEASE:compile
[INFO] | +- org.slf4j:slf4j-api:jar:1.7.13:compile
[INFO] | - org.slf4j:jcl-over-slf4j:jar:1.7.13:compile
[INFO] +- org.springframework.boot:spring-boot-starter-redis:jar:1.3.2.RELEASE:compile
[INFO] | +- org.springframework.data:spring-data-redis:jar:1.6.2.RELEASE:compile
[INFO] | - redis.clients:jedis:jar:2.7.3:compile
[INFO] | - org.apache.commons:commons-pool2:jar:2.4.2:compile
[INFO] +- org.springframework.boot:spring-boot-starter-security:jar:1.3.2.RELEASE:compile
[INFO] | +- org.springframework.security:spring-security-config:jar:4.0.3.RELEASE:compile
[INFO] | | - org.springframework.security:spring-security-core:jar:4.0.3.RELEASE:compile
[INFO] | - org.springframework.security:spring-security-web:jar:4.0.3.RELEASE:compile
[INFO] +- org.springframework.boot:spring-boot-starter-test:jar:1.3.2.RELEASE:test
[INFO] | +- org.mockito:mockito-core:jar:1.10.19:test
[INFO] | | - org.objenesis:objenesis:jar:2.1:test
[INFO] | +- org.hamcrest:hamcrest-core:jar:1.3:compile
[INFO] | +- org.hamcrest:hamcrest-library:jar:1.3:test
[INFO] | +- org.springframework:spring-core:jar:4.2.4.RELEASE:compile
[INFO] | - org.springframework:spring-test:jar:4.2.4.RELEASE:test
[INFO] - junit:junit:jar:4.12:compile

What i could be doing wrong?

@oschwald

This comment has been minimized.

Member

oschwald commented Feb 8, 2016

@akashihi, it appears you are pulling in jackson-databind 2.6.5 when maxmind-db 1.2.0 requires 2.7.0.

@akashihi

This comment has been minimized.

akashihi commented Feb 8, 2016

@oschwald maven is so weird :-/

@oschwald

This comment has been minimized.

Member

oschwald commented Mar 18, 2016

As this issue has come up so much, I have specified a hard minimum dependency on 2.7.0. The soft dependency in maxmind-db was not working too well in the case of conflicts with Maven's nearest-first resolution rules.

@sonya

This comment has been minimized.

sonya commented Apr 28, 2016

For people running into this on Spark, the hard minimum dependency in this library may be ignored at runtime because Spark provides its own version (2.4.4 as of Spark 1.6.1). For now I think @laxatives' solution is the way to get around this (though I'm not sure why you only had to shade core). For sbt I used the following

assemblyShadeRules in assembly := Seq(
  ShadeRule.rename("com.fasterxml.jackson.core.**"       -> "shadedjackson.core.@1").inAll,
  ShadeRule.rename("com.fasterxml.jackson.annotation.**" -> "shadedjackson.annotation.@1").inAll,
  ShadeRule.rename("com.fasterxml.jackson.databind.**"   -> "shadedjackson.databind.@1").inAll
)
@oschwald

This comment has been minimized.

Member

oschwald commented Apr 28, 2016

Although I am not familiar with Spark, if you are unable to provide your own Jackson version, you could try using a version of this library that doesn't use 2.7-specific functionality, e.g., 2.5.0.

@deepakas

This comment has been minimized.

deepakas commented Nov 28, 2016

I am having the same problem allocating the correct dependency version while using spark from Zeppelin. Any ideas ?

@oschwald

This comment has been minimized.

Member

oschwald commented Nov 28, 2016

@deepakas, does @sonya's suggestion above work for you? Alternatively, you could use 2.5.0 of geoip2, which doesn't require a recent version of Jackson.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment