Transitive dependency on jackson-databind pulls in -SNAPSHOT

[rocketraman]

The transitive dependency on com.maxmind.db:maxmind-db contains an open-ended version clause for jackson-databind: [2.7.0,). This causes gradle to pull in the 2.9.0-SNAPSHOT of jackson-databind into the project.

+--- com.maxmind.geoip2:geoip2:2.8.0
|    +--- com.maxmind.db:maxmind-db:1.2.1
|    |    \--- com.fasterxml.jackson.core:jackson-databind:[2.7.0,) -> 2.9.0-SNAPSHOT (*)
|    +--- org.apache.httpcomponents:httpclient:4.5.2 (*)
|    \--- com.fasterxml.jackson.core:jackson-databind:2.8.2 -> 2.9.0-SNAPSHOT (*)

This of course is a big problem -- the runtime now fails when the rest of the project uses a 2.8 release version of jackson.

I'm not sure why this has caused problems in our builds only recently because AFAICT this is not a new thing in maxmind-db. Still, it would be better if maxmind-db declared the version dependency explicitly instead of giving an open-ended range, which I think is the issue.

[oschwald]

The issue with a simple dependency such as jackson-databind: 2.7.0 is that Maven prefers the nearest declared dependency. This leads to situations where versions of jackson-databind older than 2.7.0 get pulled in, leading to runtime errors like #65 and #57 as 2.7.0+ is required. Although Maven's handling of version ranges is suboptimal, it was the only way to stop such reports.

As far as your particular issue, in my testing, Maven appears to prefer the nearest version as long as that version is in the range specified by maxmind-db. I am not sure how Gradle's dependency resolution differs, but I would be open to suggests that resolve both this and the original issue.

[rocketraman]

I see your point, but its very unusual for a library to set a dependency version this way. As a library author -- especially one that's not a core system library -- its not very nice force things your way when by doing so you're just moving the errors from your code to someone else's code. And you can't expect to cater your version declarations to work correctly with every build system that every project may use, and in every combination with every other transitive dependency.

I believe this is just a general transitive dependency resolution complexity that every competent developer has to deal with, since the lack of uptake on versioned module systems like OSGi prevented a comprehensive solution to versioning hell... Specifying which specific version you prefer and leaving it at that is following the principle of least surprise, as that is what pretty much every library author does [1]. And every consumer of your library has to be aware of the dependencies you pull in, and whether there are any conflicts in their projects -- both maven and gradle offer all the necessary tools to do this. IMO, you should document your version requirements front and center and leave it at that. Though you do have some additional options:

1. You could add runtime checks and warnings if this is a problem that happens often, making it possible for devs to more easily diagnose this issue on their own.

2. You may choose to offer shaded versions of your jars with dependencies included, to make it easy on devs that have to deal with irreconcilable version conflicts.

[1] In my current project, I have literally 7 other libraries that pull in jackson-databind, and maxmind is the only one that defines the version this way.

[rocketraman]

In addition: defining a version range makes builds non-repeatable. In my case, the same build that worked last week failed this week, I suspect because the first release of jackson 2.9.0-SNAPSHOT caused the dependency graph of my build to change, because of the way the maxmind dep is declared. At least if you specified a specific version, it might not prevent issues like the ones you referenced, but at least people's builds are repeatable!

[oschwald]

Version ranges are a standard part of Maven. Setting a minimum version when there is a hard minimum version seems completely appropriate. We went from several reports of issues per week without the minimum version to one issue in 6+ months with it. I agree that given Maven's poor dependency resolution algorithm, the version range is suboptimal, but both (1) and (2) would likely incur high support overhead for us and even more confusion for most users of the library.

As I mentioned, Maven 3.3.9, at least, does consistently pull in the nearest declared version within the version range for me. As geoip2 declares 2.8.2 as a dependency, it uses that unless another dependency is declared in the pom.xml or by a nearer dependency in the dependency tree. For example, if I create a pom.xml with only a dependency on geoip2, my dependency tree looks like:

[INFO] \- com.maxmind.geoip2:geoip2:jar:2.8.0:compile
[INFO]    +- com.maxmind.db:maxmind-db:jar:1.2.1:compile
[INFO]    +- org.apache.httpcomponents:httpclient:jar:4.5.2:compile
[INFO]    |  +- org.apache.httpcomponents:httpcore:jar:4.4.4:compile
[INFO]    |  +- commons-logging:commons-logging:jar:1.2:compile
[INFO]    |  \- commons-codec:commons-codec:jar:1.9:compile
[INFO]    \- com.fasterxml.jackson.core:jackson-databind:jar:2.8.2:compile
[INFO]       +- com.fasterxml.jackson.core:jackson-annotations:jar:2.8.0:compile
[INFO]       \- com.fasterxml.jackson.core:jackson-core:jar:2.8.2:compile

Perhaps you could provide a test case that reproduces the issue of the snapshot release being pulled in when you use geoip2.

[rocketraman]

Sure, all I have to do is pull in maxmind with a snapshots repo enabled, and nothing else:

build.gradle:

buildscript {
  repositories {
    mavenCentral()
  }
}

repositories {
  mavenCentral()
  maven {
    url "https://oss.sonatype.org/content/repositories/snapshots"
  }
}

apply plugin: 'java'

dependencies {
  compile("com.maxmind.geoip2:geoip2:2.8.0") {
    // workaround https://github.com/maxmind/GeoIP2-java/issues/77
    //exclude group: 'com.fasterxml.jackson.core'
  }
}

task wrapper(type: Wrapper) {
  gradleVersion = '3.1'
}

Dependency analysis (gradle dependencies --configuration compile):

\--- com.maxmind.geoip2:geoip2:2.8.0
     +--- com.maxmind.db:maxmind-db:1.2.1
     |    \--- com.fasterxml.jackson.core:jackson-databind:[2.7.0,) -> 2.9.0-SNAPSHOT
     |         +--- com.fasterxml.jackson.core:jackson-annotations:2.9.0-SNAPSHOT
     |         \--- com.fasterxml.jackson.core:jackson-core:2.9.0-SNAPSHOT
     +--- org.apache.httpcomponents:httpclient:4.5.2
     |    +--- org.apache.httpcomponents:httpcore:4.4.4
     |    +--- commons-logging:commons-logging:1.2
     |    \--- commons-codec:commons-codec:1.9
     \--- com.fasterxml.jackson.core:jackson-databind:2.8.2 -> 2.9.0-SNAPSHOT (*)

Gradle defaults to the highest version of the transitive deps if not specified as a primary dep. This isn't limited to -SNAPSHOT builds -- once Jackson upstream releases 2.9, the same issue would occur without a snapshot repo enabled.

You could define your range as [2.7.0,2.9.0) rather than keeping it open-ended. I still don't like it -- if every author of every non-core library like maxmind did the same, computing a closure for dependency graphs in any reasonable-size project will become a nightmare.

[rocketraman]

Version ranges are indeed part of maven. That doesn't mean they should be used everywhere -- there is a reason they aren't. Fundamentally they make builds unrepeatable -- the version chosen depends on changes external to the build -- and that is ok in some limited use cases, but not usually ok.

[oschwald]

From the above comment, this appears to be a Gradle-specific issue rather than a Maven issue. It is not making builds with Maven fundamentally unrepeatable as mentioned above. I'll take a look at Gradle to see if there is something we can do to stop this behavior.

[rocketraman]

@oschwald I think that's only because maven by default doesn't pull in snapshots. Once jackson 2.9.0 gets released, I suspect you'll have the same issue with maven too. If the jackson dep on geomind happens to be nearest to the top of the dependency tree in people's builds, their build will change to pull in 2.9.0.

[oschwald]

@rocketraman, I don't think that is the case. Maven will use the nearest declared dependency if it is in the range specified. Notice how it is pulling in 2.8.2 rather than 2.8.4. This is because geoip2 has depends on 2.8.2. It does not pull in the latest version in the range, as Gradle apparently does.

[rocketraman]

You're right, I was partially mistaken. The sample build above pulls in 2.8.2 because geoip2-java declares it, and as you say that's "nearer" the top of the dependency tree. But what if someone depended on geoip2-db directly, for some reason? In that case geoip2-db could be "nearer" the top of the tree. For example, this build pulls in 2.9.0-SNAPSHOT, and breaks the 2.8.2 dep required by jjwt:

<project xmlns="http://maven.apache.org/POM/4.0.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
                      http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>test</groupId>
  <artifactId>test</artifactId>
  <version>1.0</version>

  <repositories>
    <repository>
      <id>snapshots-repo</id>
      <url>https://oss.sonatype.org/content/repositories/snapshots</url>
      <releases><enabled>false</enabled></releases>
      <snapshots><enabled>true</enabled></snapshots>
    </repository>
  </repositories>

  <dependencies>
    <dependency>
      <groupId>com.maxmind.db</groupId>
      <artifactId>maxmind-db</artifactId>
      <version>1.2.1</version>
    </dependency>    
    <dependency>
      <groupId>io.jsonwebtoken</groupId>
      <artifactId>jjwt</artifactId>
      <version>0.7.0</version>
    </dependency>
  </dependencies>
</project>

results in this tree:

[INFO] test:test:jar:1.0
[INFO] +- com.maxmind.db:maxmind-db:jar:1.2.1:compile
[INFO] |  \- com.fasterxml.jackson.core:jackson-databind:jar:2.9.0-SNAPSHOT:compile (version selected from constraint [2.7.0,))
[INFO] |     +- com.fasterxml.jackson.core:jackson-annotations:jar:2.9.0-SNAPSHOT:compile
[INFO] |     \- com.fasterxml.jackson.core:jackson-core:jar:2.9.0-SNAPSHOT:compile
[INFO] \- io.jsonwebtoken:jjwt:jar:0.7.0:compile

even though jjwt wants jackson-databind 2.8.2 (http://search.maven.org/#artifactdetails%7Cio.jsonwebtoken%7Cjjwt%7C0.7.0%7Cjar)

[oschwald]

I agree that someone depending on maxmind-db will need to explicitly depend on jackson-databind to force a particular version. Fortunately, using maxmind-db directly is a pretty specialized use case and this is not an issue for most users.

[rocketraman]

You're relying on very specific configuration of your project(s), along with very specific behavior of very specific build systems here, by specifying a range. I still don't think its a good idea, when you have other options like runtime checks, and so forth.

However, in any case, I've raised an issue with the gradle team here: gradle/gradle#759. I know the workaround in gradle to force the dep version is simple, but I don't like that some random library can take my repeatable build and make it unrepeatable by default because they've catered to a maven-style dep resolution, and hopefully the gradle guys can think of a good solution to that.

[ben-bourdin451]

Looks like this issue happened for us this morning out of the blue. We're using maven and declaring a simple dependency to geoip2 2.7.0

<dependency>
    <groupId>com.maxmind.geoip2</groupId>
    <artifactId>geoip2</artifactId>
    <version>2.7.0</version>
</dependency>

Our pom dependencies haven't changed in a while (few months) so this came as a big surprise as to why this was suddenly happening. I haven't got much expertise with maven so I'm not sure what the root cause of the issue is but we've fixed it by excluding jackson explicitly and forcing our own version:

<dependency>
	<groupId>com.maxmind.geoip2</groupId>
	<artifactId>geoip2</artifactId>
	<version>2.7.0</version>
	<exclusions>
		<exclusion>
			<artifactId>jackson-core</artifactId>
			<groupId>com.fasterxml.jackson.core</groupId>
		</exclusion>
		<exclusion>
			<artifactId>jackson-databind</artifactId>
			<groupId>com.fasterxml.jackson.core</groupId>
		</exclusion>
	</exclusions>
</dependency>

[tholu]

+1 affected us as well. Will try the solution by @ben-bourdin451 - thanks!

[dlyle65535]

+1 also an issue for us. Would love to see it fixed here.

[tholu]

Solution by @ben-bourdin451 worked, here is our full solution in pom.xml with latest versions:

<!-- geoip -->
    <!-- explicitly included because of https://github.com/maxmind/GeoIP2-java/issues/77-->
    <dependency>
      <groupId>com.fasterxml.jackson.core</groupId>
      <artifactId>jackson-core</artifactId>
      <version>2.8.7</version>
    </dependency>
    <dependency>
      <groupId>com.fasterxml.jackson.core</groupId>
      <artifactId>jackson-databind</artifactId>
      <version>2.8.7</version>
    </dependency>
    <dependency>
      <groupId>com.maxmind.geoip2</groupId>
      <artifactId>geoip2</artifactId>
      <version>2.8.0</version>
      <exclusions>
        <exclusion>
          <artifactId>jackson-core</artifactId>
          <groupId>com.fasterxml.jackson.core</groupId>
        </exclusion>
        <exclusion>
          <artifactId>jackson-databind</artifactId>
          <groupId>com.fasterxml.jackson.core</groupId>
        </exclusion>
      </exclusions>
    </dependency>

Would be great if it is fixed here directly.

[nickwallen]

+1 Unpleasant surprise this morning. Would love to see a fix.

https://issues.apache.org/jira/browse/METRON-734

[oschwald]

The issue this morning is somewhat unrelated to the original Gradle issue, but we have stopped using version ranges, at least temporarily due to it.

[oschwald]

maxmind-db 1.2.2 and geoip2 2.8.1 have been released, which should fix this issue.

[ben-bourdin451]

Thanks @oschwald, apologies if this wasn't related to original issue. I can confirm the new version resolved this morning's problems

[tholu]

@oschwald When will 2.8.1 arrive on Maven Central? Thanks!

[oschwald]

@tholu, it was released on oss.sonatype.org ~15 minutes ago. I don't know the exact syncing schedule, but I would suspect it should be on most mirrors in a couple of hours.

[rocketraman]

The issue this morning is somewhat unrelated to the original Gradle issue, but we have stopped using version ranges, at least temporarily due to it.

I wouldn't say its unrelated at all -- if you read my comments above, I provided fair warning that using version ranges in the manner they were being used was not a good idea, regardless of build system. I'm glad you've come to that realization finally. :-)

[oschwald]

@rocketraman, your issue was specific to Gradle and only tangentially related to this issue. I do expect an onslaught of new reports of people experiencing issues due to their jackson-databind version being less than 2.7.0. As with most things in software, it is a delicate balance between the costs and benefits of the change.

[rocketraman]

@rocketraman, your issue was specific to Gradle

It was specific to Gradle, but I specifically stated you would in the future run into issues with Maven as well. I'm not going to argue it -- I'll let the previous record, and the current situation, speak for itself.

[marko-stankovic]

My company also had unpleasant surprise from yesterday, when our build started failing because it tries to fetch SNAPSHOT libraries dependencies that do not exist.

We managed to resolve it by excluding jackson-databind dependency from MaxMind library, but please do consider removing open version range from maxmind-db library for good (not just temporarily). Ranges make more problems than they solve, especially when used in libraries that are used by many other projects.

[gbartolomebb]

Hello, I actually sent customer service a problem with your new build. It seems to be missing a package that's trying to get a jackson method. I've been using your service for two years and nothing has failed except for the 2.8.1 release.

This is what I sent:

I think I found the error in why this method is not showing up:

java -cp ~/bod-ip-locate-1.0.jar com.bod.ipGeolocator.ipMainIPOnly
Exception in thread "main" java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.node.ArrayNode.(Lcom/fasterxml/jackson/databind/node/JsonNodeFactory;Ljava/util/List;)V
at com.maxmind.db.Decoder.decodeArray(Decoder.java:272)
at com.maxmind.db.Decoder.decodeByType(Decoder.java:156)
at com.maxmind.db.Decoder.decode(Decoder.java:147)
at com.maxmind.db.Decoder.decodeMap(Decoder.java:281)
at com.maxmind.db.Decoder.decodeByType(Decoder.java:154)
at com.maxmind.db.Decoder.decode(Decoder.java:147)
at com.maxmind.db.Decoder.decode(Decoder.java:87)
at com.maxmind.db.Reader.(Reader.java:132)
at com.maxmind.db.Reader.(Reader.java:89)
at com.maxmind.geoip2.DatabaseReader.(DatabaseReader.java:33)
at com.maxmind.geoip2.DatabaseReader.(DatabaseReader.java:23)
at com.maxmind.geoip2.DatabaseReader$Builder.build(DatabaseReader.java:129)

I blew up the geoip2-2.8.1.jar:

jar -tvf ~/geoip2-2.8.1.jar and I did not see package called: com.maxmind.db and I think that’s why it’s giving me a no method error it’s looking for a package with a method in it that does not exist. Please confirm this with your engineers.

 0 Wed Feb 22 07:58:22 PST 2017 META-INF/

465 Wed Feb 22 07:58:22 PST 2017 META-INF/MANIFEST.MF
0 Wed Feb 22 07:58:18 PST 2017 com/
0 Wed Feb 22 07:58:18 PST 2017 com/maxmind/
0 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/
0 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/model/
0 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/record/
0 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/exception/
771 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/WebServiceClient$1.class
1284 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/model/CountryResponse.class
1658 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/model/IspResponse.class
1368 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/model/ConnectionTypeResponse.class
2180 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/model/InsightsResponse.class
1849 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/model/AbstractResponse.class
2466 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/model/EnterpriseResponse.class
2665 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/model/AbstractCountryResponse.class
1766 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/model/ConnectionTypeResponse$ConnectionType.class
2539 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/model/CityResponse.class
4068 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/model/AbstractCityResponse.class
995 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/model/DomainResponse.class
1690 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/model/AnonymousIpResponse.class
790 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/record/MaxMind.class
1625 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/record/Subdivision.class
3559 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/record/Traits.class
1596 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/record/Country.class
899 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/record/Postal.class
2228 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/record/AbstractNamedRecord.class
1546 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/record/RepresentedCountry.class
1963 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/record/Location.class
1845 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/record/AbstractRecord.class
1317 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/record/City.class
1338 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/record/Continent.class
2518 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/WebServiceClient$Builder.class
408 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/GeoIp2Provider.class
2386 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/DatabaseReader$Builder.class
6986 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/DatabaseReader.class
1051 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/exception/HttpException.class
626 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/exception/PermissionRequiredException.class
681 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/exception/AddressNotFoundException.class
1166 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/exception/InvalidRequestException.class
633 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/exception/GeoIp2Exception.class
678 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/exception/AuthenticationException.class
672 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/exception/OutOfQueriesException.class
1393 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/JsonInjector.class
14522 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/WebServiceClient.class
774 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/DatabaseProvider.class
228 Wed Feb 22 07:58:18 PST 2017 com/maxmind/geoip2/DatabaseReader$1.class
0 Wed Feb 22 07:58:22 PST 2017 META-INF/maven/
0 Wed Feb 22 07:58:22 PST 2017 META-INF/maven/com.maxmind.geoip2/
0 Wed Feb 22 07:58:22 PST 2017 META-INF/maven/com.maxmind.geoip2/geoip2/
8228 Wed Feb 22 07:58:16 PST 2017 META-INF/maven/com.maxmind.geoip2/geoip2/pom.xml
116 Wed Feb 22 07:58:22 PST 2017 META-INF/maven/com.maxmind.geoip2/geoip2/pom.properties

[oschwald]

@gbartolomebb, it sounds like you are pulling in an old version pre-2.7.0 of jackson-databind. This is the issue that the dependency range in the previous releases was there to solve. Maven pulls in the nearest dependency by default. See #65 and #57 for previous discussions of this.

[gbartolomebb]

Thanks my pom.xml explicitly references the current Jackson core build (2.8.7), geoip2-2.8.1.jar is missing a package folder called db. It's trying to access a method in a class com.maxmind.db.Decoder.decodeArray from Jackson that doesn't exist:

com.maxmind.db does not exist:
jar -tvf ~/geoip2-2.8.1.jar
com/maxmind/geoip2/
com/maxmind/geoip2/model/
com/maxmind/geoip2/record/
com/maxmind/geoip2/exception/

pom dependencies:
dependencies
dependency
groupIdcom.bod/groupId
artifactIdbod-awsBB-etl/artifactId
version1.0/version
/dependency
dependency
groupIdcom.maxmind.geoip2/groupId
artifactIdgeoip2/artifactId
version2.8.1/version
/dependency
dependency
groupIdcom.fasterxml.jackson.core/groupId
artifactIdjackson-databind/artifactId
version2.8.7/version
/dependency
/dependencies

[oschwald]

That constructor exists in 2.8.7. See here for the call on our side.

If you can create a self-contained test project to exhibit your issue, I would be happy to look into it further. However, given the above and that I can successfully use jackson-databind 2.8.7 with this library, I am not sure what else I can offer.

[gbartolomebb]

Thanks now that you pointed me out to your code I saw that Jackson was a dependency of the MaxMind Reader DB once I added an explicit dependency on: <dependency> <groupId>com.maxmind.db</groupId> <artifactId>maxmind-db</artifactId> <version>1.2.1</version> </dependency> It worked. Odd I didn’t have to do that with 2.8.0 Best,

[tckb]

Nearing a build release, I think this would a rather of a big surprise to see all your "production" ready builds failing. We have multiple of our projects depending on it and all of them do depend on "maxmind-db" . I'd had to explicitly remove the dependency of jackson-databind and add non-snapshot dependency to by pass this issue. Glad to see this is fixed in 2.8.1 will check if it is working.