java.lang.NoSuchMethodError on com.google.common.io.Closeables.closeQuietly

[raymondtay]

Encountered a NoSuchMethodError when running persistence inside a cluster, here's the stack trace:

Uncaught error from thread [TrailingHammer-akka.persistence.dispatchers.default-plugin-dispatcher-25] shutting down JVM since 'akka.jvm-exit-on-fatal-error' is enabled for ActorSystem[TrailingHammer]
java.lang.NoSuchMethodError: com.google.common.io.Closeables.closeQuietly(Ljava/io/Closeable;)V
    at org.iq80.leveldb.impl.MMapLogWriter.close(MMapLogWriter.java:83)
    at org.iq80.leveldb.impl.VersionSet.initializeIfNeeded(VersionSet.java:111)
    at org.iq80.leveldb.impl.VersionSet.<init>(VersionSet.java:91)
    at org.iq80.leveldb.impl.DbImpl.<init>(DbImpl.java:167)
    at org.iq80.leveldb.impl.Iq80DBFactory.open(Iq80DBFactory.java:59)
    at akka.persistence.journal.leveldb.LeveldbStore$class.preStart(LeveldbStore.scala:114)
    at akka.persistence.journal.leveldb.LeveldbJournal.preStart(LeveldbJournal.scala:20)
    at akka.actor.Actor$class.aroundPreStart(Actor.scala:470)
    at akka.persistence.journal.leveldb.LeveldbJournal.aroundPreStart(LeveldbJournal.scala:20)
    at akka.actor.ActorCell.create(ActorCell.scala:580)
    at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:456)
    at akka.actor.ActorCell.systemInvoke(ActorCell.scala:478)
    at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:263)
    at akka.dispatch.Mailbox.run(Mailbox.scala:219)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:722)

The application is not running native LevelDB as i've turned native off
akka.persistence.journal.leveldb.native=off

Application is build using:

• Oracle JDK 1.7.0_17
• SBT 0.13.5-M2
• Play 2.3, Akka 2.3 (persistence + cluster)

Application is packaged using sbt dist

[raymondtay]

Based on the libraries that sbt dist has produced, i could see that there's 1 library com.google.guava.guava-16.0.1.jar and decompiled the Java class in question which is Closeable.java and it does appear that the akka-persistence is using another version of the google guava libs than the one it is expecting to have.

Raymonds-MacBook-Pro:io raymondtay$ javap Closeables.class
Compiled from "Closeables.java"
public final class com.google.common.io.Closeables {
  static final java.util.logging.Logger logger;
  public static void close(java.io.Closeable, boolean) throws java.io.IOException;
  static {};
}

Read some information from Google Guava's website (https://code.google.com/p/guava-libraries/issues/list?can=1&q=closeQuietly&colspec=ID+Type+Status+Package+Summary&cells=tiles) suggests that closeQuietly was deprecated and now someone wants to bring closeQuietly back. Frankly, i need some help here.
Would appreciate it if the team could advise me which version of Guava to use?

[raymondtay]

Turns out the Akka 2.3.2 is using

val levelDB       = "org.iq80.leveldb"            % "leveldb"                      % "0.5"

as highlighted by this → https://github.com/akka/akka/blob/v2.3.2/project/AkkaBuild.scala

It really should be

val levelDB       = "org.iq80.leveldb"            % "leveldb"                      % "0.7"

In the Play 2.3 application's build file Build.scala i've overrided that _0.5_jar with the 0.7 jar. Works now.

[rkuhn]

Are these two versions binary compatible? (given the version numbers I doubt that) Unfortunately we cannot break BC within the 2.3.x series.

[agourlay]

Hi @RayGit,
FYI this is a problem that I already encountered see #2063
It was merged in master.

[raymondtay]

@agourlay , thanks for the referral to that ticket number. Hadn't seen that ticket til now, but i noticed the codebase is as what you said i.e. merged to master. I'm sort of stuck as i can't push the code to trunk as it disrupts our release cycle significantly though i know its already merged to master but i have a workaround so that abates the issue.

@rkuhn, still running the tests for my setup over here. But noted on not being able to break BC. At least i have a workaround.

[raymondtay]

@rkuhn the tests i've run are mostly persisting and read-back of the data seem ok but i haven't test it extensively (meaning to say code coverage is around 80%); hence i'll wait till a stable release is made. closing this for now.

[patriknw]

This was fixed in master. The reason we have not fixed it in release-2.3 is that we are afraid that changing leveldb version brings in a new version of guava in a subtle way that might break binary compatibility of other parts of the application.

[patriknw]

The workaround is to define the dependency in the application build:

val akkaPersistence       = "com.typesafe.akka"       %% "akka-persistence-experimental"   % V.akka     exclude("org.iq80.leveldb","leveldb")
val levelDbFixed          = "org.iq80.leveldb"        %  "leveldb"                         % "0.7"

[raymondtay]

Hi @patriknw, thanks for getting back on this issue. Yeah i've discovered that workaround the day i closed this issue and i guessed you guys had a good reason for not releasing it as part of 2.3.