3.3.0 gives java.lang.NoSuchMethodError when used in Spark

[tomvandenberge]

When using json4s 3.3.0 in Spark (1.5.0), I'm getting a NoSuchMethodError on org.json4s.Serialization$class.read(Lorg/json4s/Serialization;Ljava/lang/String;Lorg/json4s/Formats;Lscala/reflect/Manifest;)Ljava/lang/Object.

The error does not occur when using json4s 3.2.11 (!)

To reproduce the problem, all you need to do is parse a trivial piece of json. First, start the Spark console:
bin/spark-shell --packages "org.json4s:json4s-native_2.10:3.3.0"
Then, execute

import org.json4s._
import org.json4s.native.Serialization
import org.json4s.native.Serialization.read

case class Fu(bar: String)
val rdd = sc.parallelize(Array("""{"bar":"baz"}"""))
val parsed = rdd.map({json => 
  implicit val formats = native.Serialization.formats(NoTypeHints)
  read[Fu](json)
})
parsed.count

Strangly enough, this works fine when not using Spark, or when using an older version of json4s (3.2.11) with Spark.

[xuwei-k]

Spark 1.5.0 depends on json4s 3.2.10. json4s 3.2.10 and 3.3.0 does not have binary compatibility. You can't use spark 1.5.0 and json4s 3.3.0 together.