ScalaIteratorSerializer overrides final method withResolved

[ryantenney]

A change introduced in jackson-databind 2.6.0-rc3 breaks jackson-module scala
with the error (edited for ease of reading) class ScalaIteratorSerializer overrides final method withResolved(BeanProperty;TypeSerializer;JsonSerializer;)AsArraySerializerBase;. Would a better option be to make that method non-final so all modules which use it don't break?

FasterXML/jackson-databind@9ed3abf#diff-1a6d4c0e2d1a53346aa8b087ecddbec2R138

[cowtowncoder]

@ryantenney Bit tough call here; method is being deprecated, replaced by new variant:

/**
 * @since 2.6
 */
public abstract AsArraySerializerBase<T> withResolved(BeanProperty property,
        TypeSerializer vts, JsonSerializer<?> elementSerializer,
        Boolean unwrapSingle);

and final is there to explicitly for sub-classes to migrate; since the deprecated method is no longer being called by anything within jackson-databind, so allowing override will most likely just hide the problem.

So: the real fix is to change override to match new variant.
If it makes sense I can also remove final from obsolete variant, but I am not sure that really helps.

[christophercurrie]

I don't think there is a need to remove final; the RC releases are to find migration challenges like this. I'll fix it before rc4.

[cowtowncoder]

Thanks. These are tricky changes, and Scala module is one of modules that require deep(er) integration than most. XML dataformat is similarly fragile by necessity.

FWIW, this specific change is to allow use of new JsonFormat.Feature.WRITE_SINGLE_ELEM_ARRAYS_UNWRAPPED on per-property basis.

[christophercurrie]

This has been fixed in 2.6.0-rc4.

IMPORTANT NOTE: Version 2.6.0 was incorrectly released and should not be used. An updated release will be available as soon as possible.

[anujlal01]

i am using 2.6.1 and still getting this error
main" java.lang.VerifyError: class com.fasterxml.jackson.module.scala.ser.ScalaIteratorSerializer overrides final method withResolved.(Lcom/fasterxml/jackson/databind/BeanProperty;Lcom/fasterxml/jackson/databind/jsontype/TypeSerializer;Lcom/fasterxml/jackson/databind/JsonSerializer;)Lcom/fasterxml/jackson/databind/ser/std/AsArraySerializerBase;

[cowtowncoder]

@anujlal01 Make sure you do not have multiple versions of scala module in your classpath.

[anujlal01]

i have checked all dependency
SFO1212434751D:SparkStreaming-Master_forHWX 212434751$ mvn dependency:tree | grep -i 'jackson'
[INFO] +- com.fasterxml.jackson.core:jackson-annotations:jar:2.4.4:compile
[INFO] +- com.fasterxml.jackson.core:jackson-databind:jar:2.4.4:compile
[INFO] +- com.fasterxml.jackson.core:jackson-core:jar:2.4.4:compile
[INFO] | +- org.json4s:json4s-jackson_2.10:jar:3.2.10:compile
[INFO] | +- com.fasterxml.jackson.module:jackson-module-scala_2.10:jar:2.4.4:compile
[INFO] | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.8.8:compile
[INFO] | | | - org.codehaus.jackson:jackson-xc:jar:1.8.3:compile
[INFO] +- org.codehaus.jackson:jackson-core-asl:jar:1.8.8:compile
[INFO] +- org.codehaus.jackson:jackson-jaxrs:jar:1.8.8:compile

[anujlal01]

sorry, this error comes when i am using 2.6.1 version not when i ma using 2.4.4

[anujlal01]

when i switched to 1.6.1, I am getting java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.JavaType.isReferenceType()Z

Exception in thread "main" java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.JavaType.isReferenceType()Z
at com.fasterxml.jackson.databind.ser.BasicSerializerFactory.findSerializerByLookup(BasicSerializerFactory.java:309)
at com.fasterxml.jackson.databind.ser.BeanSerializerFactory._createSerializer2(BeanSerializerFactory.java:218)
at com.fasterxml.jackson.databind.ser.BeanSerializerFactory.createSerializer(BeanSerializerFactory.java:153)
at com.fasterxml.jackson.databind.SerializerProvider._createUntypedSerializer(SerializerProvider.java:1199)
at com.fasterxml.jackson.databind.SerializerProvider._createAndCacheUntypedSerializer(SerializerProvider.java:1153)
at com.fasterxml.jackson.databind.SerializerProvider.findValueSerializer(SerializerProvider.java:481)
at com.fasterxml.jackson.databind.SerializerProvider.findTypedValueSerializer(SerializerProvider.java:679)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:107)
at com.fasterxml.jackson.databind.ObjectMapper._configAndWriteValue(ObjectMapper.java:3525)
at com.fasterxml.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:2915)
at org.apache.spark.rdd.RDDOperationScope.toJson(RDDOperationScope.scala:50)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:141)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:108)
at org.apache.spark.SparkContext.withScope(SparkContext.scala:681)
at org.apache.spark.SparkContext.newAPIHadoopRDD(SparkContext.scala:1094)
at com.ge.datalake.rss.HBaseTest$.main(HBaseTest.scala:54)
at com.ge.datalake.rss.HBaseTest.main(HBaseTest.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:665)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:170)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:193)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:112)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

My pom dependency is like this
mvn dependency:tree | grep -i 'jackson'
[INFO] +- com.fasterxml.jackson.core:jackson-annotations:jar:2.6.1:compile
[INFO] +- com.fasterxml.jackson.core:jackson-databind:jar:2.6.1:compile
[INFO] +- com.fasterxml.jackson.core:jackson-core:jar:2.6.1:compile
[INFO] +- com.fasterxml.jackson.module:jackson-module-scala_2.10:jar:2.6.1:compile
[INFO] +- com.fasterxml.jackson.module:jackson-module-paranamer:jar:2.6.1:compile
[INFO] | +- org.json4s:json4s-jackson_2.10:jar:3.2.10:compile
[INFO] | +- org.codehaus.jackson:jackson-mapper-asl:jar:1.8.8:compile
[INFO] | | | - org.codehaus.jackson:jackson-xc:jar:1.8.3:compile
[INFO] +- org.codehaus.jackson:jackson-core-asl:jar:1.8.8:compile
[INFO] +- org.codehaus.jackson:jackson-jaxrs:jar:1.8.8:compile

[anujlal01]

never mind. this was false alarm.

[phrinx]

I ran into the same problem and found that I was not using the same version of the java and scala module:

com.fasterxml.jackson.core:jackson-databind
com.fasterxml.jackson.module:jackson-module-scala

Using the same version fixes it for me.

[K3T-1]

@phrinx This helped me too. Thanks. Here's how my gradle configuration looks like:
compile ("com.fasterxml.jackson.module:jackson-module-scala_2.10:2.6.5")
compile ("com.fasterxml.jackson.core:jackson-databind:2.6.5")

[azzurolilc]

@phrinx Thanks~~~This helped me tooo0!
My scala dependencies somehow point to jackson version 2.4.4
so I added library dependencies to my sbt file:
"com.fasterxml.jackson.core" % "jackson-databind" % "2.7.4",
"com.fasterxml.jackson.module" % "jackson-module-scala_2.10" % "2.7.4"

Issue fixed!

[sreeraaman]

@phrinx That helped me as well. Thanks a ton.
I was using play 2.5.8 / spark 2.0.0 and cassandra 3.7
The following entries in my build.sbt resolved the issue.

"com.datastax.spark" %% "spark-cassandra-connector-unshaded" % "2.0.0-M3",
"com.typesafe.play" %% "play-json" % "2.5.8" exclude("com.fasterxml.jackson.core", "jackson-databind"),
"com.fasterxml.jackson.core" % "jackson-databind" % "2.8.2",
"com.fasterxml.jackson.module" % "jackson-module-scala_2.11" % "2.8.2",
"org.apache.spark" %% "spark-core" % "2.0.0-preview",
"org.apache.spark" %% "spark-sql" % "2.0.0-preview"