Jaeger 1.5.0 breaks dependency processing due to NullPointerException

[masteinhauser]

With the latest release of Jaeger, the data schema used internally and within the Elasticsearch integration triggers a Java NullPointerException during processing. (I uncertain about Cassandra, as I am using Elasticsearch only.)

Specifically, this appears related to jaegertracing/jaeger#831 and the expectation this field exists during deserialization: https://github.com/jaegertracing/spark-dependencies/blob/master/jaeger-spark-dependencies-elasticsearch/src/main/java/io/jaegertracing/spark/dependencies/elastic/json/SpanDeserializer.java#L46

Caused by: java.lang.NullPointerException
        at io.jaegertracing.spark.dependencies.elastic.json.SpanDeserializer.deserialize(SpanDeserializer.java:46)
        at io.jaegertracing.spark.dependencies.elastic.json.SpanDeserializer.deserialize(SpanDeserializer.java:32)
        at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3736)
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2726)
        at io.jaegertracing.spark.dependencies.elastic.ElasticTupleToSpan.call(ElasticTupleToSpan.java:31)
        at io.jaegertracing.spark.dependencies.elastic.ElasticTupleToSpan.call(ElasticTupleToSpan.java:25)
        at org.apache.spark.api.java.JavaPairRDD$$anonfun$toScalaFunction$1.apply(JavaPairRDD.scala:1040)
        at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)
        at scala.collection.Iterator$$anon$11.next(Iterator.scala:409)
        at org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:150)
        at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:96)
        at org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:53)
        at org.apache.spark.scheduler.Task.run(Task.scala:99)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:322)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

See the attached log file for the full log of the job processing. This is taken from a Lab environment using Hotrod as I am still working on our production deployment.

jaeger-sparkdependencies - parentSpanID nullPointerException - full log

[pavolloffay]

@masteinhauser thanks for reporting, now we will have to read references array #8.

Before fixing this issue we should also tag it for legacy users.

[pavolloffay]

We could implement references, but also keep the code for paretnId so it can be used with older schema. Simple null check around parentSpanID would be sufficient.

[pavolloffay]

done in #33.

@masteinhauser the change should be in the latest image.