The class 'ExecutionPlan_1.2' is not in the trusted packages | Old Changes in "develop" branch not available under latest release tags.

[ganeshkumar269]

When trying to use kafka-gateway to ingest lineage data that was created from a spark job, I got the below error.

spline-demo-spline-kafka-server-1 | org.springframework.kafka.listener.ListenerExecutionFailedException: Listener failed; nested exception is java.lang.IllegalArgumentException: The class 'ExecutionPlan_1.2' is not in the trusted packages: [java.util, java.lang]. If you believe this class is safe to deserialize, please provide its name. If the serialization is only done by a trusted source, you can also enable trust all (*).

Versions

Spark Version: 3.3.0, Scala Version: 2.12
spark-spline-agent:2.0.0.
spline:0.7.8 / kafka-gateway:0.7.8

I went through the code (checkout to tag release/0.7.8) and in KafkaGatewayConfig

private val typeMappings = Map[String, Class[_]](
    "ExecutionPlan" -> classOf[za.co.absa.spline.producer.model.v1_1.ExecutionPlan],
    "ExecutionEvent" -> classOf[za.co.absa.spline.producer.model.v1_1.ExecutionEvent]
  )

found ExecutionEvent_1.2 is not added to the typeMappings. but the mappings are present in develop branch.

private val typeMappings: Map[String, Class[_]] = Map(
    "ExecutionPlan" -> classOf[za.co.absa.spline.producer.model.v1_1.ExecutionPlan],
    "ExecutionEvent" -> classOf[za.co.absa.spline.producer.model.v1_1.ExecutionEvent],

    "ExecutionPlan_1.1" -> classOf[za.co.absa.spline.producer.model.v1_1.ExecutionPlan],
    "ExecutionEvent_1.1" -> classOf[za.co.absa.spline.producer.model.v1_1.ExecutionEvent],

    "ExecutionPlan_1.2" -> classOf[za.co.absa.spline.producer.model.v1_2.ExecutionPlan],
    "ExecutionEvent_1.2" -> classOf[za.co.absa.spline.producer.model.v1_2.ExecutionEvent],
  )

one more thing to notice is the changes in develop branch are merged 2y back and yet not available on latest release tags.

[cerveada]

As a workaround you can configure the agent to use the older version API (v1_1) though you will lose labels added in v1_2. Otherwise, there is no big difference.

[wajda]

Producer model v1.2 support was added in Spline server 1.0.0 which will be released in a few days.