Problem with encoding

[zaleslaw]

Hi, I've got the next error on the code example

The datasets could be downloaded here

Exception in thread "main" java.util.NoSuchElementException: None.get
at scala.None$.get(Option.scala:529)
at scala.None$.get(Option.scala:527)
at org.apache.spark.sql.KotlinReflection$.$anonfun$serializerFor$12(KotlinReflection.scala:658)
at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:238)
at scala.collection.IndexedSeqOptimized.foreach(IndexedSeqOptimized.scala:36)
at scala.collection.IndexedSeqOptimized.foreach$(IndexedSeqOptimized.scala:33)
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:198)
at scala.collection.TraversableLike.map(TraversableLike.scala:238)
at scala.collection.TraversableLike.map$(TraversableLike.scala:231)
at scala.collection.mutable.ArrayOps$ofRef.map(ArrayOps.scala:198)
at org.apache.spark.sql.KotlinReflection$.$anonfun$serializerFor$1(KotlinReflection.scala:655)
at scala.reflect.internal.tpe.TypeConstraints$UndoLog.undo(TypeConstraints.scala:69)
at org.apache.spark.sql.KotlinReflection.cleanUpReflectionObjects(KotlinReflection.scala:864)
at org.apache.spark.sql.KotlinReflection.cleanUpReflectionObjects$(KotlinReflection.scala:863)
at org.apache.spark.sql.KotlinReflection$.cleanUpReflectionObjects(KotlinReflection.scala:47)
at org.apache.spark.sql.KotlinReflection$.serializerFor(KotlinReflection.scala:494)
at org.apache.spark.sql.KotlinReflection$.serializerFor(KotlinReflection.scala:481)
at org.apache.spark.sql.KotlinReflection.serializerFor(KotlinReflection.scala)
at org.jetbrains.spark.api.ApiV1Kt.kotlinClassEncoder(ApiV1.kt:102)
at org.jetbrains.spark.api.ApiV1Kt.generateEncoder(ApiV1.kt:91)
at ExampleKt.main(example.kt:109)
at ExampleKt.main(example.kt)

[asm0dey]

Thank you for reporting this!

This is due to naming of fields in data class does not correspond to field names in dataset. Error message will be improved/

[zaleslaw]

@asm0dey it doesn't working yet