Spark MapType conversion no supported?

[kurtmaile]

Hi,

I have a MapType struct in my schema, and I am getting the error below when converting - I assume then this isnt supported? Is there any (soonish) plans to do so?

Thanks!

java.lang.IllegalArgumentException: Unsupported type: MapType(StringType,BooleanType,true)
at com.samelamin.spark.bigquery.converters.SchemaConverters$.typeToJson(SchemaConverters.scala:73) at com.samelamin.spark.bigquery.converters.SchemaConverters$.com$samelamin$spark$bigquery$converters$SchemaConverters$$fieldToJson(SchemaConverters.scala:81) at com.samelamin.spark.bigquery.converters.SchemaConverters$$anonfun$typeToJson$2.apply(SchemaConverters.scala:64) at com.samelamin.spark.bigquery.converters.SchemaConverters$$anonfun$typeToJson$2.apply(SchemaConverters.scala:64) at ....

[samelamin]

Hi @kurtmaile at the moment MapType isnt supported, please feel free to send a pr

The code for the converters can be found here

[kurtmaile]

Hey thanks - Id love to, not so familiar with the codebase, based on your experience of other types how long and how hard do you think this is (as up against it for time on a PoC but would like to contribute back to your repo :)

[samelamin]

Hey @kurtmaile shouldnt take more than an hour or two, just needs to add the maptype to the switch statements and the reverse is true

you have to figure out what the maptype would translate to in BigQuery world

[kurtmaile]

cool Ill submit the PR in the next few days then - cheers for your help

[samelamin]

@kurtmaile thanks! and if you can get a chance can you please rate the package