-
Notifications
You must be signed in to change notification settings - Fork 4.5k
Open
Labels
Description
Our existing data pipeline failed when we added the new JSON column to a Spanner table that it writes to. The following is the stack trace.
Error message from worker: java.lang.IllegalArgumentException: Unknown spanner type JSON
org.apache.beam.sdk.io.gcp.spanner.SpannerSchema$Column.parseSpannerType(SpannerSchema.java:179)
org.apache.beam.sdk.io.gcp.spanner.SpannerSchema$Column.create(SpannerSchema.java:142)
org.apache.beam.sdk.io.gcp.spanner.SpannerSchema$Builder.addColumn(SpannerSchema.java:74)
org.apache.beam.sdk.io.gcp.spanner.ReadSpannerSchema.processElement(ReadSpannerSchema.java:63)
I included a proposed patch below that has not yet been tested.
https://github.com/apache/beam/compare/master...vilai-paypal:spanner_json_schema?expand=1
Imported from Jira BEAM-13470. Original Jira may contain additional context.
Reported by: vilai.