Skip to content

Spanner JSON Type Not Supported When Reading Schema #21337

@damccorm

Description

@damccorm

Our existing data pipeline failed when we added the new JSON column to a Spanner table that it writes to.  The following is the stack trace.


 Error message from worker: java.lang.IllegalArgumentException: Unknown spanner type JSON
  org.apache.beam.sdk.io.gcp.spanner.SpannerSchema$Column.parseSpannerType(SpannerSchema.java:179)

 org.apache.beam.sdk.io.gcp.spanner.SpannerSchema$Column.create(SpannerSchema.java:142)
  org.apache.beam.sdk.io.gcp.spanner.SpannerSchema$Builder.addColumn(SpannerSchema.java:74)

 org.apache.beam.sdk.io.gcp.spanner.ReadSpannerSchema.processElement(ReadSpannerSchema.java:63)

I included a proposed patch below that has not yet been tested.

https://github.com/apache/beam/compare/master...vilai-paypal:spanner_json_schema?expand=1

Imported from Jira BEAM-13470. Original Jira may contain additional context.
Reported by: vilai.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions