Skip to content

[Bug] Avro throws ClassCastException when writing MULTISET with a non-string element type #9064

Description

@thswlsqls

Search before asking

  • I searched in the issues and found nothing similar.

Paimon version
master @ a5e87eb (2.1-SNAPSHOT)

Compute Engine
Engine-agnostic — the failure is in paimon-format (avro).

Minimal reproduce step
Create a table with a MULTISET<INT> column and 'file.format' = 'avro', then insert one row.

DDL succeeds: AvroFileFormat.validateDataFields() only calls AvroSchemaConverter.convertToSchema(), which handles MULTISET and MAP in one branch and emits the array-of-record encoding for a non-string key.

The first write then fails:

java.lang.ClassCastException: MultisetType cannot be cast to ArrayType
  at org.apache.paimon.format.avro.AvroSchemaVisitor.visit(AvroSchemaVisitor.java:61)
  at org.apache.paimon.format.avro.FieldWriterFactory.visitUnion(FieldWriterFactory.java:107)

AvroSchemaVisitor.visit() handles only MapType and VectorType in its ARRAY branch and casts anything else to ArrayType, which is final. Reads go through the same method.

What doesn't meet your expectations?
Expected: the row is written and read back, as ORC and Parquet already do for the same schema.
Actual: every write of a MULTISET with a non-string element type throws.

Anything else?
MULTISET<STRING> is unaffected — it uses the native avro map encoding.

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions