Search before asking
Paimon version
master @ a5e87eb (2.1-SNAPSHOT)
Compute Engine
Engine-agnostic — the failure is in paimon-format (avro).
Minimal reproduce step
Create a table with a MULTISET<INT> column and 'file.format' = 'avro', then insert one row.
DDL succeeds: AvroFileFormat.validateDataFields() only calls AvroSchemaConverter.convertToSchema(), which handles MULTISET and MAP in one branch and emits the array-of-record encoding for a non-string key.
The first write then fails:
java.lang.ClassCastException: MultisetType cannot be cast to ArrayType
at org.apache.paimon.format.avro.AvroSchemaVisitor.visit(AvroSchemaVisitor.java:61)
at org.apache.paimon.format.avro.FieldWriterFactory.visitUnion(FieldWriterFactory.java:107)
AvroSchemaVisitor.visit() handles only MapType and VectorType in its ARRAY branch and casts anything else to ArrayType, which is final. Reads go through the same method.
What doesn't meet your expectations?
Expected: the row is written and read back, as ORC and Parquet already do for the same schema.
Actual: every write of a MULTISET with a non-string element type throws.
Anything else?
MULTISET<STRING> is unaffected — it uses the native avro map encoding.
Are you willing to submit a PR?
Search before asking
Paimon version
master @ a5e87eb (2.1-SNAPSHOT)
Compute Engine
Engine-agnostic — the failure is in
paimon-format(avro).Minimal reproduce step
Create a table with a
MULTISET<INT>column and'file.format' = 'avro', then insert one row.DDL succeeds:
AvroFileFormat.validateDataFields()only callsAvroSchemaConverter.convertToSchema(), which handlesMULTISETandMAPin one branch and emits the array-of-record encoding for a non-string key.The first write then fails:
AvroSchemaVisitor.visit()handles onlyMapTypeandVectorTypein itsARRAYbranch and casts anything else toArrayType, which isfinal. Reads go through the same method.What doesn't meet your expectations?
Expected: the row is written and read back, as ORC and Parquet already do for the same schema.
Actual: every write of a
MULTISETwith a non-string element type throws.Anything else?
MULTISET<STRING>is unaffected — it uses the native avro map encoding.Are you willing to submit a PR?