Docs: Union type support spec

[funcheetah]

Summary

Apache Iceberg does not support non-optional union types (e.g. [“int”, “string”]). This PR add spec to support non-optional union types.

Representation

non-optional union type can be converted to original type for single type union or struct for complex union.

The struct representations converted from non-optional union types are consistent with non-optional union support added in Trino in trinodb/trino#3483.

Deep nested non-optional union types are supported.

Examples

Basic

[“int”, “string”] -> struct<tag int, field0 int, field1 string>

Single type

[“int”] -> int

Implementation PRs

• Support non-optional union types for Avro #4242
• Support non-optional union types for ORC #4654

TODO

• Handle single type union (e.g. [“int”]) as its original type
• Support for schema pruning within a complex union
• Support in non-Spark environments (e.g. iceberg-data, flink, hive, etc.)

[rdblue]

@funcheetah, these change cannot go in the spec. Iceberg does not allow union data.

It is fine for Iceberg implementations to support reading unions as structs for backward compatibility, but that is optional behavior and should not be required.