Iceberg Flink: Schema Evolution/How to handle empty lists or unknown types

[peach12345]

Query engine

Flink

Question

Hi everyone,
We're using Apache Iceberg with Apache Flink and are currently evaluating how to handle optional or dynamically present fields in our event schemas.
We are also using the DynamicSink which was merged to main branch a few days ago.

Use case: An event contains an optional list of strings. If the list is empty, we currently remove the field entirely from both the event and the schema as a workaround—since the type isn't known at that point. Once the field is later populated in incoming events, we evolve the schema to include it.

While we're aware that using UnknownType in Java could be a theoretical alternative, we've encountered type conversion errors when trying to update the table schema this way.

👉 So our question is: Are there best practices in Iceberg (especially with Flink) for managing such evolving fields—where a field might initially be empty or missing, but could appear later in the data stream?

[peach12345]

Any recommendation on this topic?

Or solutions how you guys did it in your projects?