If a table evolves to add a new optional container type i.e. struct, map, list with a required field inside of it, OrcSchemaUtil.buildOrcProjection fails with saying the the nested child field was required but not found in file. Iceberg supports such an evolution and so this should not fail.
e.g.
@Test
public void testEvolutionNestedField() throws IOException {
Schema baseSchema = new Schema(
required(1, "int", Types.IntegerType.get())
);
Schema evolvedSchema = new Schema(
required(1, "int", Types.IntegerType.get()),
optional(2, "struct", Types.StructType.of(
required(3, "long", Types.LongType.get())
))
);
ORCSchemaUtil.buildOrcProjection(evolvedSchema, ORCSchemaUtil.convert(baseSchema));
}
java.lang.IllegalArgumentException: Field 3 of type long is required and was not found.
at org.apache.iceberg.orc.ORCSchemaUtil.buildOrcProjection(ORCSchemaUtil.java:305)
at org.apache.iceberg.orc.ORCSchemaUtil.buildOrcProjection(ORCSchemaUtil.java:271)
at org.apache.iceberg.orc.ORCSchemaUtil.buildOrcProjection(ORCSchemaUtil.java:271)
at org.apache.iceberg.orc.ORCSchemaUtil.buildOrcProjection(ORCSchemaUtil.java:254)
at org.apache.iceberg.orc.TestBuildOrcProjection.testEvolutionNestedField(TestBuildOrcProjection.java:152)
If a table evolves to add a new optional container type i.e. struct, map, list with a required field inside of it,
OrcSchemaUtil.buildOrcProjectionfails with saying the the nested child field was required but not found in file. Iceberg supports such an evolution and so this should not fail.e.g.