Upstream Pydantic 2.12.0 has a Regression that will break PyIceberg

[HonahX]

Apache Iceberg version

None

Please describe the bug 🐞

Pydantic recently 2.12.0 and unfortunately it contains a regression: pydantic/pydantic#12347 that ValidationInfo object will lose field_names in some cases.

I have a draft PR that experiments 2.12.0 and confirmed that the regression affect us: #2589

table_metadata = ValidationInfo(config={'title': 'TableMetadataV2', 'validate_by_alias': True, 'validate_by_name': True}, context=None, data=None, field_name=None)

    def check_schemas(table_metadata: TableMetadata) -> TableMetadata:
        """Check if the current-schema-id is actually present in schemas."""
>       current_schema_id = table_metadata.current_schema_id
E       AttributeError: 'pydantic_core._pydantic_core.ValidationInfo' object has no attribute 'current_schema_id'

More info:
pydantic 2.11.10

pydantic 2.12.0

While waiting for upstream fix, I think we may want to pin the pydantic version to <2.12.0

Willingness to contribute

• I can contribute a fix for this bug independently
• I would be willing to contribute a fix for this bug with guidance from the Iceberg community
• I cannot contribute a fix for this bug at this time

[kevinjqliu]

reopened the issue. lets close when there's a resolution to the underlying issue :)

[FranMorilloAWS]

Is there any workaround to make pyiceberg work? I tried reinstalling pydantic to 2.11.0 and pyiceberg to 0.9.1 and still got the same error

pydantic_core._pydantic_core.ValidationInfo' object has no attribute 'current_schema_id

[nickjoanis]

Is there any workaround to make pyiceberg work? I tried reinstalling pydantic to 2.11.0 and pyiceberg to 0.9.1 and still got the same error

pydantic_core._pydantic_core.ValidationInfo' object has no attribute 'current_schema_id

Hi @FranMorilloAWS,

After some investigation, it looks like both pyiceberg 0.9.x and pyiceberg 0.10.x don’t currently exclude pydantic 2.12.0 in their pyproject.toml file:
Reference line in pyproject.toml

However, the main branch already includes an explicit exclusion for this version, which seems to address the problem.

For now, the only stable workaround I’ve found is to:

• Downgrade pyiceberg to 0.9.1
• Pin pydantic to <2.10.0

I haven’t yet had the chance to test with pydantic 2.10.0 or 2.11.0, so they might still work — but I can only confirm that <2.10.0 has fixed the issue on my end.

This combination allows my data connectors to run without triggering the validation error.

[HonahX]

Hi @FranMorilloAWS ,

The other CI pipline I had that initially discovered this issue now works with pyiceberg==0.10.0 and pydantic==2.11.10. Could you please check if the 2.11.10 or <2.10.0 as suggested by @nickjoanis works? If it still not work, could you please share the example that you observed this error? Thanks!