BUG: pandera check fails with pandas `KeyError` on certain columns starting with version 2.3.3

[melohr]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd
import pandera.pandas as pa
import pandera.typing as pt


df = pd.DataFrame({
    'Status': ['Available', 'In_Use'],
    'Completed': ['true', 'false'],
    'UnaffectedData': [0, 3],
})

class PanderaTest(pa.DataFrameModel):
    Status: pt.Series[str] = pa.Field(
        isin=[
            "Available",
            "In_Use",
            "Reserved",
            "Intended_For_Deactivation",
            "Expired",
        ],
        nullable=False,
    )
    Completed: pt.Series[str] = pa.Field(
        isin=["True", "true", "False", "false", "TRUE", "FALSE"], nullable=False
    )
    UnaffectedData: pt.Series[int] = pa.Field(nullable=False)

PanderaTest.validate(df)

Traceback

c:; cd 'c:\Users\<UID>\GitHub\<repo>'; & 'c:\Users\<UID>\AppData\Local\miniconda3\envs\pandas_fail\python.exe' 'c:\Users\<UID>\.vscode\extensions\ms-python.debugpy-2025.14.1-win32-x64\bundled\libs\debugpy\launcher' '53595' '--' 'C:\Users\<UID>\GitHub\<repo>\.vscode\pandas-reprex.py' 
Traceback (most recent call last):
  File "c:\Users\<UID>\AppData\Local\miniconda3\envs\pandas_fail\Lib\runpy.py", line 198, in _run_module_as_main
    return _run_code(code, main_globals, None,
                     "__main__", mod_spec)
  File "c:\Users\<UID>\AppData\Local\miniconda3\envs\pandas_fail\Lib\runpy.py", line 88, in _run_code
    exec(code, run_globals)
    ~~~~^^^^^^^^^^^^^^^^^^^
  File "c:\Users\<UID>\.vscode\extensions\ms-python.debugpy-2025.14.1-win32-x64\bundled\libs\debugpy\launcher/../..\debugpy\__main__.py", line 71, in <module>
    cli.main()
    ~~~~~~~~^^
  File "c:\Users\<UID>\.vscode\extensions\ms-python.debugpy-2025.14.1-win32-x64\bundled\libs\debugpy\launcher/../..\debugpy/..\debugpy\server\cli.py", line 508, in main
    run()
    ~~~^^
  File "c:\Users\<UID>\.vscode\extensions\ms-python.debugpy-2025.14.1-win32-x64\bundled\libs\debugpy\launcher/../..\debugpy/..\debugpy\server\cli.py", line 358, in run_file
    runpy.run_path(target, run_name="__main__")
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\<UID>\.vscode\extensions\ms-python.debugpy-2025.14.1-win32-x64\bundled\libs\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_runpy.py", line 310, in run_path
    return _run_module_code(code, init_globals, run_name, pkg_name=pkg_name, script_name=fname)
  File "c:\Users\<UID>\.vscode\extensions\ms-python.debugpy-2025.14.1-win32-x64\bundled\libs\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_runpy.py", line 127, in _run_module_code
    _run_code(code, mod_globals, init_globals, mod_name, mod_spec, pkg_name, script_name)
    ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\<UID>\.vscode\extensions\ms-python.debugpy-2025.14.1-win32-x64\bundled\libs\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_runpy.py", line 118, in _run_code
    exec(code, run_globals)
    ~~~~^^^^^^^^^^^^^^^^^^^
  File "C:\Users\<UID>\GitHub\<repo>\.vscode\pandera-reprex.py", line 28, in <module>
    PanderaTest.validate(df)
    ~~~~~~~~~~~~~~~~~~~~^^^^
  File "c:\Users\<UID>\AppData\Local\miniconda3\envs\pandas_fail\Lib\site-packages\pandera\api\pandas\model.py", line 192, in validate
    cls.to_schema().validate(
    ~~~~~~~~~~~~~~~~~~~~~~~~^
        check_obj, head, tail, sample, random_state, lazy, inplace
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ),
    ^
  File "c:\Users\<UID>\AppData\Local\miniconda3\envs\pandas_fail\Lib\site-packages\pandera\api\pandas\container.py", line 117, in validate
    return self._validate(
           ~~~~~~~~~~~~~~^
        check_obj=check_obj,
        ^^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
        inplace=inplace,
        ^^^^^^^^^^^^^^^^
    )
    ^
  File "c:\Users\<UID>\AppData\Local\miniconda3\envs\pandas_fail\Lib\site-packages\pandera\api\pandas\container.py", line 138, in _validate
    return self.get_backend(check_obj).validate(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        check_obj,
        ^^^^^^^^^^
    ...<6 lines>...
        inplace=inplace,
        ^^^^^^^^^^^^^^^^
    )
    ^
  File "c:\Users\<UID>\AppData\Local\miniconda3\envs\pandas_fail\Lib\site-packages\pandera\backends\pandas\container.py", line 104, in validate
    error_handler = self.run_checks_and_handle_errors(
        error_handler,
    ...<8 lines>...
        random_state,
    )
  File "c:\Users\<UID>\AppData\Local\miniconda3\envs\pandas_fail\Lib\site-packages\pandera\backends\pandas\container.py", line 191, in run_checks_and_handle_errors
    error_handler.collect_error(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        validation_type(result.reason_code),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
        result.original_exc,
        ^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "c:\Users\<UID>\AppData\Local\miniconda3\envs\pandas_fail\Lib\site-packages\pandera\api\base\error_handler.py", line 54, in collect_error
    raise schema_error from original_exc
  File "c:\Users\<UID>\AppData\Local\miniconda3\envs\pandas_fail\Lib\site-packages\pandera\backends\pandas\container.py", line 226, in run_schema_component_checks
    result = schema_component.validate(
        check_obj, lazy=lazy, inplace=True
    )
  File "c:\Users\<UID>\AppData\Local\miniconda3\envs\pandas_fail\Lib\site-packages\pandera\api\dataframe\components.py", line 148, in validate
    return self.get_backend(check_obj).validate(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        check_obj,
        ^^^^^^^^^^
    ...<6 lines>...
        inplace=inplace,
        ^^^^^^^^^^^^^^^^
    )
    ^
  File "c:\Users\<UID>\AppData\Local\miniconda3\envs\pandas_fail\Lib\site-packages\pandera\backends\pandas\components.py", line 141, in validate
    validated_column = validate_column(
        check_obj,
        column_name,
        return_check_obj=True,
    )
  File "c:\Users\<UID>\AppData\Local\miniconda3\envs\pandas_fail\Lib\site-packages\pandera\backends\pandas\components.py", line 101, in validate_column
    error_handler.collect_error(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        validation_type(err.reason_code), err.reason_code, err
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "c:\Users\<UID>\AppData\Local\miniconda3\envs\pandas_fail\Lib\site-packages\pandera\api\base\error_handler.py", line 54, in collect_error
    raise schema_error from original_exc
  File "c:\Users\<UID>\AppData\Local\miniconda3\envs\pandas_fail\Lib\site-packages\pandera\backends\pandas\components.py", line 77, in validate_column
    validated_check_obj = super(ColumnBackend, self).validate(
        check_obj,
    ...<6 lines>...
        inplace=inplace,
    )
  File "c:\Users\<UID>\AppData\Local\miniconda3\envs\pandas_fail\Lib\site-packages\pandera\backends\pandas\array.py", line 74, in validate
    error_handler = self.run_checks_and_handle_errors(
        error_handler,
    ...<5 lines>...
        random_state=random_state,
    )
  File "c:\Users\<UID>\AppData\Local\miniconda3\envs\pandas_fail\Lib\site-packages\pandera\backends\pandas\array.py", line 138, in run_checks_and_handle_errors
    error_handler.collect_error(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        validation_type(result.reason_code),
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<2 lines>...
        original_exc=result.original_exc,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "c:\Users\<UID>\AppData\Local\miniconda3\envs\pandas_fail\Lib\site-packages\pandera\api\base\error_handler.py", line 54, in collect_error
    raise schema_error from original_exc
pandera.errors.SchemaError: Error while executing check function: KeyError("<class 'pandas.core.series.Series'>")
Traceback (most recent call last):
  File "c:\Users\<UID>\AppData\Local\miniconda3\envs\pandas_fail\Lib\site-packages\pandera\backends\pandas\components.py", line 240, in run_checks
    self.run_check(
    ~~~~~~~~~~~~~~^
        check_obj, schema, check, check_index, *check_args
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "c:\Users\<UID>\AppData\Local\miniconda3\envs\pandas_fail\Lib\site-packages\pandera\backends\pandas\base.py", line 115, in run_check
    check_result: CheckResult = check(check_obj, *args)
                                ~~~~~^^^^^^^^^^^^^^^^^^
  File "c:\Users\<UID>\AppData\Local\miniconda3\envs\pandas_fail\Lib\site-packages\pandera\api\checks.py", line 230, in __call__
    return backend(check_obj, column)
  File "c:\Users\<UID>\AppData\Local\miniconda3\envs\pandas_fail\Lib\site-packages\pandera\backends\pandas\checks.py", line 349, in __call__
    check_output = self.apply(check_obj)
  File "c:\Users\<UID>\AppData\Local\miniconda3\envs\pandas_fail\Lib\site-packages\pandera\backends\pandas\checks.py", line 148, in apply
    return apply_fn(check_obj)
  File "c:\Users\<UID>\AppData\Local\miniconda3\envs\pandas_fail\Lib\site-packages\pandera\backends\pandas\checks.py", line 156, in apply_field
    return self.check_fn(check_obj)
           ~~~~~~~~~~~~~^^^^^^^^^^^
  File "c:\Users\<UID>\AppData\Local\miniconda3\envs\pandas_fail\Lib\site-packages\pandera\api\function_dispatch.py", line 24, in __call__
    fn = self._function_registry[input_data_type]
         ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
KeyError: <class 'pandas.core.series.Series'>

Issue Description

I use pandera to check my pandas dataframe. Up to pandas version 2.3.2, checks passed as intended. Starting with version 2.3.3, I start encountering KeyError: <class 'pandas.core.series.Series'>.

I work with anaconda. The two venv reprex are:

conda create -y --name pandas_pass python==3.13.9 pandas==2.3.2 pandera

conda create -y --name pandas_fail python==3.13.9 pandas==2.3.3 pandera

The pandera version installed in the venv is the current 0.26.1, but I also tested 0.26.0, 0.25.0, and 0.24.0 and all work just fine with pandas <= 2.3.2.

Expected Behavior

The pandera check is expected to pass (not throwing an exception) with correct data.

Installed Versions

pytest : None
python-calamine : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlsxwriter : None
zstandard : None
tzdata : 2025.2
qtpy : None
pyqt5 : None

[jbrockmendel]

For those of us who don't use pandera or know anything about it, can you elaborate on what the pandas bug is?

[melohr]

Pandera is a framework for validating dataframes, see https://github.com/unionai-oss/pandera.
I cannot pinpoint the actual root cause of the error despite investigating the call stack to my best knowledge and experience. Therefore, I am not able to definitely link it to pandas but I can explain my reasoning to file the bug here:

• I use pandas and pandera together for approx. two years without issues (apart from the usual documented changes the user has to follow)
• I did check release notes of both packages back to 2.3.0 for pandas and 0.24.0 for pandera for related topics, found some regression and bug fixes on pandas side I was unsure if somehow related to this issue
• As mentioned, I did test different combinations of versions and the only difference I found was the latest pandas version being increased
• As pandas uses semantic versioning with the only change on patch level (.2 -> .3) I don't expect APIs to break though I cannot tell the way pandera accesses pandas dataframes for its tasks; the KeyError clearly pointing to pandas.core.series.Series, however, made me finally file it here

Since pandera is somehow built on top of pandas, I can understand the questioning of a pandas bug. Mainly the last point made me think if the root cause of this issue was a potential regression in pandas, especially as the release of pandera 0.24.0 dates back to May 15 and 0.26.1 is from Aug 26 which even overlaps with the pandas 2.2.3 release from Sept 2024.
In the end, my only motivation is to get the issue solved and if helpful, I can file the bug also on pandera's side to cross-reference.

[glatterf42]

I'm seeing a very similar issue, at least this is my traceback:

Traceback

_____ ERROR at setup of TestCoreIamcReadOnly.test_mp_tabulate_big[sqlite] ______

request = <SubRequest 'platform_med' for <Function test_mp_tabulate_big_async[sqlite]>>

    def platform_with_td(
        request: pytest.FixtureRequest,
    ) -> Generator[Platform, Any, None]:
        type = request.param
        postgres_dsn = request.config.option.postgres_dsn
        bctx = get_backend_context(type, postgres_dsn)
    
        with bctx as backend:
            platform = Platform(_backend=backend)
>           td.load_dataset(platform)

tests/conftest.py:175: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
tests/fixtures/__init__.py:353: in load_dataset
    cls.load_datapoints(platform)
tests/fixtures/__init__.py:346: in load_datapoints
    cls.load_dp_df(platform, cls.datapoints)
tests/fixtures/__init__.py:342: in load_dp_df
    cls.load_run_datapoints(platform, (model, scenario, version), dps)
tests/fixtures/__init__.py:319: in load_run_datapoints
    run.iamc.add(annual, type=ixmp4.DataPoint.Type.ANNUAL)
ixmp4/core/iamc/data.py:63: in add
    df = AddDataPointFrameSchema.validate(df)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.venv/lib/python3.14/site-packages/pandera/api/pandas/model.py:191: in validate
    cls.to_schema().validate(
.venv/lib/python3.14/site-packages/pandera/api/pandas/container.py:118: in validate
    return self._validate(
.venv/lib/python3.14/site-packages/pandera/api/pandas/container.py:139: in _validate
    return self.get_backend(check_obj).validate(
.venv/lib/python3.14/site-packages/pandera/backends/pandas/container.py:104: in validate
    error_handler = self.run_checks_and_handle_errors(
.venv/lib/python3.14/site-packages/pandera/backends/pandas/container.py:192: in run_checks_and_handle_errors
    error_handler.collect_error(
.venv/lib/python3.14/site-packages/pandera/api/base/error_handler.py:54: in collect_error
    raise schema_error from original_exc
.venv/lib/python3.14/site-packages/pandera/backends/pandas/container.py:227: in run_schema_component_checks
    result = schema_component.validate(
.venv/lib/python3.14/site-packages/pandera/api/dataframe/components.py:149: in validate
    return self.get_backend(check_obj).validate(
.venv/lib/python3.14/site-packages/pandera/backends/pandas/components.py:142: in validate
    validated_column = validate_column(
.venv/lib/python3.14/site-packages/pandera/backends/pandas/components.py:102: in validate_column
    error_handler.collect_error(
.venv/lib/python3.14/site-packages/pandera/api/base/error_handler.py:54: in collect_error
    raise schema_error from original_exc
.venv/lib/python3.14/site-packages/pandera/backends/pandas/components.py:78: in validate_column
    validated_check_obj = super(ColumnBackend, self).validate(
.venv/lib/python3.14/site-packages/pandera/backends/pandas/array.py:74: in validate
    error_handler = self.run_checks_and_handle_errors(
.venv/lib/python3.14/site-packages/pandera/backends/pandas/array.py:138: in run_checks_and_handle_errors
    error_handler.collect_error(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <pandera.api.base.error_handler.ErrorHandler object at 0x7f5036e12c80>
error_type = <ValidationScope.DATA: 'data'>
reason_code = <SchemaErrorReason.CHECK_ERROR: 'check_error'>
schema_error = SchemaError('Error while executing check function: KeyError("<class \'pandas.core.series.Series\'>")\nTraceback (most ...nput_data_type]\n         ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^\nKeyError: <class \'pandas.core.series.Series\'>\n')
original_exc = KeyError(<class 'pandas.core.series.Series'>)

    def collect_error(
        self,
        error_type: ErrorCategory,
        reason_code: Optional[SchemaErrorReason],
        schema_error: SchemaError,
        original_exc: Union[BaseException, None] = None,
    ):
        """Collect schema error, raising exception if lazy is False.
    
        :param error_type: type of error
        :param reason_code: string representing reason for error
        :param schema_error: ``SchemaError`` object.
        """
        if not self._lazy:
>           raise schema_error from original_exc
E           pandera.errors.SchemaError: Error while executing check function: KeyError("<class 'pandas.core.series.Series'>")
E           Traceback (most recent call last):
E             File "/home/runner/work/ixmp4/ixmp4/.venv/lib/python3.14/site-packages/pandera/backends/pandas/components.py", line 242, in run_checks
E               self.run_check(
E               ~~~~~~~~~~~~~~^
E                   check_obj, schema, check, check_index, *check_args
E                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E               )
E               ^
E             File "/home/runner/work/ixmp4/ixmp4/.venv/lib/python3.14/site-packages/pandera/backends/pandas/base.py", line 115, in run_check
E               check_result: CheckResult = check(check_obj, *args)
E                                           ~~~~~^^^^^^^^^^^^^^^^^^
E             File "/home/runner/work/ixmp4/ixmp4/.venv/lib/python3.14/site-packages/pandera/api/checks.py", line 234, in __call__
E               return backend(check_obj, column)
E             File "/home/runner/work/ixmp4/ixmp4/.venv/lib/python3.14/site-packages/pandera/backends/pandas/checks.py", line 349, in __call__
E               check_output = self.apply(check_obj)
E             File "/home/runner/work/ixmp4/ixmp4/.venv/lib/python3.14/site-packages/pandera/backends/pandas/checks.py", line 148, in apply
E               return apply_fn(check_obj)
E             File "/home/runner/work/ixmp4/ixmp4/.venv/lib/python3.14/site-packages/pandera/backends/pandas/checks.py", line 156, in apply_field
E               return self.check_fn(check_obj)
E                      ~~~~~~~~~~~~~^^^^^^^^^^^
E             File "/home/runner/work/ixmp4/ixmp4/.venv/lib/python3.14/site-packages/pandera/api/function_dispatch.py", line 24, in __call__
E               fn = self._function_registry[input_data_type]
E                    ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
E           KeyError: <class 'pandas.core.series.Series'>

.venv/lib/python3.14/site-packages/pandera/api/base/error_handler.py:54: SchemaError

Though for me, this occurs on iiasa/ixmp4#209, which is an attempt to check for compatibility with Python 3.14. So I'm using pandas 2.3.1 and pandera 0.25.0, which works fine with Python 3.13, but when I shift to Python 3.14, it doesn't anymore.
I checked that Pandera doesn't officially support Python 3.14 yet, so maybe that's all it would take. Unfortunately, I see no issue or PR there to check what their status is regarding the support.

[melohr]

The anaconda pandas_pass venv on my local machine reports a python version of 3.13.9, so no issue related to officially unsupported 3.14. I adjusted the issue reprex nevertheless.

[rhshadrach]

Has this issue been reported to pandera? Please show the full output of pd.show_versions() in the OP.

Edit: I see it's being mentioned in the PR linked above. Closing for now, if this is suspected to be due to an unintentional change in pandas comment here and we can reopen.