Type annotations on dataclass base class cannot be resolved if future annotations are used

[matsjoyce-refeyn]

Initial Checks

• I confirm that I'm using Pydantic V2

Description

When the base class of a dataclass has an annotation type that is not imported by the module the subclass is in, and from __future__ import annotations is in use (or in my case, Cython is in use), an error occurs (full traceback from the example code below).

The root cause is that when pydantic builds up the type namespace, it does not consider base classes (https://github.com/pydantic/pydantic/blob/main/pydantic/_internal/_generate_schema.py#L1474). This is fine for models, because each base class builds a schema immediately, but does not work for plain stdlib dataclasses. One solution would be to iterate though all of the base classes of the dataclass that are also dataclasses, and push their namespaces as well. For example:

cm_stack = contextlib.ExitStack()
with self._config_wrapper_stack.push(config), cm_stack:
    for cls in reversed(dataclass.__mro__):
        cm_stack.enter_context(self._types_namespace_stack.push(cls))

I can make this into a PR if it looks reasonable.

Full traceback for the example

Traceback (most recent call last):
  File "C:\Users\matthew.joyce\Repos\lw8\venv\venv_develop\lib\site-packages\pydantic\type_adapter.py", line 207, in __init__
    core_schema = _getattr_no_parents(type, '__pydantic_core_schema__')
  File "C:\Users\matthew.joyce\Repos\lw8\venv\venv_develop\lib\site-packages\pydantic\type_adapter.py", line 98, in _getattr_no_parents
    raise AttributeError(attribute)
AttributeError: __pydantic_core_schema__

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\matthew.joyce\Repos\lw8\venv\venv_develop\lib\site-packages\pydantic\_internal\_generate_schema.py", line 679, in _resolve_forward_ref
    obj = _typing_extra.eval_type_backport(obj, globalns=self._types_namespace)
  File "C:\Users\matthew.joyce\Repos\lw8\venv\venv_develop\lib\site-packages\pydantic\_internal\_typing_extra.py", line 240, in eval_type_backport
    return typing._eval_type(  # type: ignore
  File "C:\Program Files\Python310\lib\typing.py", line 327, in _eval_type
    return t._evaluate(globalns, localns, recursive_guard)
  File "C:\Program Files\Python310\lib\typing.py", line 694, in _evaluate
    eval(self.__forward_code__, globalns, localns),
  File "<string>", line 1, in <module>
NameError: name 'IntAlias' is not defined

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\matthew.joyce\Repos\lw8\b.py", line 13, in <module>
    print(pydantic.TypeAdapter(Sub).validate_python({"x": 1}))
  File "C:\Users\matthew.joyce\Repos\lw8\venv\venv_develop\lib\site-packages\pydantic\type_adapter.py", line 209, in __init__
    core_schema = _get_schema(type, config_wrapper, parent_depth=_parent_depth + 1)
  File "C:\Users\matthew.joyce\Repos\lw8\venv\venv_develop\lib\site-packages\pydantic\type_adapter.py", line 81, in _get_schema
    schema = gen.generate_schema(type_)
  File "C:\Users\matthew.joyce\Repos\lw8\venv\venv_develop\lib\site-packages\pydantic\_internal\_generate_schema.py", line 501, in generate_schema
    schema = self._generate_schema(obj)
  File "C:\Users\matthew.joyce\Repos\lw8\venv\venv_develop\lib\site-packages\pydantic\_internal\_generate_schema.py", line 739, in _generate_schema
    schema = self._post_process_generated_schema(self._generate_schema_inner(obj))
  File "C:\Users\matthew.joyce\Repos\lw8\venv\venv_develop\lib\site-packages\pydantic\_internal\_generate_schema.py", line 766, in _generate_schema_inner
    return self.match_type(obj)
  File "C:\Users\matthew.joyce\Repos\lw8\venv\venv_develop\lib\site-packages\pydantic\_internal\_generate_schema.py", line 840, in match_type
    return self._dataclass_schema(obj, None)
  File "C:\Users\matthew.joyce\Repos\lw8\venv\venv_develop\lib\site-packages\pydantic\_internal\_generate_schema.py", line 1489, in _dataclass_schema
    args = sorted(
  File "C:\Users\matthew.joyce\Repos\lw8\venv\venv_develop\lib\site-packages\pydantic\_internal\_generate_schema.py", line 1490, in <genexpr>
    (self._generate_dc_field_schema(k, v, decorators) for k, v in fields.items()),
  File "C:\Users\matthew.joyce\Repos\lw8\venv\venv_develop\lib\site-packages\pydantic\_internal\_generate_schema.py", line 942, in _generate_dc_field_schema
    common_field = self._common_field_schema(name, field_info, decorators)
  File "C:\Users\matthew.joyce\Repos\lw8\venv\venv_develop\lib\site-packages\pydantic\_internal\_generate_schema.py", line 1047, in _common_field_schema
    schema = self._apply_annotations(
  File "C:\Users\matthew.joyce\Repos\lw8\venv\venv_develop\lib\site-packages\pydantic\_internal\_generate_schema.py", line 1749, in _apply_annotations
    schema = get_inner_schema(source_type)
  File "C:\Users\matthew.joyce\Repos\lw8\venv\venv_develop\lib\site-packages\pydantic\_internal\_schema_generation_shared.py", line 82, in __call__
    schema = self._handler(__source_type)
  File "C:\Users\matthew.joyce\Repos\lw8\venv\venv_develop\lib\site-packages\pydantic\_internal\_generate_schema.py", line 1730, in inner_handler
    schema = self._generate_schema(obj)
  File "C:\Users\matthew.joyce\Repos\lw8\venv\venv_develop\lib\site-packages\pydantic\_internal\_generate_schema.py", line 739, in _generate_schema
    schema = self._post_process_generated_schema(self._generate_schema_inner(obj))
  File "C:\Users\matthew.joyce\Repos\lw8\venv\venv_develop\lib\site-packages\pydantic\_internal\_generate_schema.py", line 756, in _generate_schema_inner
    return self.generate_schema(self._resolve_forward_ref(obj))
  File "C:\Users\matthew.joyce\Repos\lw8\venv\venv_develop\lib\site-packages\pydantic\_internal\_generate_schema.py", line 681, in _resolve_forward_ref
    raise PydanticUndefinedAnnotation.from_name_error(e) from e
pydantic.errors.PydanticUndefinedAnnotation: name 'IntAlias' is not defined

For further information visit https://errors.pydantic.dev/2.6/u/undefined-annotation

Example Code

### In a.py
from __future__ import annotations

import dataclasses
from typing import NewType

IntAlias = NewType("IntAlias", int)


@dataclasses.dataclass
class Base:
    x: IntAlias

### In b.py
import dataclasses

import pydantic

import a


@dataclasses.dataclass
class Sub(a.Base): ...


if __name__ == "__main__":
    print(pydantic.TypeAdapter(Sub).validate_python({"x": 1}))

Python, Pydantic & OS Version

pydantic version: 2.6.0
        pydantic-core version: 2.16.1
          pydantic-core build: profile=release pgo=true
                 install path: C:\Users\matthew.joyce\Repos\lw8\venv\venv_develop\Lib\site-packages\pydantic
               python version: 3.10.10 (tags/v3.10.10:aad5f6a, Feb  7 2023, 17:20:36) [MSC v.1929 64 bit (AMD64)]
                     platform: Windows-10-10.0.19045-SP0
             related packages: mypy-1.2.0 typing_extensions-4.9.0
                       commit: unknown

[sydney-runkle]

@alexmojaki,

Could you please take a look at this issue when you have a chance? Thanks!

[alexmojaki]

The root cause is that when pydantic builds up the type namespace, it does not consider base classes (https://github.com/pydantic/pydantic/blob/main/pydantic/_internal/_generate_schema.py#L1474). This is fine for models, because each base class builds a schema immediately, but does not work for plain stdlib dataclasses. One solution would be to iterate though all of the base classes of the dataclass that are also dataclasses, and push their namespaces as well. For example:

cm_stack = contextlib.ExitStack()
with self._config_wrapper_stack.push(config), cm_stack:
    for cls in reversed(dataclass.__mro__):
        cm_stack.enter_context(self._types_namespace_stack.push(cls))

I can make this into a PR if it looks reasonable.

Yes please! I think your diagnosis and proposal sound sensible.