Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(2.4.0/2.4.1) LookupError during schema gen (simple repro) #7645

Closed
1 task done
zakstucke opened this issue Sep 26, 2023 · 3 comments · Fixed by #7653
Closed
1 task done

(2.4.0/2.4.1) LookupError during schema gen (simple repro) #7645

zakstucke opened this issue Sep 26, 2023 · 3 comments · Fixed by #7653
Assignees
Labels
bug V2 Bug related to Pydantic V2 unconfirmed Bug not yet confirmed as valid/applicable

Comments

@zakstucke
Copy link
Contributor

zakstucke commented Sep 26, 2023

Initial Checks

  • I confirm that I'm using Pydantic V2

Description

Was hoping the 2.4.1 patch would fix, but there's a new schema error on these two new versions. (same code works 2.3.0)

Edit: managed to make a simple reproduction.

Traceback (most recent call last):
  File "repo/ptest.py", line 33, in <module>
    Foo.model_json_schema()
  File "repo/venv/lib/python3.11/site-packages/pydantic/main.py", line 385, in model_json_schema
    return model_json_schema(
           ^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/json_schema.py", line 2153, in model_json_schema
    return schema_generator_instance.generate(cls.__pydantic_core_schema__, mode=mode)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/json_schema.py", line 410, in generate
    json_schema: JsonSchemaValue = self.generate_inner(schema)
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/json_schema.py", line 549, in generate_inner
    json_schema = current_handler(schema)
                  ^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/_internal/_schema_generation_shared.py", line 36, in __call__
    return self.handler(__core_schema)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/json_schema.py", line 506, in handler_func
    json_schema = generate_for_schema_type(schema_or_field)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/json_schema.py", line 1730, in definitions_schema
    self.generate_inner(definition)
  File "repo/venv/lib/python3.11/site-packages/pydantic/json_schema.py", line 549, in generate_inner
    json_schema = current_handler(schema)
                  ^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/_internal/_schema_generation_shared.py", line 36, in __call__
    return self.handler(__core_schema)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/json_schema.py", line 523, in new_handler_func
    json_schema = js_modify_function(schema_or_field, current_handler)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/main.py", line 601, in __get_pydantic_json_schema__
    return __handler(__core_schema)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/_internal/_schema_generation_shared.py", line 36, in __call__
    return self.handler(__core_schema)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/json_schema.py", line 523, in new_handler_func
    json_schema = js_modify_function(schema_or_field, current_handler)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py", line 213, in modify_model_json_schema
    json_schema = handler(schema_or_field)
                  ^^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/_internal/_schema_generation_shared.py", line 36, in __call__
    return self.handler(__core_schema)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/json_schema.py", line 506, in handler_func
    json_schema = generate_for_schema_type(schema_or_field)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/json_schema.py", line 1318, in model_schema
    json_schema = self.generate_inner(schema['schema'])
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/json_schema.py", line 549, in generate_inner
    json_schema = current_handler(schema)
                  ^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/_internal/_schema_generation_shared.py", line 36, in __call__
    return self.handler(__core_schema)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/json_schema.py", line 506, in handler_func
    json_schema = generate_for_schema_type(schema_or_field)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/json_schema.py", line 1410, in model_fields_schema
    json_schema = self._named_required_fields_schema(named_required_fields)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/json_schema.py", line 1221, in _named_required_fields_schema
    field_json_schema = self.generate_inner(field).copy()
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/json_schema.py", line 549, in generate_inner
    json_schema = current_handler(schema)
                  ^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/_internal/_schema_generation_shared.py", line 36, in __call__
    return self.handler(__core_schema)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/json_schema.py", line 541, in new_handler_func
    json_schema = js_modify_function(schema_or_field, current_handler)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py", line 955, in json_schema_update_func
    json_schema = {**handler(schema), **json_schema_updates}
                     ^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/_internal/_schema_generation_shared.py", line 36, in __call__
    return self.handler(__core_schema)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/json_schema.py", line 506, in handler_func
    json_schema = generate_for_schema_type(schema_or_field)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/json_schema.py", line 1289, in model_field_schema
    return self.generate_inner(schema['schema'])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/json_schema.py", line 549, in generate_inner
    json_schema = current_handler(schema)
                  ^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/_internal/_schema_generation_shared.py", line 36, in __call__
    return self.handler(__core_schema)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/json_schema.py", line 506, in handler_func
    json_schema = generate_for_schema_type(schema_or_field)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/json_schema.py", line 1016, in nullable_schema
    inner_json_schema = self.generate_inner(schema['schema'])
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/json_schema.py", line 549, in generate_inner
    json_schema = current_handler(schema)
                  ^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/_internal/_schema_generation_shared.py", line 36, in __call__
    return self.handler(__core_schema)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/json_schema.py", line 526, in new_handler_func
    original_schema = current_handler.resolve_ref_schema(json_schema)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "repo/venv/lib/python3.11/site-packages/pydantic/_internal/_schema_generation_shared.py", line 57, in resolve_ref_schema
    raise LookupError(
LookupError: Could not find a ref for #/$defs/__main____Bar-Input__1. Maybe you tried to call resolve_ref_schema from within a recursive model?

Example Code

from pydantic import BaseModel


class Foo(BaseModel):
    maybe_bar: "Bar | None"


class Bar(BaseModel):
    foo: Foo


Foo.model_json_schema()

Python, Pydantic & OS Version

pydantic version: 2.4.1
        pydantic-core version: 2.10.1
          pydantic-core build: profile=release pgo=true
                 install path: /repo/venv/lib/python3.11/site-packages/pydantic
               python version: 3.11.5 (main, Aug 25 2023, 13:19:50) [GCC 11.4.0]
                     platform: Linux-6.2.0-33-generic-x86_64-with-glibc2.35
             related packages: typing_extensions-4.8.0 fastapi-0.103.1 mypy-1.5.1 email-validator-2.0.0
@zakstucke zakstucke added bug V2 Bug related to Pydantic V2 unconfirmed Bug not yet confirmed as valid/applicable labels Sep 26, 2023
@zakstucke zakstucke changed the title (2.4.0/2.4.1) LookupError (2.4.0/2.4.1) LookupError during schema gen (simple repro) Sep 26, 2023
@vincentsarago
Copy link

vincentsarago commented Sep 26, 2023

I've got a slightly related issue with recursive type in geojson-pydantic. Happy to open a new issue if it's not related at all

from pydantic import BaseModel
from typing_extensions import Annotated
from typing import List, Literal, Union, Tuple
from pydantic import Field


class Point(BaseModel):
    type: Literal["Point"]
    coordinates: Union[Tuple[float, float], Tuple[float, float, float]]


class GeometryCollection(BaseModel):
    type: Literal["GeometryCollection"]
    geometries: List["Geometry"]


Geometry = Annotated[
    Union[
        Point,
        GeometryCollection,
    ],
    Field(discriminator="type"),
]

GeometryCollection.model_rebuild()

---------------------------------------------------------------------------
SchemaError                               Traceback (most recent call last)
<ipython-input-2-2c593a18e1c0> in <cell line: 25>()
     23 ]
     24 
---> 25 GeometryCollection.model_rebuild()
     26 

~/Dev/venv/py39/lib/python3.9/site-packages/pydantic/main.py in model_rebuild(cls, force, raise_errors, _parent_namespace_depth, _types_namespace)
    468             # manually override defer_build so complete_model_class doesn't skip building the model again
    469             config = {**cls.model_config, 'defer_build': False}
--> 470             return _model_construction.complete_model_class(
    471                 cls,
    472                 cls.__name__,

~/Dev/venv/py39/lib/python3.9/site-packages/pydantic/_internal/_model_construction.py in complete_model_class(cls, cls_name, config_wrapper, raise_errors, types_namespace)
    496 
    497     # debug(schema)
--> 498     cls.__pydantic_core_schema__ = schema = validate_core_schema(schema)
    499     cls.__pydantic_validator__ = create_schema_validator(schema, core_config, config_wrapper.plugin_settings)
    500     cls.__pydantic_serializer__ = SchemaSerializer(schema, core_config)

~/Dev/venv/py39/lib/python3.9/site-packages/pydantic/_internal/_core_utils.py in validate_core_schema(schema)
    629     if 'PYDANTIC_SKIP_VALIDATING_CORE_SCHEMAS' in os.environ:
    630         return schema
--> 631     return _validate_core_schema(schema)

SchemaError: Invalid Schema:
definitions.definitions.1.model.schema.model-fields.fields.geometries.schema.list.items_schema.tagged-union.choices.GeometryCollection
  Recursion error - cyclic reference detected [type=recursion_loop, input_value={'type': 'model', 'cls': ...yCollection:4353815568'}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.4/v/recursion_loop

@tlambert03
Copy link
Contributor

tlambert03 commented Sep 26, 2023

seeing it here too, and a local git blame puts the change at #7624

and a slight variant on the original reproducible example gives a RecursionError rather than a LookupError

from typing import Optional

from pydantic import BaseModel


class Foo(BaseModel):
    bar: "Bar"


class Bar(BaseModel):
    foo: Optional[Foo]


Foo.model_rebuild()
print(Bar.model_json_schema())
  File "/Users/talley/mambaforge/envs/useq-schema/lib/python3.11/site-packages/pydantic/json_schema.py", line 215, in <dictcomp>
    self.remap_defs_ref(DefsRef(key)): self.remap_json_schema(value)
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/talley/mambaforge/envs/useq-schema/lib/python3.11/site-packages/pydantic/json_schema.py", line 219, in remap_json_schema
    schema[key] = self.remap_json_schema(value)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/talley/mambaforge/envs/useq-schema/lib/python3.11/site-packages/pydantic/json_schema.py", line 219, in remap_json_schema
    schema[key] = self.remap_json_schema(value)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/talley/mambaforge/envs/useq-schema/lib/python3.11/site-packages/pydantic/json_schema.py", line 219, in remap_json_schema
    schema[key] = self.remap_json_schema(value)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/talley/mambaforge/envs/useq-schema/lib/python3.11/site-packages/pydantic/json_schema.py", line 208, in remap_json_schema
    return [self.remap_json_schema(item) for item in schema]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/talley/mambaforge/envs/useq-schema/lib/python3.11/site-packages/pydantic/json_schema.py", line 208, in <listcomp>
    return [self.remap_json_schema(item) for item in schema]
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/talley/mambaforge/envs/useq-schema/lib/python3.11/site-packages/pydantic/json_schema.py", line 219, in remap_json_schema
    schema[key] = self.remap_json_schema(value)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/talley/mambaforge/envs/useq-schema/lib/python3.11/site-packages/pydantic/json_schema.py", line 206, in remap_json_schema
    return self.remap_json_ref(JsonRef(schema))
                               ^^^^^^^^^^^^^^^
RecursionError: maximum recursion depth exceeded while calling a Python object

@dmontagu
Copy link
Contributor

I just want to thank everyone for being so quick to report these schema building issues introduced in 2.4 with clean reproductions of the issue. Every report is helping us improve our test suite, and we are working urgently to address these problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug V2 Bug related to Pydantic V2 unconfirmed Bug not yet confirmed as valid/applicable
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants