Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

model_json_schema fails with empty enum #7896

Closed
1 task done
jurasofish opened this issue Oct 22, 2023 · 11 comments · Fixed by #7927
Closed
1 task done

model_json_schema fails with empty enum #7896

jurasofish opened this issue Oct 22, 2023 · 11 comments · Fixed by #7927
Assignees
Labels
bug V2 Bug related to Pydantic V2

Comments

@jurasofish
Copy link

jurasofish commented Oct 22, 2023

Initial Checks

  • I confirm that I'm using Pydantic V2

Description

When a field is annotated as an enum and that enum has no members then model_json_schema fails.
MRE and traceback below.
In pydantic v1 this produces {"title": "MyModel", "type": "object", "properties": {"e": {"title": "E", "enum": []}}, "required": ["e"]} (using schema_json).

from pydantic import BaseModel
from enum import Enum


class MyEnum(Enum):
    pass


class MyModel(BaseModel):
    e: MyEnum


print(MyModel.model_json_schema())
Traceback (most recent call last):
  File "scratch_308.py", line 13, in <module>
    print(MyModel.model_json_schema())
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/pydantic/main.py", line 385, in model_json_schema
    return model_json_schema(
           ^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/pydantic/json_schema.py", line 2153, in model_json_schema
    return schema_generator_instance.generate(cls.__pydantic_core_schema__, mode=mode)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/pydantic/json_schema.py", line 410, in generate
    json_schema: JsonSchemaValue = self.generate_inner(schema)
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/pydantic/json_schema.py", line 549, in generate_inner
    json_schema = current_handler(schema)
                  ^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/pydantic/_internal/_schema_generation_shared.py", line 36, in __call__
    return self.handler(__core_schema)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/pydantic/json_schema.py", line 523, in new_handler_func
    json_schema = js_modify_function(schema_or_field, current_handler)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/pydantic/main.py", line 601, in __get_pydantic_json_schema__
    return __handler(__core_schema)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/pydantic/_internal/_schema_generation_shared.py", line 36, in __call__
    return self.handler(__core_schema)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/pydantic/json_schema.py", line 523, in new_handler_func
    json_schema = js_modify_function(schema_or_field, current_handler)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py", line 212, in modify_model_json_schema
    json_schema = handler(schema_or_field)
                  ^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/pydantic/_internal/_schema_generation_shared.py", line 36, in __call__
    return self.handler(__core_schema)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/pydantic/json_schema.py", line 506, in handler_func
    json_schema = generate_for_schema_type(schema_or_field)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/pydantic/json_schema.py", line 1318, in model_schema
    json_schema = self.generate_inner(schema['schema'])
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/pydantic/json_schema.py", line 549, in generate_inner
    json_schema = current_handler(schema)
                  ^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/pydantic/_internal/_schema_generation_shared.py", line 36, in __call__
    return self.handler(__core_schema)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/pydantic/json_schema.py", line 506, in handler_func
    json_schema = generate_for_schema_type(schema_or_field)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/pydantic/json_schema.py", line 1410, in model_fields_schema
    json_schema = self._named_required_fields_schema(named_required_fields)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/pydantic/json_schema.py", line 1221, in _named_required_fields_schema
    field_json_schema = self.generate_inner(field).copy()
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/pydantic/json_schema.py", line 549, in generate_inner
    json_schema = current_handler(schema)
                  ^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/pydantic/_internal/_schema_generation_shared.py", line 36, in __call__
    return self.handler(__core_schema)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/pydantic/json_schema.py", line 541, in new_handler_func
    json_schema = js_modify_function(schema_or_field, current_handler)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/pydantic/_internal/_generate_schema.py", line 963, in json_schema_update_func
    json_schema = {**handler(schema), **json_schema_updates}
                     ^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/pydantic/_internal/_schema_generation_shared.py", line 36, in __call__
    return self.handler(__core_schema)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/pydantic/json_schema.py", line 506, in handler_func
    json_schema = generate_for_schema_type(schema_or_field)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/pydantic/json_schema.py", line 1289, in model_field_schema
    return self.generate_inner(schema['schema'])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/pydantic/json_schema.py", line 549, in generate_inner
    json_schema = current_handler(schema)
                  ^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/pydantic/_internal/_schema_generation_shared.py", line 36, in __call__
    return self.handler(__core_schema)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/pydantic/json_schema.py", line 506, in handler_func
    json_schema = generate_for_schema_type(schema_or_field)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/pydantic/json_schema.py", line 766, in is_instance_schema
    return self.handle_invalid_for_json_schema(schema, f'core_schema.IsInstanceSchema ({schema["cls"]})')
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/pydantic/json_schema.py", line 2069, in handle_invalid_for_json_schema
    raise PydanticInvalidForJsonSchema(f'Cannot generate a JsonSchema for {error_info}')
pydantic.errors.PydanticInvalidForJsonSchema: Cannot generate a JsonSchema for core_schema.IsInstanceSchema (<enum 'MyEnum'>)

For further information visit https://errors.pydantic.dev/2.4/u/invalid-for-json-schema

Example Code

~~ see above

Python, Pydantic & OS Version

pydantic version: 2.4.2
        pydantic-core version: 2.10.1
          pydantic-core build: profile=release pgo=false
                 install path: .../lib/python3.11/site-packages/pydantic
               python version: 3.11.4 (main, Jul  5 2023, 08:40:20) [Clang 14.0.6 ]
                     platform: macOS-13.6-arm64-arm-64bit
             related packages: mypy-1.6.1 email-validator-2.0.0.post2 typing_extensions-4.8.0 pydantic-settings-2.0.3 fastapi-0.103.2
@jurasofish jurasofish added bug V2 Bug related to Pydantic V2 pending Awaiting a response / confirmation labels Oct 22, 2023
@jurasofish
Copy link
Author

Perhaps a more practical example is like below, where you would practically provide an empty dict for e. Fails similarly

from pydantic import BaseModel
from enum import Enum


class MyEnum(Enum):
    pass


class MyModel(BaseModel):
    e: dict[MyEnum, str]


print(MyModel.model_json_schema())

@alexmojaki
Copy link
Contributor

I'm really curious, what's the goal here? Do you have empty enums? Do you actually want the only valid value for e to be {} for some reason?

@jurasofish
Copy link
Author

Do you have empty enums? Do you actually want the only valid value for e to be {} for some reason?

Yes and yes. In my case the empty enums are there as part of a pattern and just happen to be empty. Perhaps things could be structured "better" to make that not a thing but 🤷

Like uhh here's a contrived example

from pydantic import BaseModel
from enum import StrEnum


class GoodJapaneseCars(StrEnum):
    ...


class GoodSouthAfricanCars(StrEnum):
    pass  # Well maybe they'll make some in the future :)


class MyModel(BaseModel):
    japanese_cars: list[GoodJapaneseCars]
    south_african_cars: list[GoodSouthAfricanCars]


print(MyModel.model_json_schema())

@hramezani
Copy link
Member

Seems an intended behavior based on

if not cases:
# Use an isinstance check for enums with no cases.
# This won't work with serialization or JSON schema, but that's okay -- the most important
# use case for this is creating typevar bounds for generics that should be restricted to enums.
# This is more consistent than it might seem at first, since you can only subclass enum.Enum
# (or subclasses of enum.Enum) if all parent classes have no cases.

But, let's wait for @dmontagu to make it more clear.

@hramezani hramezani removed the pending Awaiting a response / confirmation label Oct 23, 2023
@alexmojaki
Copy link
Contributor

In pydantic v1 this produces {"title": "MyModel", "type": "object", "properties": {"e": {"title": "E", "enum": []}}, "required": ["e"]} (using schema_json).

If I put that into https://www.jsonschemavalidator.net/ I get an error: "Enum array must have at least one value"

But this seems to do the trick: "allOf": [{"type": "string"}, {"type": "integer"}]

@jurasofish
Copy link
Author

If I put that into https://www.jsonschemavalidator.net/ I get an error: "Enum array must have at least one value"

Seems an empty enum is okay per this spec (which pydantic is compliant with), granted it should have at least one element. Seems a bit of a grey area
https://json-schema.org/draft/2020-12/json-schema-validation#name-enum

This array SHOULD have at least one element.

@dmontagu
Copy link
Contributor

dmontagu commented Oct 24, 2023

If the issue is just generating a JSON schema, I'm sure we can make that work for enums with no cases. I'm assuming we don't need to actually implement anything different for validation (given there are no cases that it could be validated into). Since right now it just errors, it seems fine to me to make it not error and generate some best-effort JSON schema.

If you know what you want the JSON schema to be let me know otherwise I will try to just make it consistent with the existing JSON schemas and just have zero cases.

@alexmojaki
Copy link
Contributor

It seems risky to produce a schema that's technically spec-compliant but might be refused by a fussy validator.

@jurasofish
Copy link
Author

If the issue is just generating a JSON schema, I'm sure we can make that work for enums with no cases

Yeah this is the core issue for me - only reason I noticed this issue was because of fastapi docs failing to load. I'm not fussed about exactly what schema gets generated

@alexmojaki
Copy link
Contributor

Are you able to control the schema generation? If so you can do this in the meantime:

from pydantic.json_schema import GenerateJsonSchema


class MyGen(GenerateJsonSchema):
    def is_instance_schema(self, *_):
        return {"enum": []}


print(MyModel.model_json_schema(schema_generator=MyGen))

@jurasofish
Copy link
Author

Cheers I'll have a play with that

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug V2 Bug related to Pydantic V2
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants