Choices enum not reflected in Swagger

[smythp]

I'm using Piccolo with FastAPI, which generates documentation in Swagger. In my tables file, I've created a table (relationship) that has a field based on an enum

    class RelationshipType(str, enum.Enum):
        colleague = "colleague"
        friend = "friend"
        ...

    type= Varchar(choices=RelationshipType)

When I use create_pydantic_model to generate a pydantic model based on the table, I had expected the "choices" field to get passed along to Swagger, but that doesn't seem to be the case. Swagger treats it as a vanilla string.

I dumped the schemas for both a Pydantic model created directly that uses an enum and a Pydantic model created using the create_pydantic_model method, and they're laid out pretty differently. I'll add them at the end.

Do you have any guidance on how to get the enum created in piccolo tables to get reflected in Swagger? Do I have to drop down and create Pydantic models directly if I'd like this functionality?

# Direct Pydantic model 
model.construct().schema()
{'title': 'RelationshipTest', 'type': 'object', 'properties': {'class_1': {'title': 'Class 1', 'type': 'integer'}, 'class2': {'title': 'Class2', 'type': 'integer'}, 'type': {'$ref': '#/definitions/RelationshipType'}}, 'required': ['class_1', 'class2', 'type'], 'definitions': {'RelationshipType': {'title': 'RelationshipType', 'description': 'An enumeration.', 'enum': ['colleague', 'friend', 'relative'], 'type': 'string'}}}

# Creating a model from Piccolo table
model.construct().schema()
{'title': 'RelationshipIn', 'type': 'object', 'properties': {'first_contact': {'title': 'First Contact', 'extra': {'foreign_key': True, 'to': 'contact', 'target_column': 'id', 'help_text': None, 'choices': None}, 'nullable': True, 'type': 'integer'}, 'second_contact': {'title': 'Second Contact', 'extra': {'foreign_key': True, 'to': 'contact', 'target_column': 'id', 'help_text': None, 'choices': None}, 'nullable': True, 'type': 'integer'}, 'type': {'title': 'Type', 'extra': {'help_text': None, 'choices': {'colleague': {'display_name': 'Colleague', 'value': 'colleague'}, 'friend': {'display_name': 'Friend', 'value': 'friend'}, 'relative': {'display_name': 'Relative', 'value': 'relative'}, 'partner': {'display_name': 'Partner', 'value': 'partner'}, 'ex': {'display_name': 'Ex', 'value': 'ex'}, 'child': {'display_name': 'Child', 'value': 'child'}, 'teacher': {'display_name': 'Teacher', 'value': 'teacher'}, 'student': {'display_name': 'Student', 'value': 'student'}}}, 'nullable': False, 'maxLength': 255, 'type': 'string'}}, 'help_text': None}

[dantownsend]

@smythp Thanks for reporting this issue. You're right, the column choices don't currently reflect properly in the Pydantic schema.

If we have the following table:

class RelationshipType(str, enum.Enum):
    colleague = "colleague"
    friend = "friend"

class Person(Table):
    type = Varchar(choices=RelationshipType)    

It should becomes this Pydantic model:

class PersonModel(BaseModel):
    type: RelationshipType

The code responsible is here:

piccolo/piccolo/utils/pydantic.py

Lines 210 to 229 in 52f01ae

	if isinstance(column, (Decimal, Numeric)):
	value_type: t.Type = pydantic.condecimal(
	max_digits=column.precision, decimal_places=column.scale
	)
	elif isinstance(column, Varchar):
	value_type = pydantic.constr(max_length=column.length)
	elif isinstance(column, Array):
	value_type = t.List[column.base_column.value_type] # type: ignore
	elif isinstance(column, (JSON, JSONB)):
	if deserialize_json:
	value_type = pydantic.Json
	else:
	value_type = column.value_type
	validators[f"{column_name}_is_json"] = pydantic.validator(
	column_name, allow_reuse=True
	)(pydantic_json_validator)
	else:
	value_type = column.value_type
	_type = t.Optional[value_type] if is_optional else value_type

If the column has defaults provided, then the value_type should be the choices enum.

You'll notice that we do add the choices under the extra field:

piccolo/piccolo/utils/pydantic.py

Lines 241 to 244 in 52f01ae

	extra = {
	"help_text": column._meta.help_text,
	"choices": column._meta.get_choices_dict(),
	}

This is for Piccolo Admin. In retrospect it would have made more sense to implement choices in the official Pydantic way - it was probably just ignorance at the time about this particular Pydantic feature.

I think it would be good to make this change (i.e. changing the value_type to an Enum, but also keeping the choices under extra for backwards compatibility). We just need to do some testing to make sure Piccolo Admin doesn't break for some reason.

If it's not possible to do it in a backwards compatible way, we have been considering an alternative API to create_pydantic_model (see #331), where we can tidy up these issues.

In the immediate term, defining a Pydantic model by hand is the best workaround until we get this issue fixed.

[dantownsend]

One other workaround for now, instead of defining the entire Pydantic model from scratch, you can override some of the fields:

PersonModel = create_pydantic_model(table=Person)

class PersonModelExtended(PersonModel):
    type: RelationshipType # you can just override certain fields

[smythp]

@dantownsend That workaround isn't too onerous. Glad I wasn't missing anything , and I'll keep an eye on the issue. :)

[dantownsend]

@smythp Cool, thanks

[waldner]

Not sure it's the right place, but another minor annoyance I've come across when generating pydantic models from Table is that there's no way to specify that extra fields are forbidden (ie extra = 'forbid' in pydantic's Config subclass). This could perhaps be an extra option to the create_pydantic_model function (apologies if it's already there; I wasn't able to find it)

The current solution is to manually (re)define the pydantic model adding the Config subclass, but I think there's room for improvement.

[dantownsend]

Not sure it's the right place, but another minor annoyance I've come across when generating pydantic models from Table is that there's no way to specify that extra fields are forbidden (ie extra: 'forbid' in pydantic's Config subclass). This could perhaps be an extra option to the create_pydantic_model function (apologies if it's already there; I wasn't able to find it)

I haven't used that feature of Pydantic before, but I can see it being useful.

It seems like extra: allow is the default behaviour.

We can definitely add this feature - just needs an extra argument to create_pydantic_model.