EmailStr & friends error as Custom Data Types with typing.Annotated. #6506
Comments
zakstucke
added
bug V2
|
Thanks @zakstucke for reporting this 🙏
|
|
You can achieve what you want with the following, since import typing as tp
from pydantic import BaseModel, EmailStr
class Foo(BaseModel):
email: EmailStr |
|
@Kludex it's definitely a contrived example but from what I understand this is a valid thing to do? The use cases are to potentially end up with a different type than I've gotten my understanding from class Model(BaseModel):
post_code: Annotated[str, PostCodeAnnotation]it implements the same method |
|
It's not really a valid thing to do. The lack of the handler argument is a quirk of us never really deciding if we want to support two signatures or one, I'd ignore the fact that it works at all for now until we document it. Perhaps you want one of these two options: from typing import Annotated, Any
from pydantic import AfterValidator, GetCoreSchemaHandler, TypeAdapter, ValidationError
from pydantic.networks import EmailStr
from pydantic_core import CoreSchema, core_schema
UpperEmailStr = Annotated[EmailStr, AfterValidator(str.upper)]
ta = TypeAdapter(UpperEmailStr)
assert ta.validate_python('adrian@example.com') == 'ADRIAN@EXAMPLE.COM'
try:
ta.validate_python('not an email')
except ValidationError as e:
print(e)
"""
1 validation error for function-after[upper(), function-after[_validate(), str]]
value is not a valid email address: The email address is not valid. It must have exactly one @-sign. [type=value_error, input_value='not an email', input_type=str]
"""
class MyType(str):
@classmethod
def __get_pydantic_core_schema__(cls, _source_type: Any, handler: GetCoreSchemaHandler) -> CoreSchema:
email_schema = handler.generate_schema(EmailStr)
return core_schema.no_info_after_validator_function(MyType, email_schema)
ta = TypeAdapter(MyType)
assert isinstance(ta.validate_python('adrian@example.com'), MyType)
try:
ta.validate_python('not an email')
except ValidationError as e:
print(e)
"""
1 validation error for function-after[upper(), function-after[_validate(), str]]
value is not a valid email address: The email address is not valid. It must have exactly one @-sign. [type=value_error, input_value='not an email', input_type=str]
""" |
|
@adriangb okay I get what you’re saying. So I think this issue becomes more general: Why shouldn’t all custom pydantic types be usable as intermediary validatory types in Annotated is the unsung hero of V2, it’s just so much more ergonomic and concise than creating custom types for reusables in a lot of scenarios, we’re able to remove loads of custom types from V1 that were a pain and messed with type hints around the place, it’s so nice to have validation but keep the original type statically. At the minute with the Pitch:
def as_b_v(t: type) -> BeforeValidator:
return BeforeValidator(lambda v: TypeAdapter(t).validate_python(v))
This seems so intuitive and catch all to me! Here's a complete example using the import typing as tp
from pydantic_core import core_schema
from pydantic import EmailStr, BeforeValidator, TypeAdapter
encrypt = decrypt = lambda s: "".join(reversed(s))
class User:
email: str
def __init__(self, email: str):
self.email = email
def __repr__(self):
return "User(email='{}')".format(self.email)
@classmethod
def __get_pydantic_core_schema__(cls, *args, **kwargs) -> core_schema.CoreSchema:
return core_schema.no_info_after_validator_function(
cls.from_email,
core_schema.any_schema(),
)
@classmethod
def from_email(cls, email: str) -> tp.Self:
"""Returns the user model from an email address"""
print("User.from_email() run with email: {}".format(email))
return User(email=email)
class StrCustomDecoding(str):
"""Decodes str with custom decoding"""
@classmethod
def __get_pydantic_core_schema__(cls, *args, **kwargs) -> core_schema.CoreSchema:
return core_schema.no_info_after_validator_function(
cls.decoder,
core_schema.str_schema(),
)
@classmethod
def decoder(cls, v):
print("StrCustomDecoding.decoder() run with v: {}".format(v))
return decrypt(v)
def as_b_v(t: type) -> BeforeValidator:
return BeforeValidator(lambda v: TypeAdapter(t).validate_python(v))
UserFromEmail = tp.Annotated[User, as_b_v(EmailStr), as_b_v(StrCustomDecoding)]
user = TypeAdapter(UserFromEmail).validate_python(encrypt("foo@bar.com"))
print(user)Note: without the |
|
Conveniently #6531 suffers from exactly this generic problem, and would be easily answered if this concept was supported with: from pydantic import constr
tp.Annotated[str, Field(min_length=3, max_length=10), constr(strip_whitespace=True)]This can also be shimmed as-is with import typing as tp
from pydantic import BeforeValidator, TypeAdapter, constr, Field
def as_bv(t: type) -> BeforeValidator:
"""Converts a type into a BeforeValidator"""
ta = TypeAdapter(t)
return BeforeValidator(lambda v: ta.validate_python(v))
MyT = tp.Annotated[str, Field(min_length=3, max_length=10), as_bv(constr(strip_whitespace=True))]
res = TypeAdapter(MyT).validate_python(" 123456789 ")
print(f"res: '{res}'") |
|
I probably should have posted #6531 (comment) here, but for those reasons, I don't think this is a viable approach and I'm going to have to close the issue. @zakstucke I really appreciate you thinking about this and am happy to discuss other approaches that solve the problem for you but I just don't think this is a viable path for forward for Pydantic. |
|
@adriangb fair enough, all valid critiques! Just to note I've been using this shim a lot today, and seems to do the job, so it's okay if you don't think it's practical internally as it is easy to replicate outside as a user, I'm just sure less performant than it could be :) Having said that, here's my thoughts on all your critiques:
I think the main point of this whole thing is I'm finding really great value in Feels like there's this wall between the two conceptually that doesn't need to be there! Let me know if it's worth discussing further and if so where that should be done :) |
|
How about we wait a bit to see what other solutions come up or how many others need something like this? |
|
Sounds good! |
Initial Checks
mainbranch, or equivalentDescription
When
EmailStrand it looks like most of the types inpydantic/networks.pyare used as subtypes withtyping.Annotatedthey error with below traceback.I believe all that's needed is adding
_handler: _annotated_handlers.GetCoreSchemaHandleras a third argument to each__get_pydantic_core_schema__()method inpydantic/networks.py, it seems to fix it forEmailStrat least, this might be prevalant across more types outsidenetworks.pyas well.https://docs.pydantic.dev/latest/usage/types/custom/#creating-custom-classes-using-__get_pydantic_core_schema__
Example Code
Python, Pydantic & OS Version
Selected Assignee: @dmontagu
The text was updated successfully, but these errors were encountered: