validate_assignment decorator improvements

[samuelcolvin]

Future improvements to the validate_assignment decorator #1179:

• arguments to the decorator, including: validators, custom config (partially fixed by #1663), return value validation
• rewrite front page to explain the 3 or 4 primary interfaces to pydantic:
  • BaseModel
  • dataclasses
  • parse_obj_as
  • validate_arguments
• perhaps change the error raised for invalid arguments to inherit from TypeError as other argument errors do. (Currently the normal ValidationError is raised which inherits from ValueError)
• allow validate_assignment to be set in "strict" mode so types are not coerced, this is really just #1098
• option to enable or disable validation via an environment variable or similar - e.g. to only use the function in development or testing or on a subset of calls
• option to still call the function if validation fails
• get rid of the extra arguments to the underlying model so it's useful
• instance methods #1222

[RileyMShea]

Python Version: 3.8
Pydantic Version: 1.17
Vscode: v1.50.1
Pylance: v2020.10.2

Using the decorator on v1.17 with arguments seems to mangle the function signature in vscode, specifically with their pylance language server. Haven't tested jedi or microsoft language servers.

@validate_arguments(config=dict(arbitrary_types_allowed=True))
# or
@validate_arguments(config=None)

# modifies signature from def foo(a:int,b:str)->str:... to def foo()->Any:

The validation still seems to work correctly, but all intellisense is lost.

---

@validate_arguments by itself works as expected though

Pycharm with pydantic extension does show correct signature on hover though.

[Kilo59]

The only feature so far that I wish @validate_arguments could provide is to use_enum_value.

I ran into this in the context of building a convenience interface to a web service that takes some query parameters.

Example
https://petstore.swagger.io/#/pet/findPetsByStatus

import enum
from typing import List

import httpx
from pydantic import validate_arguments

class Status(str, enum.Enum):
    available = "available"
    pending = "pending"
    sold = "sold"

@validate_arguments
def find_pets_by_status(status: List[Status]) -> List[dict]:
    r = httpx.get("https://petstore.swagger.io/v2/pet/findByStatus", params={"status": status})
    print(r.request.url)
    return r.json()

print(find_pets_by_status(["pending", "sold"]))
>>>https://petstore.swagger.io/v2/pet/findByStatus?status=Status.sold&status=Status.pending
>>>[]

The solution is easy enough, just extract the values for the enum.
[s.value for s in status]
Nonetheless, I miss use_enum_value 😄 .

Is this perhaps something we could just make default behavior if the enum is a mixin type?

[markedwards]

I think validate_arguments should support an underscore arg, for args that are ignored in the body. Right now it fails with:

 File "pydantic/main.py", line 406, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for Foo
_
  extra fields not permitted (type=value_error.extra)

This is not to say that such a field should be validated, it should simply be ignored by validate_arguments, because it is an argument that is only there because it must be in the signature and is not otherwise used.

[markedwards]

My use case for validate_arguments is to gather any errors related to the argument validation, and return them after applying custom formatting. So I do something like:

try:
    result = validate_arguments(config=dict(arbitrary_types_allowed=True))(fn)(*args, **kwargs)
except PydanticValidationError as e:
    # apply formatting to errors

The issue with this is it also catches any PydanticValidationError exceptions that occur in the body of function, and I only want to catch and format the errors related to the input.

It would be useful to be able to get the validated arguments separately from executing the function. Is this possible already, without effectively writing my own version of validate_arguments?

[PrettyWood]

Hi @markedwards
Have you seen https://pydantic-docs.helpmanual.io/usage/validation_decorator/#validate-without-calling-the-function?

[markedwards]

@PrettyWood I had not, and that seems to do the job. Thanks! One minor nitpick I would mention is mypy can't understand it, but that isn't a showstopper.

What I ideally want is a way to get the validated arguments (handling exceptions), and then call the function with the validated arguments. Is there a way to get the validated args and kwargs from validate(), so I can avoid re-validating?

[rsokl]

Note that #1272 added support for instance-methods; the checkbox

should be checked.

[RobertCraigie]

Not sure if this is the correct place for this issue but I've been trying to add the @validate_arguments function to https://github.com/RobertCraigie/prisma-client-py and it gets stuck in an infinite loop when the function is called.

Pydantic seems to continuously call create_model for the TypedDict types I'm using (there are a lot of auto-generated and nested types). Naively patching create_model_from_typeddict to cache models fixes the issue.

from pydantic import annotated_types

create_model_from_typeddict = annotated_types.create_model_from_typeddict

def patched_create_model(
    typeddict_cls: Type[Any], **kwargs: Any
) -> Type[BaseModel]:
    if hasattr(typeddict_cls, '__pydantic_model__'):
        return typeddict_cls.__pydantic_model__

    kwargs.setdefault('__module__', typeddict_cls.__module__)
    model = create_model_from_typeddict(typeddict_cls, **kwargs)
    typeddict_cls.__pydantic_model__ = model
    return model

annotated_types.create_model_from_typeddict = patched_create_model

I'm using the development version of pydantic, installed from GitHub.

Here's a minimal repro that will crash with a recursion error:

from pydantic import validate_arguments
from typing import Optional, TypedDict


class UserCreateInput(TypedDict):
    name: str
    post: 'PostCreateInput'


class PostCreateInput(TypedDict):
    title: str
    author: Optional['UserCreateInput']


@validate_arguments
def create_user(data: UserCreateInput) -> None:
    print(data)


create_user({'name': 'Robert'})

Edit: Just realised an issue was just created for this bug: #3297

[RobertCraigie]

I would also suggest adding an argument to make model creation lazy.

I'm using validate_arguments on a lot of auto-generated functions with a lot of recursive / nested types and this significantly increases import time and memory usage.

e.g. @validate_arguments(lazy=True)

[markedwards]

Just following up on my earlier question above about getting the validated args and kwargs, I see that this is possible:

def validate(fn):
    validator = validate_arguments()(fn)

    @wraps(fn)
    def wrapped(*args, **kwargs):
        validated = validator.validate(*args, **kwargs)
        validated_args = [getattr(validated, validator.vd.arg_mapping[I]) for i in range(len(args))]
        validated_kwargs = {k: getattr(validated, k) for k in kwargs.keys()}

Can someone speak to whether there is a better way to achieve this? Better as in less hacky and likely to continue to work?