Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

validate_assignment decorator improvements #1205

Open
8 tasks
samuelcolvin opened this issue Feb 2, 2020 · 10 comments
Open
8 tasks

validate_assignment decorator improvements #1205

samuelcolvin opened this issue Feb 2, 2020 · 10 comments
Labels
feature request strictness

Comments

@samuelcolvin
Copy link
Owner

@samuelcolvin samuelcolvin commented Feb 2, 2020

Future improvements to the validate_assignment decorator #1179:

  • arguments to the decorator, including: validators, custom config (partially fixed by #1663), return value validation
  • rewrite front page to explain the 3 or 4 primary interfaces to pydantic:
    • BaseModel
    • dataclasses
    • parse_obj_as
    • validate_arguments
  • perhaps change the error raised for invalid arguments to inherit from TypeError as other argument errors do. (Currently the normal ValidationError is raised which inherits from ValueError)
  • allow validate_assignment to be set in "strict" mode so types are not coerced, this is really just #1098
  • option to enable or disable validation via an environment variable or similar - e.g. to only use the function in development or testing or on a subset of calls
  • option to still call the function if validation fails
  • get rid of the extra arguments to the underlying model so it's useful
  • instance methods #1222
@RileyMShea
Copy link

@RileyMShea RileyMShea commented Oct 28, 2020

Python Version: 3.8
Pydantic Version: 1.17
Vscode: v1.50.1
Pylance: v2020.10.2

Using the decorator on v1.17 with arguments seems to mangle the function signature in vscode, specifically with their pylance language server. Haven't tested jedi or microsoft language servers.

@validate_arguments(config=dict(arbitrary_types_allowed=True))
# or
@validate_arguments(config=None)

# modifies signature from def foo(a:int,b:str)->str:... to def foo()->Any:

The validation still seems to work correctly, but all intellisense is lost.


@validate_arguments by itself works as expected though

Pycharm with pydantic extension does show correct signature on hover though.

@Kilo59
Copy link
Contributor

@Kilo59 Kilo59 commented Apr 16, 2021

The only feature so far that I wish @validate_arguments could provide is to use_enum_value.

I ran into this in the context of building a convenience interface to a web service that takes some query parameters.

Example
https://petstore.swagger.io/#/pet/findPetsByStatus

import enum
from typing import List

import httpx
from pydantic import validate_arguments

class Status(str, enum.Enum):
    available = "available"
    pending = "pending"
    sold = "sold"

@validate_arguments
def find_pets_by_status(status: List[Status]) -> List[dict]:
    r = httpx.get("https://petstore.swagger.io/v2/pet/findByStatus", params={"status": status})
    print(r.request.url)
    return r.json()

print(find_pets_by_status(["pending", "sold"]))
>>>https://petstore.swagger.io/v2/pet/findByStatus?status=Status.sold&status=Status.pending
>>>[]

The solution is easy enough, just extract the values for the enum.
[s.value for s in status]
Nonetheless, I miss use_enum_value 😄 .

Is this perhaps something we could just make default behavior if the enum is a mixin type?

@markedwards
Copy link

@markedwards markedwards commented Jul 30, 2021

I think validate_arguments should support an underscore arg, for args that are ignored in the body. Right now it fails with:

 File "pydantic/main.py", line 406, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for Foo
_
  extra fields not permitted (type=value_error.extra)

This is not to say that such a field should be validated, it should simply be ignored by validate_arguments, because it is an argument that is only there because it must be in the signature and is not otherwise used.

@markedwards
Copy link

@markedwards markedwards commented Sep 9, 2021

My use case for validate_arguments is to gather any errors related to the argument validation, and return them after applying custom formatting. So I do something like:

try:
    result = validate_arguments(config=dict(arbitrary_types_allowed=True))(fn)(*args, **kwargs)
except PydanticValidationError as e:
    # apply formatting to errors

The issue with this is it also catches any PydanticValidationError exceptions that occur in the body of function, and I only want to catch and format the errors related to the input.

It would be useful to be able to get the validated arguments separately from executing the function. Is this possible already, without effectively writing my own version of validate_arguments?

@PrettyWood
Copy link
Collaborator

@PrettyWood PrettyWood commented Sep 10, 2021

@markedwards
Copy link

@markedwards markedwards commented Sep 10, 2021

@PrettyWood I had not, and that seems to do the job. Thanks! One minor nitpick I would mention is mypy can't understand it, but that isn't a showstopper.

What I ideally want is a way to get the validated arguments (handling exceptions), and then call the function with the validated arguments. Is there a way to get the validated args and kwargs from validate(), so I can avoid re-validating?

@rsokl
Copy link

@rsokl rsokl commented Oct 5, 2021

Note that #1272 added support for instance-methods; the checkbox

image

should be checked.

@RobertCraigie
Copy link

@RobertCraigie RobertCraigie commented Oct 7, 2021

Not sure if this is the correct place for this issue but I've been trying to add the @validate_arguments function to https://github.com/RobertCraigie/prisma-client-py and it gets stuck in an infinite loop when the function is called.

Pydantic seems to continuously call create_model for the TypedDict types I'm using (there are a lot of auto-generated and nested types). Naively patching create_model_from_typeddict to cache models fixes the issue.

from pydantic import annotated_types

create_model_from_typeddict = annotated_types.create_model_from_typeddict

def patched_create_model(
    typeddict_cls: Type[Any], **kwargs: Any
) -> Type[BaseModel]:
    if hasattr(typeddict_cls, '__pydantic_model__'):
        return typeddict_cls.__pydantic_model__

    kwargs.setdefault('__module__', typeddict_cls.__module__)
    model = create_model_from_typeddict(typeddict_cls, **kwargs)
    typeddict_cls.__pydantic_model__ = model
    return model

annotated_types.create_model_from_typeddict = patched_create_model

I'm using the development version of pydantic, installed from GitHub.

Here's a minimal repro that will crash with a recursion error:

from pydantic import validate_arguments
from typing import Optional, TypedDict


class UserCreateInput(TypedDict):
    name: str
    post: 'PostCreateInput'


class PostCreateInput(TypedDict):
    title: str
    author: Optional['UserCreateInput']


@validate_arguments
def create_user(data: UserCreateInput) -> None:
    print(data)


create_user({'name': 'Robert'})

Edit: Just realised an issue was just created for this bug: #3297

@RobertCraigie
Copy link

@RobertCraigie RobertCraigie commented Oct 10, 2021

I would also suggest adding an argument to make model creation lazy.

I'm using validate_arguments on a lot of auto-generated functions with a lot of recursive / nested types and this significantly increases import time and memory usage.

e.g. @validate_arguments(lazy=True)

@markedwards
Copy link

@markedwards markedwards commented Jan 15, 2022

Just following up on my earlier question above about getting the validated args and kwargs, I see that this is possible:

def validate(fn):
    validator = validate_arguments()(fn)

    @wraps(fn)
    def wrapped(*args, **kwargs):
        validated = validator.validate(*args, **kwargs)
        validated_args = [getattr(validated, validator.vd.arg_mapping[I]) for i in range(len(args))]
        validated_kwargs = {k: getattr(validated, k) for k in kwargs.keys()}

Can someone speak to whether there is a better way to achieve this? Better as in less hacky and likely to continue to work?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request strictness
Projects
None yet
Development

No branches or pull requests

7 participants