Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-use validators? #940

Closed
chopraaa opened this issue Oct 25, 2019 · 12 comments · Fixed by #941
Closed

Re-use validators? #940

chopraaa opened this issue Oct 25, 2019 · 12 comments · Fixed by #941
Labels

Comments

@chopraaa
Copy link

@chopraaa chopraaa commented Oct 25, 2019

Is it possible to create a validator and import it everywhere else it's used?

I have validators for certain fields that are present in multiple classes that I'd like to call simply without defining them.

@chopraaa chopraaa added the question label Oct 25, 2019
@samuelcolvin

This comment has been minimized.

Copy link
Owner

@samuelcolvin samuelcolvin commented Oct 25, 2019

I'm on my mobile, but should be as simple as doing something like:

class MyModel(BaseModel):
    check_foo = validator('foo')(reused_validator)

Have a try and let me know if that works or doesn't work or I have the syntax slightly wrong.

@dmontagu

This comment has been minimized.

Copy link
Collaborator

@dmontagu dmontagu commented Oct 25, 2019

@samuelcolvin I don't think that works due to this check:

        if not in_ipython():  # pragma: no branch
            ref = f.__module__ + '.' + f.__qualname__
            if ref in _FUNCS:
                raise ConfigError(f'duplicate validator function "{ref}"')
            _FUNCS.add(ref)

I think we can probably fix this though.

@dmontagu

This comment has been minimized.

Copy link
Collaborator

@dmontagu dmontagu commented Oct 25, 2019

Here's a self contained demonstration that it doesn't currently work.

from pydantic import BaseModel, validator


def double_validator(cls, value):
    return value * 2


class A(BaseModel):
    x: int

    double = validator('x')(double_validator)


class B(BaseModel):
    x: int

    double = validator('x')(double_validator)

# pydantic.errors.ConfigError: duplicate validator function "__main__.double_validator"
@samuelcolvin

This comment has been minimized.

Copy link
Owner

@samuelcolvin samuelcolvin commented Oct 25, 2019

Humm, possibly. We could either drop that check or make it optional.

Or you could setup a validator which just calls the reused validator.

Maybe allow_name_reuse as another argument on a validator?

@samuelcolvin samuelcolvin reopened this Oct 25, 2019
@dmontagu

This comment has been minimized.

Copy link
Collaborator

@dmontagu dmontagu commented Oct 25, 2019

Here's a hack that works for now:

from copy import deepcopy
from typing import Callable

from pydantic import BaseModel, validator
from pydantic.typing import AnyCallable


def reused_validator(
    *fields: str,
    pre: bool = False,
    each_item: bool = False,
    always: bool = False,
    check_fields: bool = True,
    whole: bool = None,
) -> Callable[[AnyCallable], classmethod]:
    def dec(f: AnyCallable) -> classmethod:
        f = deepcopy(f)
        f.__qualname__ = f"{f.__qualname__}{reused_validator.count}"
        reused_validator.count += 1
        return validator(*fields, pre=pre, whole=whole, always=always, check_fields=check_fields)(f)

    return dec


reused_validator.count = 0


def double_validator(cls, value):
    return value * 2


class A(BaseModel):
    x: int
    double = reused_validator('x')(double_validator)


class B(BaseModel):
    x: int
    double = reused_validator('x')(double_validator)


print(A(x=1))
print(B(x=1))
# x=2
# x=2

This is using v1 imports, but I also checked that if you fix the imports it works in v0.32.2.

@samuelcolvin related -- are models supposed to print like that now? On master I'm not seeing the model class name when I print the model.

@dmontagu

This comment has been minimized.

Copy link
Collaborator

@dmontagu dmontagu commented Oct 25, 2019

@samuelcolvin I think it makes sense to keep that error in many cases because the most common situation is that you've accidentally duplicated the name of the validator inside the class, which would cause problems if the logic changed (and is presumably why you added the error in the first place).

I was thinking it could make sense to just look for a . in the __qualname__ before raising the error. If the function was defined globally in a module, you almost certainly meant to reuse it.

Separately, I also think it would be good to add allow_name_reuse as a keyword argument so that you could do the same thing with a function defined at the class level.

Also, we could add a check for whether something is already a classmethod when building the validator so that you could define it as such inside the class and not have a misleading signature.

I can start putting together a PR for this.

@samuelcolvin

This comment has been minimized.

Copy link
Owner

@samuelcolvin samuelcolvin commented Oct 25, 2019

are models supposed to print like that now? On master I'm not seeing the model class name when I print the model

Yes, that's standard for __str__ methods, I think you reviewed the PR. Bit late to change now.

@dmontagu

This comment has been minimized.

Copy link
Collaborator

@dmontagu dmontagu commented Oct 25, 2019

Yeah, I remember it now 😅; I'm fine with it since we are calling the repr on nested values. I just haven't seen it in the wild much yet (still using 0.32.2 with fastapi :/), so it caught me by surprise.

@dmontagu dmontagu mentioned this issue Oct 25, 2019
4 of 4 tasks complete
@chopraaa

This comment has been minimized.

Copy link
Author

@chopraaa chopraaa commented Oct 25, 2019

Thanks for the quick response on this. I wasn't even aware we could call validators using check_foo = validator('foo')(reused_validator).

@dmontagu

This comment has been minimized.

Copy link
Collaborator

@dmontagu dmontagu commented Oct 25, 2019

@chopraaa that's basically all a python decorator does behind the scenes

@skewty

This comment has been minimized.

Copy link
Contributor

@skewty skewty commented Nov 6, 2019

I came across this issue this morning. Thanks for the temporary work around idea.

@dmontagu unless I am failing to see / understand something, I don't see where/how count ever gets incremented. That said it seems to compile and run. This fact almost interested me more :)

def reused_validator(
    *fields: str,
    pre: bool = False,
    each_item: bool = False,
    always: bool = False,
    check_fields: bool = True,
    whole: bool = None,
) -> Callable[[AnyCallable], classmethod]:
    def dec(f: AnyCallable) -> classmethod:
        f = deepcopy(f)
        f.__qualname__ = f"{f.__qualname__}{reused_validator.count + 1}"
        return validator(*fields, pre=pre, whole=whole, always=always, check_fields=check_fields)(f)

    return dec


reused_validator.count = 0

so I added

print(f"Function name will be: {f.__qualname__}")

and this is what came out:

Function name will be: partition_range_check1
Function name will be: partition_range_check11

which explains why it works. That said count is perhaps a poor name given the reason it works :) (unless we are counting notches on a stick)

I wasn't sure if this was by design or by mistake. I Just wanted to share for others who might also be struggling to understand.

@dmontagu

This comment has been minimized.

Copy link
Collaborator

@dmontagu dmontagu commented Nov 6, 2019

That was just a mistake -- I meant to actually increment the value. It's now fixed above!

(Clearly there are some edge cases this wouldn't handle well, e.g., if you end the validator name with a number. This is only intended as an admittedly hacky interim solution.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.