Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix typing of model_validator #6514

Merged
merged 3 commits into from Jul 11, 2023
Merged

Fix typing of model_validator #6514

merged 3 commits into from Jul 11, 2023

Conversation

adriangb
Copy link
Member

@adriangb adriangb commented Jul 7, 2023

Fixes #6277. Closes python/mypy#15620

skip change file check

Selected Reviewer: @dmontagu

@cloudflare-pages
Copy link

cloudflare-pages bot commented Jul 7, 2023

Deploying with  Cloudflare Pages  Cloudflare Pages

Latest commit: 96de51b
Status: ✅  Deploy successful!
Preview URL: https://d695b048.pydantic-docs2.pages.dev
Branch Preview URL: https://fix-typing.pydantic-docs2.pages.dev

View logs

@adriangb
Copy link
Member Author

adriangb commented Jul 7, 2023

@Kludex want to try and apply the same thing to the other protocols? We might be able to simplify quite a bit

@adriangb
Copy link
Member Author

adriangb commented Jul 7, 2023

please review

Copy link
Contributor

@ybressler ybressler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: I am familiarizing myself with the pydantic codebase through reviews – my review is therefore a bit n00b.

Left a Q about the behavior of Protocol + mypy. (Otherwise, IMO, looks great.)

Comment on lines +412 to +415
ModelAfterValidatorWithoutInfo = Callable[[_ModelType], _ModelType]
"""A `@model_validator` decorated function signature. This is used when `mode='after'` and the function does not
have info argument.
"""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly, mypy didn't recognize the the previously annotated Protocol object as callable? (Even though it had call method? By switching to explicitly annotating with Callable, the annotations are correct?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like that we're annotating callable functions as callable though. (Though Protocol is supposed to solve this for us, no?)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In theory you can use either, but it's a little bit weird because here we want to require the self argument, in particular because it helps distinguish from the other alternative signatures. But my hack to make that work does not seem to have been compatible with mypy so this is a simpler albeit less pedantic version that hopefully will be.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here we want to require the self argument, in particular because it helps distinguish from the other alternative signatures

I don't know if I'm understanding your thought correctly.
But if those validators work with other signatures as well (e.g. with classmethod signatures) it would be misleading imo. The implementation as far as I saw supports static-, class- and instance-signatures and even decorators. Therefore, I don't think you have to enforce an instance-signature here (i.e. with self argument).

Actually, even the documentation suggests a classmethod-signature for model_validator with mode="after". Does the solution of this PR really work with classmethod-signatures (and maybe with a @classmethod decorator) though, did you test it? If not, is anything wrong with doing it similar to field_validator? It seems to work there.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, even the documentation suggests a classmethod-signature for model_validator with mode="after"

This is precisely what we wanted to avoid, the docs are wrong.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I tried around by myself but I really couldn't get a Protocol typing for instance methods to work together with the decorator. At least with mypy. Very sad, maybe it's possible in the future.

Btw, why do you even want to avoid classmethod signatures? Is there anywhere a discussion about this?

Copy link
Member Author

@adriangb adriangb Jul 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Btw, why do you even want to avoid classmethod signatures? Is there anywhere a discussion about this?

You're getting an instance of the model. Of course (cls: Type['Model'], v: 'Model') and add an @classmethod works but why do that instead of (self) which works great with type checkers, etc.

Copy link

@lord-haffi lord-haffi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just ran into this issue today and found the issue and this PR and thought why not telling my thoughts on it :)
I really like the idea of Protocol. But annotating callables with it with an explicit self-argument doesn't really seem to work with mypy. However, pyright seems to get it to work. I found this little discussion (more like an issue) regarding this topic if anyone is interested:
python/typing#1040

The PR looks good, but I had a question as well.
Thanks for the good work, appreciate it :)

Comment on lines +412 to +415
ModelAfterValidatorWithoutInfo = Callable[[_ModelType], _ModelType]
"""A `@model_validator` decorated function signature. This is used when `mode='after'` and the function does not
have info argument.
"""
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here we want to require the self argument, in particular because it helps distinguish from the other alternative signatures

I don't know if I'm understanding your thought correctly.
But if those validators work with other signatures as well (e.g. with classmethod signatures) it would be misleading imo. The implementation as far as I saw supports static-, class- and instance-signatures and even decorators. Therefore, I don't think you have to enforce an instance-signature here (i.e. with self argument).

Actually, even the documentation suggests a classmethod-signature for model_validator with mode="after". Does the solution of this PR really work with classmethod-signatures (and maybe with a @classmethod decorator) though, did you test it? If not, is anything wrong with doing it similar to field_validator? It seems to work there.

pydantic/functional_validators.py Outdated Show resolved Hide resolved

_AnyModelWrapValidator = Union[ModelWrapValidator, ModelWrapValidatorWithoutInfo]
_AnyModeBeforeValidator = Union[ModelBeforeValidator, ModelBeforeValidatorWithoutInfo]
_AnyModeAfterValidator = Union[ModelAfterValidator, ModelAfterValidatorWithoutInfo]
_AnyModelAfterValidator = Union[ModelAfterValidator[_ModelType], ModelAfterValidatorWithoutInfo[_ModelType]]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this reparametrization do anything? It looks like the same typevar is used in the original definition of this thing. Is this for pseudo-documentation reasons or does it type check differently? Should it make use of a TypeAlias?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's because generic aliases require the type var reparametrization

@adriangb adriangb merged commit cbff378 into main Jul 11, 2023
48 checks passed
@adriangb adriangb deleted the fix-typing branch July 11, 2023 18:41
@lmrlyv
Copy link

lmrlyv commented Jul 13, 2023

Hi @adriangb,
Thanks for the fix!

Since this commit is already merged and addresses the typing issue, I don't want to silence it in our codebase.
Could you please share the planned date for the next release?

@Kludex
Copy link
Member

Kludex commented Jul 13, 2023

I believe that either today, or tomorrow (likely).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants