Unable to access 'fields' and 'values' in a validator when using @validate_call

[maread99]

Initial Checks

• I confirm that I'm using Pydantic V2

Description

Hi, I've come across a possible loss of functionality with v2...

In v1 is was possible to access 'fields' and 'values' in a validator when using @validate_arguments. Is this still possible in v2 with @validate_call?

The example explains what I'm after, using the case of validating later parameters depending on the values received by an earlier parameter (in the same way that a model's validators can access the values of earlier fields).

I raised the question in #6345 with a more convoluted example using custom types. I'm not so concerned which approach to use as simply being able to do this in some way.

Possibly related, this part of the docs notes that:

These names (together with "args" and "kwargs") may or may not (depending on the function's signature) appear as fields on the internal Pydantic model accessible via .model

...but from where can .model be accessed? Am I not seeing the elephant?

Thank you.

Example Code

from pydantic import BeforeValidator, validate_call
from typing import Annotated

def validator_a(a_val) -> str:
    # Validate something.
    return a_val

def validator_b(b_val) -> str:
    # Validate something according to the value received by `func`s 'a' parameter, i.e.
    # how can I access in this scope the object received by `func`s 'a' parameter and/or
    # as returned by `validator_a`?

    # If validation fails, raise a custom exception that references the parameter name,
    # i.e. can the parameter name be accessed from this scope ('b' in this case)?

    return b_val

@validate_call
def func(
    a: Annotated[str, BeforeValidator(validator_a)],
    b: Annotated[str, BeforeValidator(validator_b)],
):
    return a + b

Python, Pydantic & OS Version

pydantic version: 2.0.3
pydantic-core version: 2.3.0 release build profile
install path: C:\.venv\Lib\site-packages\pydanticpython version: 3.9.13 (tags/v3.9.13:6de2ca5, May 17 2022, 16:36:42) [MSC v.1929 64 bit (AMD64)]
platform: Windows-10-10.0.22621-SP0
optional deps. installed: ['typing-extensions']

Selected Assignee: @Kludex

[dmontagu]

There is no longer a .model in v2 — we don't use an auxiliary basemodel to do the validation in v2, but just perform the validation directly using pydantic-core.

Unfortunately, it seems that section of the docs was overlooked and is no longer correct and should be updated.

For what it's worth, in principle I think it should be possible to create a model from a callable by combining inspect.signature and pydantic.create_model if you need this functionality.

[maread99]

Thanks for the reply @dmontagu.

So it seems the functionality to access 'fields' and 'values' in a validator when using @validate_call has been lost in v2(?). If this is the case then could you label this issue as a feature request please? Thank you.

[dmontagu]

Could you provide an example of how you want to use these things? Would a function that accepts a callable and builds a model based on the signature of that callable be sufficient for your purposes?

[dmontagu]

For what it's worth, this is an attempt at replicating the v1 logic:

import inspect
from typing import Callable, Any, Type

from pydantic import BaseModel, Field, create_model
from pydantic.alias_generators import to_pascal


def validate_call_model(f: Callable[..., Any]) -> Type[BaseModel]:
    signature = inspect.signature(f)
    parameters = signature.parameters
    field_definitions: dict[str, Any] = {}
    for name, param in parameters.items():
        annotation, default = param.annotation, param.default
        if annotation is param.empty:
            annotation = Any
        if default is param.empty:
            default = Field(...)
        field_definitions[name] = (annotation, default)

    model = create_model(to_pascal(f.__name__), __module__=str(f.__module__), **field_definitions)
    return model

# Usage example:
def my_func(x: int, y, z: str = 2) -> float:
    return 1.1


my_func_model = validate_call_model(my_func)
print(my_func_model)
# > <class '__main__.MyFunc'>
print(my_func_model.model_fields)
# > {'x': FieldInfo(annotation=int, required=True), 'y': FieldInfo(annotation=Any, required=True), 'z': FieldInfo(annotation=str, required=False, default=2)}

[maread99]

Thank you for looking at this further.

Could you provide an example of how you want to use these things?

Really just as the example in my initial comment, i.e. to be able to validate later function parameters depending on the values received by earlier parameters (in the same way that a model's validators can access the values of earlier fields). My reference to .model came about from looking through the doc and thinking it might be something to explore to this end, although not being able to find it in v2.

[dmontagu]

Sorry, I was confused about your request, I understand now.

Yeah, this is definitely a piece of functionality that is not present now.

This is pretty gross, but it's the best way I can figure out to achieve this today:

from typing import Tuple, Any, Dict

from pydantic import validate_call
from pydantic_core import core_schema


def func(a: str, b: str):
    return a + b


def _validate_func_arguments(args_kwargs: Tuple[Tuple[Any, ...], Dict[str, Any]]):
    args, kwargs = args_kwargs
    a = kwargs.get('a', args[0])
    b = kwargs.get('b', args[1])

    if a == b:
        raise ValueError('a and b must not be equal')

    return args, kwargs


def _get_pydantic_core_schema_func(source_type, handler):
    schema = handler(source_type)

    schema['arguments_schema'] = core_schema.no_info_after_validator_function(
        _validate_func_arguments, schema['arguments_schema']
    )

    return schema


func.__get_pydantic_core_schema__ = _get_pydantic_core_schema_func

func = validate_call(func)

print(func('abc', 'def'))
#> 'abcdef'

func('abc', 'abc')
"""
pydantic_core._pydantic_core.ValidationError: 1 validation error for func
  Value error, a and b must not be equal [type=value_error, input_value=ArgsKwargs(('abc', 'abc')), input_type=ArgsKwargs]
    For further information visit https://errors.pydantic.dev/2.1/v/value_error
"""

That said, I wouldn't promise that this code snippet will work forever; while I don't see any reason we'd change the relevant functionality, it's also making use of various implementation details of ValidateCallWrapper etc.

I think the best thing we could do in terms of feature work to better support this sort of thing would be to add a better way to provide after-validators that can modify the args/kwargs, and to provide some sort of cleaner api for accessing their values by name regardless of how they were passed into the function.

I suspect the amount of interest in this feature relative to the amount of work it would take to support it would be fairly high, but I would be open to reviewing related PRs. (Might require getting hands dirty with pydantic core though.) If you wanted to go down this route though it would probably be best to get buy-in on the idea before spending too much time implementing though.

---

Another viable path might be creating a separate decorator that actually builds a model and accepts model validators, and then passes the values to the function. Not sure if that would be of interest to you but it would probably be easier to do without modifying existing internals, at the expense of a little additional overhead.

[maread99]

Wow, I would never have got there! Thank you for that example and for the thought you've given the issue.

I use pydantic to abstract away the validatation and parsing of input to public functions, including through these custom types. How I use it is a bit clunky although I found it preferable to validating input at the start of each function or letting errors that are rooted in invalid inputs arise where they may with unintelligible messages.

I'd love to be in a position to offer a PR, although my dependence on pydantic isn't sufficient to justify the time that I'd require to get up to speed with the internals (and learning Rust hasn't made it off my todo list yet).

I've drafted out a simple library to verify function inputs against type annotations (the tests give an idea of what it does). With the changes in v2, I think my path of least resistance has shifted to dropping pydantic in favour of this more limited, specific solution.

Thank you again, I'll certainly keep an eye on this feature request - interested to see if there are other users with similar needs.

[maread99]

I've published the library as valimp. It's limited to validating, parsing and coercing inputs to public functions. I've included a note to the README on how it compares to the Pydantic V1 @validate_argumnents and V2 @validate_call decorators. If you feel I've overlooked anything, let me know and I'll revise it. Thanks.

[samuelcolvin]

I think this is the same issue as #7448, I think we should considered reverting the behaviour to allow field_name and data to be available in all validator functions.