validators called twice

[samuelcolvin]

from @dmfigol on gitter:

I am failing to understand why validator is applied several times on the complex field:

import json
from typing import List, Dict, Any

from pydantic import BaseModel, ValidationError, validator

class DemoModel(BaseModel):
    data: Dict[str, Any] = {}

    @validator('data', pre=True, whole=True)
    def json_decode(cls, v):
        print(f'value is {v!r}, type: {type(v)}')
        if isinstance(v, str):
            try:
                return json.loads(v)
            except ValueError:
                pass
        return v

print(DemoModel(data='{"foo": {"bar": 123}}'))

# > value is '{"foo": {"bar": 123}}', type: <class 'str'>
# > value is 'foo', type: <class 'str'>
# > DemoModel data={'foo': {'bar': 123}}

I would appreciate if you could explain :) because of this I spent a lot of time debugging my code and after I found the problem, I still don't understand why it is working this way :(

[samuelcolvin]

Sorry you had trouble. I can understand that this was confusing.

First of all, for parsing json you would be much better off using the Json meta type:

from typing import Dict, Any

from pydantic import BaseModel, Json


class DemoModel(BaseModel):
    data: Json[Dict[str, Any]] = {}

print(DemoModel(data='{"foo": {"bar": 123}}'))

This is documented here unfortunately I broke that section of the docs when last deploying so it wouldn't have made much sense, but it's now fixed

On the particular problem you're having, the reason is that the validator is being called twice is that it's also being called when validating the dict keys:

https://github.com/samuelcolvin/pydantic/blob/c7c15bc41533551c3ad2a3655c06cc12ca68d6e9/pydantic/fields.py#L241

I guess this should be fixed so there's another keyword for validators key which means they're called on values, and whole validators shouldn't be called on keys or values.

[dmfigol]

@samuelcolvin thanks for the hint with JSON type and fixing docs!

I guess this should be fixed so there's another keyword for validators key which means they're called on values, and whole validators shouldn't be called on keys or values.

Yeah, I'd expect that the whole validator is applied once and is not invoked later when we are checking the type defined in the annotation.

[samuelcolvin]

@amogorkon this is not related to the issue being discussed here. Would be great if you could create a new issue and delete your comment.

[amogorkon]

Sorry, didn't notice your reply here. Opened a new issue and deleted comment as requested.