Field validators broken for nested models since v2.3.2 #331
Comments
|
Thanks @LachlanMarnham for reporting this. Actually, the example code shouldn't have worked before because By fixing the bug that we had before, you don't need a from pydantic import BaseModel
import os
from pydantic_settings import BaseSettings, SettingsConfigDict
class Child(BaseModel):
ATTRIBUTE: list[str]
class Parent(BaseSettings):
model_config = SettingsConfigDict(
env_prefix="PREFIX_",
env_nested_delimiter="__",
case_sensitive=False,
env_file_encoding="utf-8",
)
child: Child
os.environ["PREFIX_CHILD__ATTRIBUTE"] = '["a", "b", "c"]'
settings = Parent()
assert settings.child.ATTRIBUTE == ["a", "b", "c"] |
|
Ok thanks @hramezani. I think this is maybe a problem of documentation, with a conflict between
Which is fair enough. However,
It does feel a bit as if |
|
Good point @LachlanMarnham. The json parsing behavior was in |
Do you mean a flag in the Maybe the way I'm using Thanks |
|
Yes, I think the current behaviour is correct. what do you think? which part is wrong? |
I am "some people" here 😄 I would want to do custom parsing for only certain fields. How about some way to specify a list of fields for which to disable JSON parsing? Maybe a |
|
@fajpunk I updated my previous comment. the idea is to implement something like |
Was the behavior present in V1? I just updated a pre validator from v1 that always worked fine to a before validator and ran into this issue |
|
@Numerlor I am not sure. Could you give me the validator that works in V1 and doesn't work in V2. I can probably help you if there is a way |
The behaviors seem to differ between nested models and attributes directly on repro code v2Same with import json
import os
import typing
import pydantic
import pydantic_settings
from pydantic_settings import SettingsConfigDict
def parse_json_or_space_delimited_list(string: str) -> list[str]:
try:
return typing.cast(list[str], json.loads(string))
except json.JSONDecodeError:
return string.split(" ")
class Model(pydantic.BaseModel):
test: list[str] = []
test_validator = pydantic.field_validator("test", mode="before")(parse_json_or_space_delimited_list)
class Settings(pydantic_settings.BaseSettings):
test: Model
model_config = SettingsConfigDict(env_nested_delimiter="__")
os.environ["test__test"] = "a b c"
Settings()working v1import json
import os
import typing
import pydantic.v1
def parse_json_or_space_delimited_list(string: str) -> list[str]:
try:
return typing.cast(list[str], json.loads(string))
except json.JSONDecodeError:
return string.split(" ")
class Model(pydantic.v1.BaseModel):
test: list[str] = []
test_validator = pydantic.v1.validator("test", pre=True)(parse_json_or_space_delimited_list)
class Settings(pydantic.v1.BaseSettings):
test: Model
class Config:
env_nested_delimiter="__"
os.environ["test__test"] = "a b c"
Settings() |
|
@Numerlor, in V2, import os
from typing import Any, Tuple, Type
from pydantic import BaseModel
from pydantic.fields import FieldInfo
from pydantic_settings import BaseSettings, SettingsConfigDict, EnvSettingsSource, PydanticBaseSettingsSource
class MyEnvSettingsSource(EnvSettingsSource):
def decode_complex_value(self, field_name: str, field: FieldInfo, value: Any) -> Any:
return value.split(" ")
class NestedSettings(BaseModel):
test: list[str]
class Settings(BaseSettings):
test: NestedSettings
model_config = SettingsConfigDict(env_nested_delimiter="__")
@classmethod
def settings_customise_sources(
cls,
settings_cls: Type[BaseSettings],
init_settings: PydanticBaseSettingsSource,
env_settings: PydanticBaseSettingsSource,
dotenv_settings: PydanticBaseSettingsSource,
file_secret_settings: PydanticBaseSettingsSource,
) -> Tuple[PydanticBaseSettingsSource, ...]:
return init_settings, MyEnvSettingsSource(settings_cls), dotenv_settings, file_secret_settings
os.environ["test__test"] = "a b c"
print(Settings()) |
pydantic version: 2.8.0
OS: confirmed broken on Windows and Linux, haven't tested others
pydantic-settings version: 2.3.2
Hello, I noticed that there is a breaking change in
pydantic-settings==2.3.2, which I think was introduced by PR: #309. Here is a minimal working example:This test passes for
pydantic-settingsup to and including 2.3.1, but has been broken since 2.3.2. The test doesn't simply fail: the instantiation ofParentraises with the following traceback:So it's trying to JSON-decode the string
"a|b|c"and failing. But myfield_validatoris running in"before"modeso this shouldn't happen. Indeed, if I simply comment-out the
field_validatorI see exactly the same error. So I believe that decorator is either being ignored altogether, or"before"mode has been broken.The text was updated successfully, but these errors were encountered: