ValidationError raised when a field has multiple aliases and one is passed through the constructor and the other is available as an environment variable.

[lucacerone]

When a setting class uses multiple aliases for one of the fields, a validation error is raised if I use one alias in the constructor,
but there is an environment variable with the name of the other alias.

This is the version of the packages I have installed:

pydantic==2.6.4
pydantic-settings==2.2.1
pydantic_core==2.16.3

I define a class like:

from pydantic import Field, AliasChoices
from pydantic_settings import BaseSettings, SettingsConfigDict

class Example(BaseSettings):
  model_config = SettingsConfigDict(env_prefix='PREFIX')

  name: str
  last_name: str = Field(validation_alias=AliasChoices("PREFIX_LAST_NAME", "PREFIX_SURNAME"))

If I run:

Example(name="john", PREFIX_LAST_NAME="doe")

the object is validated correctly.

However, if there was an environment PREFIX_SURNAME, then the validation would fail.

For example:

import os
os.environ["PREFIX_SURNAME"] = "smith"

Example(name="john", PREFIX_LAST_NAME="doe")

fails with this error:

ValidationError                           Traceback (most recent call last)
Cell In[26], line 1
----> 1 Example(name="john", PREFIX_LAST_NAME="doe")

File ~/miniforge3/envs/domestika-redshift-dev/lib/python3.12/site-packages/pydantic_settings/main.py:84, in BaseSettings.__init__(__pydantic_self__, _case_sensitive, _env_prefix, _env_file, _env_file_encoding, _env_ignore_empty, _env_nested_delimiter, _env_parse_none_str, _secrets_dir, **values)
     71 def __init__(
     72     __pydantic_self__,
     73     _case_sensitive: bool | None = None,
   (...)
     82 ) -> None:
     83     # Uses something other than `self` the first arg to allow "self" as a settable attribute
---> 84     super().__init__(
     85         **__pydantic_self__._settings_build_values(
     86             values,
     87             _case_sensitive=_case_sensitive,
     88             _env_prefix=_env_prefix,
     89             _env_file=_env_file,
     90             _env_file_encoding=_env_file_encoding,
     91             _env_ignore_empty=_env_ignore_empty,
     92             _env_nested_delimiter=_env_nested_delimiter,
     93             _env_parse_none_str=_env_parse_none_str,
     94             _secrets_dir=_secrets_dir,
     95         )
     96     )

File ~/miniforge3/envs/domestika-redshift-dev/lib/python3.12/site-packages/pydantic/main.py:171, in BaseModel.__init__(self, **data)
    169 # `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks
    170 __tracebackhide__ = True
--> 171 self.__pydantic_validator__.validate_python(data, self_instance=self)

ValidationError: 1 validation error for Example
PREFIX_SURNAME
  Extra inputs are not permitted [type=extra_forbidden, input_value='smith', input_type=str]
    For further information visit https://errors.pydantic.dev/2.6/v/extra_forbidden

however:

import os
os.environ["PREFIX_SURNAME"] = "smith"

Example(name="john", PREFIX_SURNAME="doe")

works as I expected, and uses the value "doe" that I explicitly passed through the constructor.

Maybe I have the wrong expectation, but I thought that if I explicitly constructed the object using one of the two aliases, the value coming from the environment should have been discarded, and not passed as an extra argument to the object causing the validation error.

[hramezani]

Thanks @lucacerone for reporting this issue.

Could you please test your example on main? we have some unreleased changes that probably fixes your problem.

[lucacerone]

@hramezani thanks for the quick response.

I tried and the original issue is fixed, i.e. running:

import os
from pydantic import Field, AliasChoices
from pydantic_settings import BaseSettings, SettingsConfigDict

class Example(BaseSettings):
  model_config = SettingsConfigDict(env_prefix='PREFIX')

  name: str
  last_name: str = Field(validation_alias=AliasChoices("PREFIX_LAST_NAME", "PREFIX_SURNAME"))


os.environ["PREFIX_SURNAME"] = "smith"
Example(name="john", PREFIX_LAST_NAME="doe")

now does work.

However, the behaviour that before worked (having PREFIX_SURNAME as an env variable, and using PREFIX_SURNAME in the constructor), now raises a validation error:

import os
from pydantic import Field, AliasChoices
from pydantic_settings import BaseSettings, SettingsConfigDict

class Example(BaseSettings):
  model_config = SettingsConfigDict(env_prefix='PREFIX')

  name: str
  last_name: str = Field(validation_alias=AliasChoices("PREFIX_LAST_NAME", "PREFIX_SURNAME"))

os.environ["PREFIX_SURNAME"] = "smith"
Example(name="john", PREFIX_SURNAME="doe")

Returns this error:

---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
Cell In[4], line 4
      1 import os
      2 os.environ["PREFIX_SURNAME"] = "smith"
----> 4 Example(name="john", PREFIX_SURNAME="doe")

File ~/miniforge3/envs/pydantic-settings/lib/python3.12/site-packages/pydantic_settings/main.py:177, in BaseSettings.__init__(self, _case_sensitive, _nested_model_default_partial_update, _env_prefix, _env_file, _env_file_encoding, _env_ignore_empty, _env_nested_delimiter, _env_nested_max_split, _env_parse_none_str, _env_parse_enums, _cli_prog_name, _cli_parse_args, _cli_settings_source, _cli_parse_none_str, _cli_hide_none_type, _cli_avoid_json, _cli_enforce_required, _cli_use_class_docs_for_groups, _cli_exit_on_error, _cli_prefix, _cli_flag_prefix_char, _cli_implicit_flags, _cli_ignore_unknown_args, _cli_kebab_case, _secrets_dir, **values)
    147 def __init__(
    148     self,
    149     /,
   (...)
    175     **values: Any,
    176 ) -> None:
--> 177     super().__init__(
    178         **self._settings_build_values(
    179             values,
    180             _case_sensitive=_case_sensitive,
    181             _nested_model_default_partial_update=_nested_model_default_partial_update,
    182             _env_prefix=_env_prefix,
    183             _env_file=_env_file,
    184             _env_file_encoding=_env_file_encoding,
    185             _env_ignore_empty=_env_ignore_empty,
    186             _env_nested_delimiter=_env_nested_delimiter,
    187             _env_nested_max_split=_env_nested_max_split,
    188             _env_parse_none_str=_env_parse_none_str,
    189             _env_parse_enums=_env_parse_enums,
    190             _cli_prog_name=_cli_prog_name,
    191             _cli_parse_args=_cli_parse_args,
    192             _cli_settings_source=_cli_settings_source,
    193             _cli_parse_none_str=_cli_parse_none_str,
    194             _cli_hide_none_type=_cli_hide_none_type,
    195             _cli_avoid_json=_cli_avoid_json,
    196             _cli_enforce_required=_cli_enforce_required,
    197             _cli_use_class_docs_for_groups=_cli_use_class_docs_for_groups,
    198             _cli_exit_on_error=_cli_exit_on_error,
    199             _cli_prefix=_cli_prefix,
    200             _cli_flag_prefix_char=_cli_flag_prefix_char,
    201             _cli_implicit_flags=_cli_implicit_flags,
    202             _cli_ignore_unknown_args=_cli_ignore_unknown_args,
    203             _cli_kebab_case=_cli_kebab_case,
    204             _secrets_dir=_secrets_dir,
    205         )
    206     )

File ~/miniforge3/envs/pydantic-settings/lib/python3.12/site-packages/pydantic/main.py:214, in BaseModel.__init__(self, **data)
    212 # `__tracebackhide__` tells pytest and some other tools to omit this function from tracebacks
    213 __tracebackhide__ = True
--> 214 validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
    215 if self is not validated_self:
    216     warnings.warn(
    217         'A custom validator is returning a value other than `self`.\n'
    218         "Returning anything other than `self` from a top level model validator isn't supported when validating via `__init__`.\n"
    219         'See the `model_validator` docs (https://docs.pydantic.dev/latest/concepts/validators/#model-validators) for more details.',
    220         stacklevel=2,
    221     )

ValidationError: 1 validation error for Example
PREFIX_SURNAME
  Extra inputs are not permitted [type=extra_forbidden, input_value='doe', input_type=str]
    For further information visit https://errors.pydantic.dev/2.10/v/extra_forbidden

FYI, this is the version of the packages I am using:

pydantic==2.10.6
pydantic-settings @ git+https://github.com/pydantic/pydantic-settings.git@4b6fd3d0968bfabffb8dd7c413b4e13ac35bd3d9
pydantic_core==2.27.2

[hramezani]

Thanks @lucacerone for the feedback.

The problem is pydantic-settings will pick up the first AliasChoices - "PREFIX_LAST_NAME" even you passed PREFIX_SURNAME in env. so the final values that pydantic-settings passes to pydantic would be:

{'PREFIX_LAST_NAME': 'smith', 'name': 'john', 'PREFIX_SURNAME': 'doe'}

@kschwab do you have time to take a look and prepare a fix for this? unfortunately, I am busy these days. I think we need to touch _get_resolved_field_value:

pydantic-settings/pydantic_settings/sources.py

Line 565 in 4b6fd3d

def _get_resolved_field_value(self, field: FieldInfo, field_name: str) -> tuple[Any, str, bool]:

[lucacerone]

Thanks a lot @hramezani !

[kschwab]

Hi @lucacerone @hramezani, I created #550 for resolution. It was caused by InitSettingsSource not resolving aliases.

[lucacerone]

Many thanks for taking care of this @kschwab !