Using multiple BaseSettings with different prefixes

[alexdashkov]

Question

Hello,
I'm trying to migrate from the v1 to v2 and I'm looking for a way to implement multiple BaseSettings with different prefixes in the v2.
In the v1 I based my solution on this issue, and it worked well.

What I used to work with in v1:

from pydantic import BaseSettings

class MyConfig(BaseSettings.Config):
    @classmethod
    def prepare_field(cls, field) -> None:
        if 'env_names' in field.field_info.extra:
            return
        return super().prepare_field(field)

class A(BaseSettings):
    first: str
    second: str

    class Config(MyConfig):
        env_prefix = "base_prefix_1_"

class B(A):
    third: str

    class Config(MyConfig):
        env_prefix = "base_prefix_2_"

I can workaround it by passing Field(alias='base_prefix_1_...) for each field for each model in the hierarchy but it adds a lot of copy / paste. Could you advise me how to get a similar to v1 solution in v2?

Thanks for your help!

[hramezani]

Hello @alexdashkov ,

If I understand you correctly, you want to collect first and second with base_prefix_1_ prefix and third with base_prefix_2_ prefix when you initialize the B settings class. right?

[alexdashkov]

Hello @hramezani , Yes, exactly.

[hramezani]

The correct way to do it is defining alias on Field as you mentioned. There is a dirty way by creating a custom env settings source which I am not suggesting because you will lose the nested and complex model functionality.

BTW, here is the code if you want to use it:

import os
from typing import Any


from pydantic_settings import BaseSettings, EnvSettingsSource, PydanticBaseSettingsSource, SettingsConfigDict
from pydantic.fields import FieldInfo


class CustomEnvSettingsSource(EnvSettingsSource):
    def get_field_value(self, field: FieldInfo, field_name: str) -> tuple[Any, str, bool]:
        if field_name in KubectlSecrets.model_fields:
            env_name = (KubectlSecrets.model_config['env_prefix'] + field_name).lower()
            return self.env_vars.get(env_name), field_name, False
        else:
            return super().get_field_value(field, field_name)


class KubectlSecrets(BaseSettings):
    credentials: str
    google_auth_key: str

    model_config = SettingsConfigDict(env_prefix = "K8S_SECRET_")
        
    @classmethod
    def settings_customise_sources(
        cls,
        settings_cls: type[BaseSettings],
        init_settings: PydanticBaseSettingsSource,
        env_settings: PydanticBaseSettingsSource,
        dotenv_settings: PydanticBaseSettingsSource,
        file_secret_settings: PydanticBaseSettingsSource,
    ) -> tuple[PydanticBaseSettingsSource, ...]:
        return init_settings, CustomEnvSettingsSource(settings_cls), dotenv_settings, file_secret_settings


class Settings(KubectlSecrets):
    environment: str = "development"
    redis_db: str = "0"

    model_config = SettingsConfigDict(env_prefix = "")




os.environ["K8S_SECRET_CREDENTIALS"] = "dummy"
os.environ["K8S_SECRET_GOOGLE_AUTH_KEY"] = "dummy_key"
os.environ["ENVIRONMENT"] = "prod"
os.environ["REDIS_DB"] = "1"

print(Settings())

[alexdashkov]

Thanks for your answer. Indeed it doesn't look like a very scalable code as I'll need to hardcode class name there so with a big chance it will require to add as much code as to put an alias everywhere. Could you also precise what kind of "the nested and complex model functionality" I can lose by using this loader?

[hramezani]

Thanks for your answer. Indeed it doesn't look like a very scalable code as I'll need to hardcode class name there so with a big chance it will require to add as much code as to put an alias everywhere.

It was just an example, you can implement get_field_value in a clever way to not hardcode the class names.

Could you also precise what kind of "the nested and complex model functionality" I can lose by using this loader?

settings models that have a field with the type of another BaseModel. take a look at the examples on doc

Another simple way is to include kubectl_secrets as a field in the Settings model and do not inherit from KubectlSecrets

import os

from pydantic_settings import BaseSettings, SettingsConfigDict

os.environ["K8S_SECRET_CREDENTIALS"] = "dummy"
os.environ["K8S_SECRET_GOOGLE_AUTH_KEY"] = "dummy_key"
os.environ["ENVIRONMENT"] = "prod"
os.environ["REDIS_DB"] = "1"

class KubectlSecrets(BaseSettings):
    credentials: str
    google_auth_key: str

    model_config = SettingsConfigDict(env_prefix = "K8S_SECRET_")


class Settings(BaseSettings):
    kubectl_secrets: KubectlSecrets = KubectlSecrets()
    environment: str = "development"
    redis_db: str = "0"

    model_config = SettingsConfigDict(env_prefix = "")

print(Settings())

[alexdashkov]

Yup, I was thinking about this implementation, but in my context is not acceptable because it initiates nested classes during the class declaration not the initialisation. In my case I'm passing multiple env files during the initialisation of the settings class.

Anyway, I see a few possible ways to implement the functionality. Thanks for you help!