Environment names for complex types

[sirex]

Checks

• I added a descriptive title to this issue
• I have searched (google, github) for similar issues and couldn't find anything
• I have read and followed the docs and still think this feature/change is needed
• After submitting this, I commit to one of:
  • Look through open issues and helped at least one other person
  • Hit the "watch" button on this repo to receive notifications and I commit to help at least 2 people that ask questions in the future
  • Implement a Pull Request for a confirmed bug

Feature Request

Output of python -c "import pydantic.utils; print(pydantic.utils.version_info())":

             pydantic version: 1.7.3
            pydantic compiled: True
                 install path: /tmp/pydantic-compex-env-names/env/lib/python3.9/site-packages/pydantic
               python version: 3.9.1 (default, Dec 13 2020, 11:55:53)  [GCC 10.2.0]
                     platform: Linux-5.10.7-3-MANJARO-x86_64-with-glibc2.32
     optional deps. installed: []

from typing import Dict
from pydantic import BaseModel, BaseSettings, Field

class Backend(BaseModel):
    type: str = None
    dsn: str = None

class Settings(BaseSettings):
    backends: Dict[str, Backend] = Field(default_factory=dict)

In order to manage these settings via envvars, nested components could be separated with __, like this:

BACKENDS__DEFAULT__TYPE=postgresql
BACKENDS__DEFAULT__DSN=postgresql://localhost/db

In order to remove default backend, following envvar could be set:

BACKENDS__DEFAULT=

Or list of dict keys could be given like this:

BACKENDS=default,sqlite

This would remove all backends, except default and sqlite.

I found similar issue about adding custom value parsers, so this could be used for lists and also for overriding keys.

[daviskirk]

The issues #2232 (comment) and #1929 (comment) also touch on this.
Since there are solutions for this I would say this isn't a required feature, but if this is interesting to more people I would be open to having a go at a pull request to support this "pydantic natively".

[rmporsch]

@daviskirk We certainly would be interested in this feature as well!

[rorybyrne]

+1 I also like the idea of using __ to set nested Settings values.

I think the JSON approach is counter-intuitive because you must set multiple variables via a single environment variable.

[hramezani]

Parsing envs for nested field improved in pydantic-settings

FYI, pydantic-settings now is a separate package and is in alpha state. you can install it by pip install pydantic-settings --pre and test it.

Here is your example in pydantic-settings:

from typing import Dict, Optional

from pydantic import BaseModel, Field
from pydantic_settings import BaseSettings


class Backend(BaseModel):
    type: Optional[str] = None
    dsn: Optional[str] = None

class Settings(BaseSettings):
    backends: Dict[str, Backend] = Field(default_factory=dict)