feat: add support multiple dotenv files (#1497)

[rekyungmin]

Change Summary

Allow BaseSettings to accept multiple .env files in Config as a list or tuple.
The first value takes priority over the second value., this i changed below, the last file now takes priority.

Example:

.env file:

debug_mode=true
host=localhost
port=8000

.env.prod file:

debug_mode=false
host=https://example.com/services

test code:

from pydantic import BaseSettings


class Settings(BaseSettings):
    debug_mode: bool
    host: str
    port: int

    class Config:
        # `.env.prod` takes priority over `.env`
        env_file = ['.env', '.env.prod']  # (undated to reflect newer behaviour)
        env_file_encoding = 'utf-8'

s = Settings()
print(s.dict())
# {'debug_mode': False, 'host': 'https://example.com/services', 'port': 8000}

Related issue number

#1497

Checklist

• Unit tests for the changes exist
• Tests pass on CI and coverage remains at 100%
• Documentation reflects the changes where applicable
• changes/<pull request or issue id>-<github username>.md file added describing change
  (see changes/README.md for details)
• My PR is ready to review, please add a comment including the phrase "please review" to assign reviewers

[rekyungmin]

please review

[samuelcolvin]

otherwise LGTM.

[samuelcolvin]

I wonder if it would be smarter to split dotenv processing into a separate source?

I suggested the same thing here but this case is more clear cut.

[rekyungmin]

Could you elaborate on it?

[samuelcolvin]

See how sources work in settings. I propose moving dotenv logic into a whole new class.

[rekyungmin]

Good idea, but it will break backwards compatibility.
This is because customise_sources needs to be changed to take 4 callables as arguments.

[samuelcolvin]

let's leave to v2.

[samuelcolvin]

Please update.

[rekyungmin]

please review

[rekyungmin]

Thanks for the review. Please review.

[phillipuniverse]

@rekyungmin @samuelcolvin IMO this is counterintuitive, I would expect the files to be loaded in the order in which they are defined which would mean the last one in the list gets the final say. So instead of this:

for f in reverse(env_file):
    props = load_dotenv(f)

I would expect this:

for f in env_file:
    props = load_dotenv(f)

This also matches the semi-documented behavior for multiple sources in python-dotenv.

As long as it's consistent I suppose not a huge deal either way, I'm just excited to see this feature!! What's left here to merge?

[rekyungmin]

@rekyungmin @samuelcolvin IMO this is counterintuitive, I would expect the files to be loaded in the order in which they are defined which would mean the last one in the list gets the final say. So instead of this:

for f in reverse(env_file):
    props = load_dotenv(f)

I would expect this:

for f in env_file:
    props = load_dotenv(f)

This also matches the semi-documented behavior for multiple sources in python-dotenv.

As long as it's consistent I suppose not a huge deal either way, I'm just excited to see this feature!! What's left here to merge?

In Python, it is common for the rightmost value to win (e.g. unpacking, dict union).
However, in customize_sources, the leftmost value wins.
I think the left win strategy is better for consistency in the settings module.

@samuelcolvin, what do you think?

[lucafaggianelli]

Hey, any news on this PR? would love to see it merged! :)

[samuelcolvin]

Very nearly there, one tiny comment, but more generally I agree with @phillipuniverse - the right most entry in the list/tuple should "win".

Otherwise this is ready to go.

Please update.

[samuelcolvin]

Suggested change

	@no_type_check # noqa: C901 (ignore complexity)
	@no_type_check

[samuelcolvin]

please update.

[samuelcolvin]

Thanks @rekyungmin for this, and everyone for your feedback.