Environment variable override

[rpfernandezjr]

Describe your problem or feature you'd like added

In order to deploy into a kubernetes infrastructure with more ease, I'd like the ability for any variables defined within the environment, to supersede any entries defined in the env.json file that is provided.

Describe the solution you'd like

I'd like for env_sample.json to be loaded as the default configuration. Any value defined in the environment during initialization, should override what has been loaded as default.

For example, if env_sample.json was loaded as our default configuration, it contains the following entries for log levels

DJANGO_LOG_LEVEL: DEBUG
DJANGO_DEBUG: true
DJANGO_TEMPLATE_DEBUG: true
ROOT_LOG_LEVEL: INFO

If my environment were to have the following defined

export DJANGO_LOG_LEVEL=WARN
export ROOT_LOG_LEVEL=DEBUG

I'd like the variables from my environment to override the default settings that were loaded from the json file. In the end we end up with these values being loaded.

DJANGO_LOG_LEVEL: WARN
DJANGO_DEBUG: true
DJANGO_TEMPLATE_DEBUG: true
ROOT_LOG_LEVEL: DEBUG

Additional context

If it heps any, I was accomplishing this with this snippet in settings.py

if os.getenv("DJANGO_CONFIG_DEFAULTS"):
    # optionally load settings from an environment variable
    with open(os.getenv("DJANGO_CONFIG_DEFAULTS")) as f:
        CONFIG = json.load(f)

    # override defaults with variables defined by the environment
    ENV = {**CONFIG, **os.environ}
else:
    ENV = os.environ

downside of my snippent is that everything from the environment was loaded as a string. I then needed to cast variables that I knew were a bool (str_2_bool) or int when assigning from ENV

[emilychee]

Hi! My partner @apalmer16 and I are currently students enrolled in EECS 481. We recently saw a presentation regarding MyLA in class and wanted to contribute to fixing an open source issue. We just wanted to let you know that we will be attempting to implement this feature!

[justin0022]

Thanks @emilychee, we appreciate your help!

[jonespm]

In the code we should have defaults defined that match the json but inevitably that does get out of sync with an external configuration file.

For instance the settings.py has
DEBUG = ENV.get('DJANGO_DEBUG', True)

I'd personally like to switch to defining most of the settings in Django admin using a live settings package. I'm not sure which one of those to pick. Probably one that's been updated recently/often and is going to be compatible with Django 2. Then we could load all of the defaults into the database via migrations.

[apalmer16]

So if we decided to test using django-constance, would we install the redis backend or database backend?

[jonespm]

Just use the MySQL database backend that's already in the docker-compose and used for the application.

[jonespm]

Here were my comments from #820 on how we might support this

I feel like for this to work as individual variables we'd have to either have a library that does type inference which is going to be a little more work than this (unless there's one out there).

Probably the easier way to fulfill the initial request I'd think is just to use this special ENV called
ENV_JSON

The thing with that is it's either or. You either have the entire env defined in ENV_JSON or you have it defined in the file. What I'd do instead of this.

• Try to load The env from the file into ENV (on lines 32-38)
  8 Whether or not this passes or fails get the value ENV_JSON from the environment as ENV_JSON (lines 27-29)
• Update the values from ENV_JSON into ENV. (You can just do this with ENV.update(ENV_JSON)
• If ENV is still empty after all this, just set it to os.environ so it still runs.

This wouldn't allow someone to define individual values in the ENV and have them override but they could define a JSON in the env and it would override. I feel like this would be sufficient for the request and avoid modifying this entire file. Sorry as it looks like you did a a lot of work here.

[jonespm]

@ssciolla I think you code from https://github.com/tl-its-umich-edu/course-inventory/blame/master/environ.py#L27 would solve this issue. Could you look at this when you get a chance? Maybe it'd be useful to refactor this over to a file like environ.py just like this instead of loading it in settings.

[ssciolla]

@jonespm, yeah, I can start looking at this.

[jonespm]

This relates to #1200

[ssciolla]

@jonespm, can you QA this when you have a minute? Test plan is on PR #1425.

[zqian]

I have followed the Test plan on PR #1425:

• set "ROOT_LOG_LEVEL" to DEBUG in .env file
• set different value for "TZ" param in env.hjson or env_udp.hjson file
• verified the log message of ENV value for "TZ" overridden