Specify settings in appsettings.json files instead of Kubernetes config

[ddaspit]

Currently, most settings are configured as environment variables in the Kubernetes deployment configuration. These settings should be defined in the appropriate appsettings.json file based on the environment (connection strings, logging, engines, etc.). Only secrets should be configured in by environment variables.

By default, ASP.NET Core supports three environments: Production, Staging, and Development. appsettings.json is for production and contains the defaults for the other files. appsettings.Staging.json is for QA. We could use it for external QA. appsettings.Development.json is for development. We will probably need to add another environment for internal QA, if it has different settings than external QA. We can call it InternalStaging or something like that, so that would use appsettings.InternalStaging.json.

[johnml1135]

So here is a case for continuing with the environment variables:

• There are 3 different k8s environments that are all very similar (~80% overlap). However, these environments are almost completely different from a "bare metal" deployment for development, namely:
  • The interconnection of deployments (port routing, etc.) are the same for k8s but different for bare metal. Moreover, having all the inter-connections in the helm charts rather than separated into different appsettings files in different repos makes it easier to understand and allows for more rapid reconfiguration.
  • The ClearML and S3 authentication are different for each deployment but contain secrets that would not be able to be put in appsettings.json
  • The Auth0 credentials do change and are the only thing that could meaningfully put in separate appsettings..json files

Therefore, I recommend keeping the deployments as is, that is, using environment variables declared in helm charts for most of the configuration of the k8s deployments.

[ddaspit]

There were three main reasons I suggested this change:

1. For an ASP.NET Core project, the expectation is that most of these settings (other than secrets) will be configured in the appsettings.json. There is value in being as idiomatic as possible.
2. The settings are more clearly organized in a json file rather than the weird naming that is required when configuring using environment variables.
3. The appsettings.json provide a couple of nice features, such as environment support (dev, staging, production) with shared settings configured in the main appsettings.json and hot reload of settings when they are changed.

I will leave the decision of whether or not keeping most of the config in the appsettings file is valuable, since you are primarily responsible for maintaining most of these settings for deployment. Feel free to close this issue if you are happy with the current approach.

[johnml1135]

I believe maintaining most of the parameters in the helm chart is easier - it also allows for updating the behavior without updating the docker build. I will close this issue for the time being.