Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A plea to the devs for some sanity regarding environments and configuration #12351

Open
TehWardy opened this issue Jul 19, 2019 · 6 comments
Open
Labels

Comments

@TehWardy
Copy link

@TehWardy TehWardy commented Jul 19, 2019

PLEASE PLEASE PLEASE see some sense here!

As has been highlighted repeatedly before but for some reason always seems to get dropped (for example: #2019) the current way to "manage" the environment that the app is running under is just not flexible enough nor clear enough and presents a real headache for those experienced devs managing complex environment setups in the enterprise and newbies alike.

I'd like to suggest that the environment come from the build config so defining new environments is simply a matter of adding a build configuration in VS and then using that to build the project and as such instead of this ...

config.AddJsonFile("appsettings.json", optional: false, reloadOnChange: true);
                config.AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true, reloadOnChange: true);

... we should go back to a "config transforms style" for generating the config on build using the build configuration as the environment name we are building for which would result in this ....

config.AddJsonFile("appsettings.json", optional: false, reloadOnChange: true);

... which would be computed on build from the files referenced in the current model.

This would bring things inline with how existing .Net 4.x projects work making situations like this for example: #1328 a little more bearable.

I also don't like the idea that all servers currently need all the envrironment based configs deployed "just in case" something out of my control is gets changed at a systemic level.

The app should only have 1 config and that should be right on deploy.

More information ...

https://stackoverflow.com/questions/57110877/aspnet-core-applications-build-configs-and-deploy

@Eilon

This comment has been minimized.

Copy link
Member

@Eilon Eilon commented Jul 19, 2019

What are your thoughts on staging vs. production? For example, in a production app that I run, I deploy it to "Staging", where it runs with Environment=Staging. If I'm happy with the testing on the staging server, I swap the Staging and Production slots, and what was previously my Staging instance, is now my Production instance. If the app was "compiled for staging," then I couldn't swap Staging/Production slots because the app might be different. So, in this case, at deployment time it is not entirely known what configuration the app will run under.

I'm not discounting the merits of the proposal, but rather pointing out some aspects that might have led to the current design.

@TehWardy

This comment has been minimized.

Copy link
Author

@TehWardy TehWardy commented Jul 22, 2019

That's a great point @Eilon ...

Deployments are already a tricky prospect but as much as much possible I would urge people to write code-bases that don't change regardless of environment and to as much as possible keep their environments the same for maintainability reasons.

I accept this is not always possible in which case I would encourage more modularity in deploys and branching in the code base rather than having to have my app handle all possible environments my build service and code base should "pick the block of code relevant to the environment and build that".

For any given block of code it's generally considered best practice that you have one version of it and run it anywhere ... that's literally the motto on to which .Net Core is built is it not?

So if it's a different job (because of some environment issue) then it's a different block of code.

Perspective being:

You would run different builds for a linux, docker, or windows deploy ... those builds might pull in different "environment specific dependencies" ... but you wouldn't expect to lift and shift a build from one to the other would you?

I think your scenario is about re-purposing a staging deploy in to a production one, that is not something I would typically do. I would physically copy the deploy with a prod config on to a prod server so I know that my prod servers are physically separate to my staging ones and are managed as such. I guess it boils down to how much confidence you have in your deploys, testing and environment consistency though.

@KevinCathcart

This comment has been minimized.

Copy link

@KevinCathcart KevinCathcart commented Jul 22, 2019

You would run different builds for a linux, docker, or windows deploy ... those builds might pull in different "environment specific dependencies" ... but you wouldn't expect to lift and shift a build from one to the other would you?

It is intended that you can take the output of dotnet publish from Linux and use it on Windows and vice versa, as long as you published a Framework Dependant Deployment (FDD), which is the default, instead of a SCD or FDE.

Also for the overwhelming majority of applications it is indeed possible to use the same build in all environments. That is the use case ASP.NET Core tries to optimize for. Hence features like "appsettings.{env.EnvironmentName}.json", designed to even let you deploy all the files absolutely identically to all environments.


Now let's look at the current environment design.

The ASP.NET Core design is that the application is able to know the name of the environment it is running in. It needs to learn that from somewhere. By default it uses an environment variable to learn this. If you don't like using the environment variable, it is entirely possible to have custom code that sets the environment name in whatever way you want. However, lets look at the default case further.

When hosted in IIS, a generated web.config file can set this environment variable for your application. You can specify the environment name to be used in the generated web.config file in any of the following ways: EnvironmentName property in a publish profile, specifying an EnvironmentName msbuild property in the dotnet build or msbuild command line, specifying it in the project file, or specifying it in a Directory.Build.props file. Either of the last two can be used with build configurations so that you can define a different value per build configuration. If you don't specify the environment name, then you will get a web.config that does not set this, and will use the machine level environment variable, or fall back to "Production".

So if you can meet the ideal of running the same build in all environments, you probably would want to either use a global machine level environment variable if possible. If not you can use custom tooling to modify the web.config file for each environment, or use publish profiles to generate the web.config file for each environment, all while using exactly the same build.

If you cannot meet the ideal of the same build in all environments, then using build configurations with the MSBuild property is a reasonable workaround.

Now for non-IIS hosting, it is your responsibility to set the environment variable, and neither the publish profile nor MSBuild property tooling will have any real impact. This is most definitely unfortunate and suboptimal.

@TehWardy

This comment has been minimized.

Copy link
Author

@TehWardy TehWardy commented Jul 23, 2019

@KevinCathcart In my environments literally the only thing that changes for my code base is the config information. I went to crazy extreme lengths to ensure this was the case keeping the code simple and my stack clean.

Now please don't take this as a rant but merely constructive criticism ...

Your post doesn't address the point raised by @Eilon though as the environment variable cannot be the source in his case for multiple deploys on a single machine and there's literally zero documentation about how to test "generating different env configs" from VS, how @Eilon's kind of setup might work or even how to "set the environment variable just for the app" without setting it at a machine level from what I have seen.

I don't know a single person that manages code bases / works in a dev team that manually runs the MSBuild command line directly any more, usually this is done automatically by tooling since teams like Azure Devops seem to be pushing "use this deployment activity that hides the underlying MSBuild call from you" and the bulk of devs building .Net projects use VS so hand cranking MSBuild command lines IMO is not a reasonable work around. Also the use of the term "work around" is usually reserved for "this aint working, but it's usable with this quick hacky solution" ... is this the standard that M$ is promoting for .Net Core?
Do you guys run MSBuild instead of just hitting F5 in VS when debugging code?

So being stuck between a rock and a hard place leaves us confused as consumers.

I'd like to see a guide or something that shows me how to use the publish function in VS to deploy to multiple servers with a different value for the environment configured at the app level somewhere as I have not been able to get this to work and thought that it would just hang off the build config name or something (clearly I was wrong).

It boils down to when I do this ...

config.AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true, reloadOnChange: true);

... how do i force that EnvironmentName variable to be a given value without having to mess with the machine (potentially breaking other apps) but purely based on my build config name for just that app execution / build?

I just find the whole concept stupid that for my app to be configured and run correctly it has to hang off some string specified at the machine level.
Why would anyone even imply that's a good idea when the best practice is to put your config for that app running on that machine there and correct in the first place as part of the deploy?

Now I have to put all configs there and let the app choose one ... based on "magic" because you can be dam sure that the machine level variable will always be wrong if i have more than 1 app on the machine and they all need different settings for the environment variable..

@KevinCathcart

This comment has been minimized.

Copy link

@KevinCathcart KevinCathcart commented Jul 24, 2019

For IIS hosted sites, using the web config is one option. (As I explained, publish profiles let you handle inserting the value into the web.config for each environment, although there is no UI support for it currently. You must instead manually edit the publish profile XML).

If, for an IIS hosted project, having different web.config file generated by publish profiles is not a good option, and machine level environment variables won't work with your set-up, then yeah things get rather awkward. You basically need to set up a custom mechanism for setting the environment name in code.

For non-IIS hosted applications, depending on how you deploy the application, there may be a convenient way to set environment variables for it. Docker for example often uses environment variables to configure a container, so most Docker based deployment systems will have such support. Azure App Service also has a convenient way to do this.

However, in the general case, you are on your own. One typical suggestion would be to have a .bat file or UNIX shell script that sets the variable and then launches the application. and you would point whatever process actually starts the application to this shell script instead of dotnet MyApplication.dll. Alternatively you could change whatever invokes the dotnet executable to add an additional command line argument: dotnet MyApplication.dll environment=staging. And of course custom code to set the environment name in some other way is possible.

Adding custom code to set the environment name is pretty easy: just call .UseEnvironment("Staging") on the host builder. Obviously instead of hard coding a name you would want to do something like parsing the path to the executable, or looking for a special file that indicates the current environment. (Possibly in the parent folder, so that the deployment process does not wipe it out if configured to delete existing files).

Another possibility is to ignore the environment name completely. Let it always be Production except when launched from within Visual Studio with F5, in which case Visual Studio will set it to Development.
In that scenario, you don't use the `$"appsettings.{env.EnvironmentName}.json" files at all. You only use the main one, which you customize per environment. The built in deployment tools of Visual Studio don't really have support for this, so you would probably want to use a third party deployment tool, which usually do support this scenario. (Yes, I know your request is basically to have the built-in deployment features add support for this scenario.)

@TehWardy

This comment has been minimized.

Copy link
Author

@TehWardy TehWardy commented Jul 24, 2019

I really hate how M$ justifies making functionality worse citing "progress" when it's simply a case of fixing what wasn't broken. Every new post you write on here is "well you could do X" ... lists of workarounds / hacks don't address the fact that this should just happen, for example lets examine your last post ...

Hack 1

For IIS hosted sites, using the web config is one option. (As I explained, publish profiles let you handle inserting the value into the web.config for each environment, although there is no UI support for it currently. You must instead manually edit the publish profile XML).

... and you beat that down yourself with the paragraph that followed it.

Forcing the app to need to know about the environment beyond it's name

For non-IIS hosted applications, depending on how you deploy the application, there may be a convenient way to set environment variables for it. Docker for example often uses environment variables to configure a container, so most Docker based deployment systems will have such support. Azure App Service also has a convenient way to do this.

... how long before that goes wrong do you think?
I also thought the "environment" was always ".net" ... virtualisation is how we get the interop right?

Hack 2

However, in the general case, you are on your own. One typical suggestion would be to have a .bat file or UNIX shell script that sets the variable and then launches the application. and you would point whatever process actually starts the application to this shell script instead of dotnet MyApplication.dll. Alternatively you could change whatever invokes the dotnet executable to add an additional command line argument: dotnet MyApplication.dll environment=staging. And of course custom code to set the environment name in some other way is possible.

... just generates more work for me and doesn't address the fact that my deploy has already leaked information the environment should not be privvy to and not only that, multiple apps constantly changing machine level variables ... that'll never go wrong right?

Hack 3

Adding custom code to set the environment name is pretty easy: just call .UseEnvironment("Staging") on the host builder. Obviously instead of hard coding a name you would want to do something like parsing the path to the executable, or looking for a special file that indicates the current environment. (Possibly in the parent folder, so that the deployment process does not wipe it out if configured to delete existing files).

... what value do i give that call, it has to come from somewhere? and your suggestion is "go looking around the environment for it" ... nice now my app needs to know about the path it's expected to be deployed to and the parent structure in order to work, that'll never go wrong right?

Hack 4

Another possibility is to ignore the environment name completely. Let it always be Production except when launched from within Visual Studio with F5, in which case Visual Studio will set it to Development.

... my problem, paraphrased, and re-raised essentially which you then go on to explain ...

In that scenario, you don't use the `$"appsettings.{env.EnvironmentName}.json" files at all. You only use the main one, which you customize per environment. The built in deployment tools of Visual Studio don't really have support for this, so you would probably want to use a third party deployment tool, which usually do support this scenario. (Yes, I know your request is basically to have the built-in deployment features add support for this scenario.)

... Yup that's the idea, and my ideal is to have this tied to build configs in some way without third party tools since this is literally already in .Net 4.x so in theory should be a lift and shift fix (maybe swapping out web.config for a json transform but it's very much an internally solved problem on M$ side).

Summary

Your recommendation is "use one of these hacks which mostly solve your problem by introducing overhead,new problems, or dependencies on me, which didn't previously exist because ".

I'm literally asking that an existing feature of .Net 4 be kept in .Net Core since I depend on it and Core has no viable alternative that produces the same result.

"in the general case, you are on your own."

Count the thumbs up on all the issues listed in #2019 the could be solved by this request and then tell me that. Do a deploy that contains my app, built for the box, with only the config information that the app needs to run correctly on that box. Such a rare feature ... can't see any value in it at all right?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.