Ability to provide configuration for different environments in app/etc/local.xml

[mzeis]

My proposed story for the backlog

Provide configuration for different environments in app/etc/local.xml
"As a Magento developer, I want to be able to define different configuration settings for different environments in app/etc/local.xml."

Acceptance criteria:

• Defining important settings like database connection, cache backends and session handling for different environments in one XML file
• Extending from other environment sections so only changes have to be specified
• The environment to be used is defined by SetEnv

---

Reasoning behind this

In Magento 1, developers have to make sure that app/etc/local.xml doesn't get copied from test / staging servers to production servers because this would break the live system in most cases. app/etc/local.xml has to be excluded from versioning and/or deployment process which is error-prone.

In Magento 2, developers should be able to keep one configuration file and trigger which part of the configuration is used. I don't believe this has to be very fine-grained. In my experience the most common scenarios are different database, caching and session settings.

Nowadays, many frameworks provide the functionality to define setting for different environments in one or more configuration file(s) and load the settings corresponding to a variable set by SetEnv or similar.
The community and the Magento 2 team would have to decide on the concrete implementation but I think this is a good thing (tm).

References:

• Zend_Application Quick Start using Zend_Tool
• An example on how this could be solved by wimvds @ Stackoverflow
• Solution in Symfony2 - using different files
• Solution from FLOW3 - also using different files

[Vinai]

This is a good idea, I second that. Probably would be much easier (and less complex) to implement using different, additional, app/etc/local.xml files, then building a more complex merging of sections from one single file.

[mzeis]

@Vinai You are right, this would be easier. Do you have a suggestion how inheritance could be provided (e.g. if only small parts of the app/etc/local.xml files are different)?

[Vinai]

The simplest solution that comes to my mind to pick up your suggestion and add an ENV variable that specifies a additional local.xml file to be merged with the base local.xml, for example

$_SERVER['MAGE_ENV'] = "dev-vinai";
Merged local.xml files:
app/etc/local.xml
app/etc/local-dev-vinai.xml

$_SERVER['MAGE_ENV'] = "live";
Merged local.xml files:
app/etc/local.xml
app/etc/local-live.xml

[mzeis]

I guess a one-level inheritance would be good enough in most cases - should be the best trade-off between simplicity and flexibility.

[jigneshthummar]

Good idea..

[magento-team]

Great proposition, @mzeis !
You can implement the solution and we add it to Magento 2 repository.
Our ACs are following:

• define environment config options in separate config files, one config file per environment (@Vinai's idea)
• concrete environment config file is loaded depending on environment variable
• all config files in app/etc are loaded and merged as it is now
• environment config file has the highest priority, so it should be loaded after all config files
• all changes are covered with tests

As you can see, extending environment configuration is not included in ACs list.
Currently all *.xml configs are merged, so there is ability to define some global options and merge local options with them. Looks like it should be enough.
Could you please give some real examples, when it's necessary to merge options from different environment configs?

[Vinai]

Have you currently got resources to implement this, @mzeis?

[speedupmate]

Setting up server variables and stuff will not be much different than changing the local.xml file for each environment from CI point of view as it still depends on making a separate configuration file or setting somewhere in your CI deployment scripts. And this affects all the files that might be different for different environments (not just local.xml but .htaccess and flag files)

On this level you can make Magento just include one of the "prefix_"local.xml or local"_suffix".xml (like code pool does a fallback) file and be done with it and your deployment script just injects the right file to your environment as most CI tools are designed to differ configuration files. File naming in this context is just for ease of distinguishing the configuration file when looking at some filesystem later when it is deployed.

Note that keeping multiple access configurations within one installation might become a security risk too (if staging keeps live credentials and vice versa).

[Vinai]

Agreed, but note that by far not all Magento deployments utilize CI. In many (smaller) cases simpler deployment strategies are used.

[speedupmate]

This should apply for simpler strategies as well (and the security risks are the same). Making a "code pool" approach to local.xml like config files would follow the patterns used with other xml config files and would make distinction of environment prefixed/suffixed files more understandable for dev's with different knowledge.

There are also other naming possibilities to solve this. Naming configuration files by hostname , subfolder name (think of most used setups that are used by devs) stage_whatever_com_local.xml , www_whatever_com_local.xml. If such implementation is given a good thought before implementing then it would not need additional settings to be set (either server variables or flag files)

In my opinion the offered solution here is a bit "over thinking" as it will make one more thing to learn and coupling it with server variables makes it more abstract than needed. Different approaches are already used and existing in Magento to do the similar things (server variables, flag files for maintenance and compilation mode). In other words it does not solve the need for different config options in any way and thus won't ease the setup for those who don't use CI and does not have real value for those who use CI

[Vinai]

In years of work with different clients I have almost never experienced two identical directory structures for development andor deployment, so I believe depending on the filesystem hierarchy is not a suitable approach.
The same goes for the hostname, since most multi-website Magento instances use many hostnames, but the same database, that is local.xml file.

Magento already uses environment variables to set up the application (see MAGE_RUN_TYPE, MAGE_RUN_CODE and MAGE_IS_DEVELOPER_MODE), so building on the same approach makes sense, as it is a known paradigm for a Magento developer.

Magento 2 does introduce a more "convention-over-configuration" approach, but still is basically a configuration based framework.

For this reasons I still believe the first, environment based configuration approach is the most suitable.

The concrete implementation how an additional local.xml files are named (for example filename-prefixes, -sufixes or subdirectories) makes no real difference to me.

[mzeis]

@mage2-team:
Thanks for your response. I agree that merging of xml files should be enough for 99% of the projects. Real examples would be projects that require more complex settings than the usual webshop (which uses one database and normal one-level or two-level caches). If you want to keep the whole configuration for the shop in your Magento local.xml file, you can keep things tidy and neat. But you don't necessarily need the initially proposed extension capability.

@Vinai and @speedupmate:
I won't heave resources in the next few weeks, maybe afterwards.

I agree that this can be handled by the build process / CI but there should be the possibilty to do this in a clean way using just Magento. This will help many smaller projects which want to follow best practices and have a development project but don't have the budget or knowledge to setup the needed processes.

I prefer the approach of environment variables shown by Vinai. Magento developers know them. You can set environment variables in vhost settings, so you don't have to change .htaccess on a per-environment basis. You can define the filename so you not only can configure for different environments but also do some quick tests without editing files or starting the build process again.

The compiler gets removed by Magento 2.x so we are left with one flag file for maintenance. There are many reasons for and against using a flag file. I personally don't care too much about this and won't get into that now.

[speedupmate]

I did some additional research on this over the weekend , code pool and fallback for config files is not such a good idea after all as accidents can happen if not handled properly (imagine dev box fallback accidentally to live db and vice versa). Therefore sever variable for this can be used if properly documented however as I'm used to CI then i'll probably find no use for this.

[magento-team]

@mzeis
Ok, let's keep this discussion open, while you're coming up with a pull request.

[magento-team]

Hi @mzeis,

as 2 months have passed from your last answer, then we decided to close this ticket. The story is valuable, thus we moved it to our backlog for further implementation.

Thank you for the good proposal.

[mzeis]

Hi Magento team,
sorry I don't have time for the implementation at the moment. Thank you very much for putting the story to your backlog.

[Vinai]

I've implemented it in my branch, but haven't had the time to create any tests, so I haven't sent a pull request yet. So just FYI, if you are interested: Vinai@ca1172a

[Vinai]

For additional reference: issue from SunshineTech resolved and pull request sent: #41