Add possibility of prod-specific config.yml file override

[geerlingguy]

This issue is an offshoot of #84.

Basically, the instructions right now tell you to use one config.yml that's modified for production (with more secure variable settings)... but if you do that, then you can't have the same config.yml be useful for your local development environment in the same Drupal VM instance.

It's not the end of the world to have to clone two copies of Drupal VM—one for local, one for prod—but it would be more clean if we could allow some sort of prod.config.yml, using the same kind of idea @oxyc mentioned elsewhere about having a local.config.yml override.

So the hierarchy of config files would be prod.config.yml > local.config.yml > config.yml. Something like that. I haven't quite worked out the specifics, and maybe there's a better way to do this architecturally, but I wanted to split this issue off of #84 so I could at least get that completed.

[oxyc]

How about adding an internal variable to vars/main.yml

_include_configs:
  - config.yml
  - local.config.yml

Iterating over it in the Vagrantfile and checking if the files exist before merging them into vconfig. This would make it possible override it and even reference the configs in the example directory. To pass these on to ansible it would rely on the extra_vars issue being fixed in Vagrant 1.8.2 though.

vconfig = YAML::load_file("#{dir}/provisioning/vars/main.yml")
config_files = vconfig._include_configs
while config_files.length do
  config_file = config_files.shift
  if File.exist?(config_file)
    config = YAML::load_file(config_file) if File.exist?(config_file)
    # take into account new config files
    config_files.concat config._include_configs if defined? config._include_configs
  end
end

[geerlingguy]

True... but we also need to make sure the configs are set in Ansible itself correctly, and in the prod ansible-playbook commands. I don't want to use Vagrant to deploy to DigitalOcean (or other environments), but rather use Ansible directly.

So we would need support inside the Vagrantfile, but we could then use the new dynamic include functionality in Ansible 2.x to see if the file exists, include it if so, then see if the next one exists, include it if so, etc.

Because of that, I'm thinking of waiting until at least 2.0.1 is out, maybe a little further (for more stabilization in 2.x), before making this change.

[oxyc]

Ah, true.

[oxyc]

Taking into account that we need production specific variables like you say, my approach won't do much good. So more brainstorming.

Another idea that we discussed somewhere before was ansible tags. We could have a production tag that toggles ansible trying to include either prod.config.yml or vm.config.yml. At least for me there's a case for local.config.yml always be included.

In the Vagrantfile we could do add a skip_tags: ['production'] ref.

Downside with this approach is vagrant plugins such as AWS would inherit this skip_tags.

[colinappnovation]

Sorry I am very new to using drupalvm but needed the ability to do overrides, so ended up editing the Vagrantfile as per below with inspiration from @oxyc.

Thought would share for consideration. It is not perfect, a strategy for merge! call requires duplicating for instance apache vhosts otherwise you lose them as last merge wins scenerio.

# Use config.yml for basic VM configuration.
require 'yaml'
dir = File.dirname(File.expand_path(__FILE__))

configs = YAML::load_file("#{dir}/provisioning/vars/main.yml")
vconfig = ''
configs['_include_configs'].each do |c|  
  if File.exist?("#{dir}/#{c}")
    conf = YAML::load_file("#{dir}/#{c}") if File.exist?("#{dir}/#{c}")
    # take into account new config files
    vconfig = conf if vconfig.empty?

    vconfig.merge!(conf) if !vconfig.empty?

  end
end

# Replace jinja variables in config.
vconfig = walk(vconfig) do |value|
  while value.is_a?(String) && value.match(/{{ .* }}/)
    value = value.gsub(/{{ (.*?) }}/) { |match| match = vconfig[$1] }
  end
  value
end

[oxyc]

I'm fine with having to duplicate lists. Without that we wouldn't be able to remove defaults. Unfortunately this doesn't work for direct calls to ansible which is a requirement. It needs to be done in the playbook instead.

[heddn]

I'm bikeshedding here, but is there a vagrant / ansible standard that would lean towards calling the config files prod.config.yml, local.config.yml, etc? It seems like calling the files config.local.yml, config.prod.yml, etc is more consistent with how I've seen drupal and vagrant operate.

[oxyc]

There's in fact talk of renaming it in core https://www.drupal.org/node/2419213. For me it's just old habits though.

[geerlingguy]

I like [env].filename.ext personally, because that is simpler for renaming purposes, and also follows the example.filename.ext convention I use elsewhere.

[geerlingguy]

Let's punt this to a later time. We now have support for a local.config.yml, and since fewer people are using Drupal VM to build prod environments right now, we can hold off optimizing that use case until it becomes more of a pain point.

[oxyc]

A dirty trick to support a separate production config with #378 is:

Create a separate subdirectory with a symlinked config.yml and add a local.config.yml with your production configurations.

Run

ansible-playbook -i examples/prod/inventory provisioning/playbook.yml
  --sudo
  --ask-sudo-pass
  --extra-vars="config_dir=/Users/foo/Sites/prod-config"

[geerlingguy]

@oxyc's merged fix allows use of a prod.config.yml (or any other environment as set by an environment variable).

The next step to resolve this is a little extra documentation, maybe in /docs or maybe in /examples/prod, or maybe a new docs page that references the prod example.

[geerlingguy]

This was done by @oxyc's latest commits. Closing the issue out now. Thanks!

[oxyc]

There are still some docs to be taken care of.

[geerlingguy]

D'oh!

[geerlingguy]

@oxyc - Fixed now? Thanks for the help.

[oxyc]

I think so.