Detect ENV vars in YAML stack config and set them for command execution. Make workspace_key_prefix config DRY

[aknysh]

what

• Detect ENV vars in YAML stack config and set them for command execution
• Make workspace_key_prefix config DRY
• Don't delete the generated terraform varfiles after each command
• Update tests
• General cleanup

why

• Detect ENV vars in YAML stack config and set them for command execution - allow specifying ENV vars in YAML config files. For each component in a stack, ENV vars are deep-merged in this order: global ENV vars, terraform section ENV vars, base component ENV vars, component ENV vars. The when commands are executed, print the final set of ENV vars and set them in the executing shell. This will allow controlling the tools behavior (e.g. Terraform, helmfile) by YAML configs

Command info:
Terraform binary: /usr/local/bin/terraform
Terraform command: plan
Arguments and flags: []
Component: test/test-component-override
Base component: test/test-component
Stack: tenant1/ue2/dev
Working dir: ./examples/complete/components/terraform/test/test-component

Using ENV vars:
TEST_ENV_VAR3=val3-override
TEST_ENV_VAR1=val1-override
TEST_ENV_VAR2=val2
TEST_ENV_VAR4=val4

Executing command:
/usr/local/bin/terraform workspace select tenant1-ue2-dev-test-component-override

Executing command:
/usr/local/bin/terraform plan -var-file tenant1-ue2-dev-test-component-override.terraform.tfvars.json -out tenant1-ue2-dev-test-component-override.planfile

• Check if backend section has workspace_key_prefix for s3 backend type. If it does not, use the component name instead. It also propagates to remote_state_backend section of s3 type. This will allow to have components catalog files DRY without repeating the same config (which now can be generated automatically by the component names). Components folders are supported and taken into account in the generated workspace_key_prefix. This can be overridden as before per component.

This config

components:
  terraform:
    "test/test-component":
      backend:
        s3:
          workspace_key_prefix: test-test-component
      settings:
        spacelift:
          workspace_enabled: true

is now the same as this

components:
  terraform:
    "test/test-component":
      settings:
        spacelift:
          workspace_enabled: true

and produces the same result (confirmed by the updated tests)

backend:
  workspace_key_prefix: test-test-component
backend_type: s3
remote_state_backend:
  workspace_key_prefix: test-test-component
remote_state_backend_type: s3

• Don't delete the generated terraform varfiles after each command - sometimes it's useful to be able to run atmos terraform plan and then use the generated varfile and planfile in other terraform commands. The varfile and planfile can be deleted by now supported command tmos terraform clean <component> -s <stack>