Using variables in terraform backend config block

[glenjamin]

Terraform Version

v0.9.0

Affected Resource(s)

terraform backend config

Terraform Configuration Files

variable "azure_subscription_id" {
    type = "string"
    default = "74732435-e81f-4a43-bf68-ced435436edf"
}
variable "azure_tenant_id" {
    type = "string"
    default = "74732435-e81f-4a43-bf68-ced435436edf"
}
terraform {
    required_version = ">= 0.9.0"
    backend "azure" {
        resource_group_name = "stuff"
        storage_account_name = "morestuff"
        container_name = "terraform"
        key = "yetmorestuff.terraform.tfstate"
        arm_subscription_id = "${var.azure_subscription_id}"
        arm_tenant_id = "${var.azure_tenant_id}"
    }
}

Expected Behavior

Variables are used to configure the backend

Actual Behavior

Error initializing new backend:
Error configuring the backend "azure": Failed to configure remote backend "azure": Couldn't read access key from storage account: Error retrieving keys for storage account "morestuff": autorest#WithErrorUnlessStatusCode: POST https://login.microsoftonline.com/$%7Bvar.azure_tenant_id%7D/oauth2/token?api-version=1.0 failed with 400 Bad Request: StatusCode=400.

Steps to Reproduce

1. terraform apply

Important Factoids

I wanted to extract these to variables because i'm using the same values in a few places, including in the provider config where they work fine.

[darrensimio]

I am trying to do something like this; getting the same "configuration cannot contain interpolations" error. While it seems like this is being worked on, I wanted to also ask if this is the right way for me to use access and secret keys? Does it have to be placed here so that I don't have to check the access and secret keys to github

terraform {
backend "s3" {
bucket = "ops"
key = "terraform/state/ops-com"
region = "us-east-1"
encrypt = "true"
access_key = "${var.aws_access_key}"
secret_key = "${var.aws_secret_key}"
}
}

[antonosmond]

I have the same problem i.e. would love to see interpolations in the backend config. Now that we have "environments" in terraform, I was hoping to have a single config.tf with the backend configuration and use environments for my states. The problem is that I want to assume an AWS role based on the environment I'm deploying to. I can do this in "provider" blocks as the provider block allows interpolations so I can assume the relevant role for the environment I'm deploying to, however if I also rely on the role being set for the backend state management (e.g. when running terraform env select) it doesn't work. Instead I have to use the role_arn in the backend config which can't contain the interpolation I need.

[darrensimio]

I managed to get it working by using AWS profiles instead of the access keys directly. What I did though was not optimal; but in my build steps, I ran a bash script that called AWS configure that ultimately set the default access key and secret.

[wasfree]

We want to archive something similar than @antonosmond. At the moment we use multiple environments prod/stage and want to upload tfstate files to S3.

## State Backend
terraform {
  backend "s3" {
    bucket  = "mybucket"
    key     = "aws/${var.project}/${var.environment}"
    region  = "eu-central-1"
    profile = "default"
    encrypt = "true"
    lock_table = "terraform"
  }
}

In this case with above backend definition leads us to this Error:

• terraform.backend: configuration cannot contain interpolations

Now if we try to hardcode it like this:

## State Backend
terraform {
  backend "s3" {
    bucket  = "mybucket"
    key     = "aws/example/prod"
    region  = "eu-central-1"
    profile = "default"
    encrypt = "true"
    lock_table = "terraform"
  }
}

we get the following notification:

Do you want to copy only your current environment?
  The existing backend "local" supports environments and you currently are
  using more than one. The target backend "s3" doesn't support environments.
  If you continue, Terraform will offer to copy your current environment
  "prod" to the default environment in the target. Your existing environments
  in the source backend won't be modified. If you want to switch environments,
  back them up, or cancel altogether, answer "no" and Terraform will abort.

Is there a workaround for this problem at the moment, documentation for backend configuration does not cover working with environments.

Solved

seems my local test env was still running on terraform 0.9.1, after updating to latest version 0.9.2 it was working for me.

Do you want to migrate all environments to "s3"?
  Both the existing backend "local" and the target backend "s3" support
  environments. When migrating between backends, Terraform will copy all
  environments (with the same names). THIS WILL OVERWRITE any conflicting
  states in the destination.
  
  Terraform initialization doesn't currently migrate only select environments.
  If you want to migrate a select number of environments, you must manually
  pull and push those states.
  
  If you answer "yes", Terraform will migrate all states. If you answer
  "no", Terraform will abort.

[gsirvas]

Hi,
I'm trying to the the same as @NickMetz, I'm running terraform 0.9.3

$terraform version
Terraform v0.9.3

This is my code
terraform {
  backend "s3" {
    bucket = "tstbckt27" 
    key = "/${var.env}/t1/terraform.tfstate"
    region = "us-east-1"
  }
}

This is the message when I try to run terraform init

$ terraform init
Initializing the backend...
Error loading backend config: 1 error(s) occurred:

* terraform.backend: configuration cannot contain interpolations

The backend configuration is loaded by Terraform extremely early, before
the core of Terraform can be initialized. This is necessary because the backend
dictates the behavior of that core. The core is what handles interpolation
processing. Because of this, interpolations cannot be used in backend
configuration.

If you'd like to parameterize backend configuration, we recommend using
partial configuration with the "-backend-config" flag to "terraform init".

Is this expected behaviour on v0.9.3?

Are there any workarounds for this?

[umeat]

In case it's helpful to anyone, the way I get around this is as follows:

terraform {
  backend "s3" {}
}

data "terraform_remote_state" "state" {
  backend = "s3"
  config {
    bucket     = "${var.tf_state_bucket}"
    lock_table = "${var.tf_state_table}"
    region     = "${var.region}"
    key        = "${var.application}/${var.environment}"
  }
}

All of the relevant variables are exported at the deployment pipeline level for me, so it's easy to init with the correct information for each environment.

terraform init \ 
     -backend-config "bucket=$TF_VAR_tf_state_bucket" \ 
     -backend-config "lock_table=$TF_VAR_tf_state_table" \ 
     -backend-config "region=$TF_VAR_region" \ 
     -backend-config "key=$TF_VAR_application/$TF_VAR_environment"

I don't find this ideal, but at least I can easily switch between environments and create new environments without having to edit any terraform.

[wasfree]

@gsirvas @umeat To archive multiple environment with the same backend configuration it is not necessary to use variables/interpolation .It is expected that is not possible to use variables/interpolation in backend configuration see comment from @christofferh.

Just write it like this:

terraform {
  backend "s3" {
    bucket = "tstbckt27" 
    key = "project/terraform/terraform.tfstate"
    region = "us-east-1"
  }
}

Terraform will split and store environment state files in a path like this:
env:/${var.env}/project/terraform/terraform.tfstate

[umeat]

@NickMetz it's trying to do multiple environments with multiple backend buckets, not a single backend. You can't specify a different backend bucket in terraform environments. In my example you could still use terraform environments to prefix the state file object name, but you get to specify different buckets for the backend.

Perhaps it's better to just give accross account access to the user / role which is being used to deploy your terraform. Deploying your terraform to a different account, but using the same backend bucket. Though it's fairly reasonable to want to store the state of an environment in the same account that it's deployed to.

[wasfree]

@umeat in that case you are right, it is not possible at the moment to use different backends for each environment. It would be more comfortable to have a backend mapping for all environments what is not implemented yet.