Backend interactive prompting fails with 1 required of 2 possible

[mbrancato]

Terraform Version

Terraform v0.11.7

Terraform Configuration Files

terraform {
  backend "azurerm" {
    storage_account_name = "abcd1234"
    container_name       = "tfstate"
    key                  = "prod.terraform.tfstate"
  }
}

Expected Behavior

The azurerm backend requires either an access_key OR 5 values related to the Azure subscription. The backend should prompt for the access_key via the interactive init process for partial backend configurations.

Actual Behavior

Terraform fails with the following error:

Initializing the backend...
Error configuring the backend "azurerm": resource_group_name and credentials must be provided when access_key is absent

Please update the configuration in your Terraform files to fix this error
then run this command again.

Steps to Reproduce

1. terraform init

Additional Context

One option to force interactive prompting here could be to force the use of access_key by specifying -backend-option="access_key=". This does not currently work.

References

• https://www.terraform.io/docs/backends/config.html

[jmgomezcrespo]

I have same problem. I tried 3 documentation ways but I always have this error

[apparentlymart]

Hi all,

I expect this is a consequence of how this provider is designed. Since there are two possible ways to provide credentials here, all of them are marked as "optional" and then the validation logic within the backend code itself checks to see if a valid combination is used.

The interactive mode currently only prompts for required arguments, with the intention of not overwhelming the user with the different esoteric settings that many backends have.

In general we offer the interactive prompts here as a best-effort for helping users who are just getting started. It's not intended for regular use, and has a number of limitations. For regular use of Azure in particular, a common approach is to authenticate via the Azure CLI, which then configures both the backend and the provider simultaneously, along with any other software that you use to interact with the Azure API. Each backend has a similar native, non-interactive authentication mechanism which is what most users are expected to use once they get past initial experimentation.

[jmgomezcrespo]

Finally, I checked Attributes Reference in resource "azurerm_storage_account" where showed primary_access_key element. This resource is element "access_key" in backend configuration

https://www.terraform.io/docs/providers/azurerm/r/storage_account.html#primary_access_key

[mbrancato]

Hi @apparentlymart. I have some concerns with this approach.

For regular use of Azure in particular, a common approach is to authenticate via the Azure CLI, which then configures both the backend and the provider simultaneously, along with any other software that you use to interact with the Azure API.

This assumes the user has rights via their native AAD account permissions to write to the storage account. If this is not permitted in the IAM config, an access key must be used. Obviously, that key can be written to the .tf file, but this would create security concerns and ruins our commits to git. Keys have some advantages in that a storage admin can grant a key to a user OOB and then regenerate that key once the user is finished. IAM changes tend to take some time to take effect.

In my specific use case, our separation of duties pushes us to use access keys in this fashion. We also take care to separate where our state files are located from where we are creating resources (separate resource groups).

Also, just to clarify, am I correct in saying the backend types are not implemented by "providers"? Just trying to not confuse terminology.

[apparentlymart]

Hi @mbrancato,

If the "ambient" credentials available for all Azure clients are not acceptable, then the next most preferred option is to set environment variables. In this case, I believe the environment variable in question is ARM_ACCESS_KEY.

My reference to "providers" was indeed an error. A number of our backends re-use the same authentication code as their corresponding provider, and thus the behavior for the two is always the same. However, I forgot that for Azure in particular that is not true, and the azurerm backend uses the Azure Blob Storage SDK directly, with its own authentication code.

[codyja]

I'm encountering this as well. My normal workflow is login via "az login" (writes to my ~/.azure directory). Then putting the following in my .tf, results in the same error.

terraform{
backend "azurerm" {
storage_account_name = "account"
container_name = "container"
key = "something/terraform.tfstate"
resource_group_name = "rg"
}
}

[jmgomezcrespo]

@codyja You need access_key value, that is Storage primary key from your storage account.

You obtain it from ${azurerm_storage_account.main.primary_access_key}

storage_account_name = "mainterraform"
container_name = "tfstate"
key = "demo.terraform.tfstate"
**access_key = "W3rv.....TGHk"**

Regards

[mbrancato]

I can confirm that the ARM_ACCESS_KEY does appear to work for this. It still isn't backend prompting and requires the user to set this value prior to running Terraform.

@jmgomezcrespo Backends cannot access data sources as you suggest because of when they are initialized.

[morsh]

Added ARM_ACCESS_KEY with the value of the storage access key provided for tf, and terraform init started working.

[1arrow]

Any update on this ticket, we have multiple state files for our environment and cant use same ARM_ACCESS_KEY for all of them :(

[diroussel]

Yes I would like to just use the ARM_CLIENT_SECRET to access multiple storage accounts. Managing multiple access keys is tricky.
I can't authenticate via az cli, as the page linked to above says"

Authenticating via the Azure CLI is only supported when using a User Account. If you're using a Service Principal (for example via az login --service-principal) you should instead authenticate via the Service Principal directly (either using a Client Secret or a Client Certificate).

[ReddyMalathi]

@codyja You need access_key value, that is Storage primary key from your storage account.

You obtain it from ${azurerm_storage_account.main.primary_access_key}

storage_account_name = "mainterraform"
container_name = "tfstate"
key = "demo.terraform.tfstate"
**access_key = "W3rv.....TGHk"**

Regards

@codyja You mean to say like first create a storage account then copy the storage account keys and then pass the variable in access_key?

[aars]

To use arm_client_id and arm_client_secret instead of access_key you need to use tenant_id as well.

[adamday2]

Is it intended that the access_key is always required with the use of az login? In our use-case, the account has permission to that storage account, but the access_key appears to still be required.

This does not seems to match the documentation for backends:
https://www.terraform.io/docs/backends/types/azurerm.html

Also, the prescribed configuration for MSIs appears incorrect:

terraform {
  backend "azurerm" {
    storage_account_name = "abcd1234"
    container_name       = "tfstate"
    key                  = "prod.terraform.tfstate"
    use_msi              = true
    subscription_id  = "00000000-0000-0000-0000-000000000000"
    tenant_id        = "00000000-0000-0000-0000-000000000000"
  }
}

Results in:

Error configuring the backend "azurerm": 3 error(s) occurred:
* : invalid or unknown key: subscription_id
* : invalid or unknown key: tenant_id
* : invalid or unknown key: use_msi

[Korijn]

I am experiencing exactly the same issue as @adamday2 ; the example in the docs does not work. Specifically this one:

When authenticating using the Azure CLI or a Service Principal:

terraform {
  backend "azurerm" {
    storage_account_name = "abcd1234"
    container_name       = "tfstate"
    key                  = "prod.terraform.tfstate"
  }
}

I'm authenticated with the Azure CLI, and with this configuration I get the error:

Error configuring the backend "azurerm": resource_group_name and credentials must be provided when access_key is absent

Terraform version 0.11.13.

Update

I have taken a look at the source code, both the azurerm state backend and the azurerm provider rely on the same authentication helper found here: https://github.com/hashicorp/go-azure-helpers/blob/master/authentication/builder.go

It supports the Azure CLI token just fine actually. This is proven by the fact that the azurerm provider supports it without issue.

I believe the error is the final if-statement in configure in https://github.com/hashicorp/terraform/blob/master/backend/remote-state/azure/backend.go

It is just the wrong condition to validate!

[darrenfurr]

Same issue as @adamday2 with Terraform 0.11.13. I am unable to use the Azure MSI. Tried adding the access_key no luck.

[sampcoug]

I am also running into a similar issue:

terraform init

Initializing the backend...

Error configuring the backend "azurerm": resource_group_name and credentials must be provided when access_key is absent

Please update the configuration in your Terraform files to fix this error
then run this command again.

I have provided the ACCESS_KEY as an environment variable:

export ARM_ACCESS_KEY=<key>

This is how my Terraform code looks like:

terraform {
  backend "azurerm" {
    storage_account_name  = "storageaccount"
    container_name        = "container"
    key                   = "terraform.tfstate"
    resource_group_name  = "resourceGroup"
  }
}

Why is the code not sourcing Access Key from the Environment Variable? Any idea folks?

[sampcoug]

I am also running into a similar issue:

terraform init

Initializing the backend...

Error configuring the backend "azurerm": resource_group_name and credentials must be provided when access_key is absent

Please update the configuration in your Terraform files to fix this error
then run this command again.

I have provided the ACCESS_KEY as an environment variable:

export ARM_ACCESS_KEY=<key>

This is how my Terraform code looks like:

terraform {
  backend "azurerm" {
    storage_account_name  = "storageaccount"
    container_name        = "container"
    key                   = "terraform.tfstate"
    resource_group_name  = "resourceGroup"
  }
}

Why is the code not sourcing Access Key from the Environment Variable? Any idea folks?

Nevermind, It started working when I manually ran the export variable command instead of in a .sh script. May be the .sh script wasn't quite working, not sure. I never received any error though.

[adamday2]

Nevermind, It started working when I manually ran the export variable command instead of in a .sh script. May be the .sh script wasn't quite working, not sure. I never received any error though.

If you set an env from a .sh script, and then ran terraform outside of the script, you have to run it with . script.sh otherwise your environment scope won't be maintained after the script terminates.

[dalareo]

I had a similar problem due to required permissions not present: access to storage account with credentials supplied... just in case someone is struggling without the permissions...

[darrenfurr]

I was able to get MSI to work with terraform 0.11.13 in our CD pipeline by setting the following environment variables:
ARM_USE_MSI
ARM_ACCESS_KEY

[JustinGrote]

Here's the commit for the MSI code. According to the commit tag, it's only present in 0.12.

c928962

@darrenfurr the reason yours probably worked is that the ARM_USE_MSI was ignored and ARM_ACCESS_KEY was used for your backend. Can you verify you were actually able to terraform apply into the environment using MSI? Do the resources show as created with the MSI account?

[docunid]

Tried and tested on v0.12.0

terraform {
  backend "azurerm" {
    container_name          = "<...>"
    key                               = "<...>"
    use_msi                        = true
    resource_group_name = "<...>"
  }
}

PowerShell:
$env:ARM_SUBSCRIPTION_ID = (Get-AzContext).Subscription.Id
terraform init -backend-config "storage_account_name=$(storage_account_name)" -input=false

Strange imho:

• using "resource_group_name" in backend.tf because according to the docs that is for service principal authentication not msi
• using $env:ARM_SUBSCRIPTION_ID but is seems necessary