Terraform destroyed the resources instead of updating them

[agrcrobles]

Hi there,

TLDR; Terraform destroyed all the resources and tried to recreate them, instead it should have update them.

...
2019-01-07T12:04:20.0233989Z azurerm_application_insights.etest: Destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...components/aa-etest-prod-AppInsights)
2019-01-07T12:04:20.0239372Z azurerm_cdn_endpoint.etest: Destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...prod-CdnProfile/endpoints/etest-prod)
...

Terraform Version

0.11.11

Expected Behavior

Terraform should have updated the resources

Actual Behavior

Terraform destroyed the resources and attemped to recreate them.

Steps to Reproduce

terraform" init -backend-config="arm_subscription_id=xxxxxx-7ec3-42a0-b413-xxxxxxxxxx" -backend-config="arm_tenant_id=***" -backend-config="arm_client_id=***" -backend-config="arm_client_secret=***" -input=false -no-color

terraform" apply -auto-approve -input=false -no-color

Additional Context

Our backend.tf is configured this way and the tfstate file is saved in our Azure storage as it is suggested

terraform {
  backend "azurerm" {
    resource_group_name  = "#{StateResourceGroupName}#"
    storage_account_name = "#{StateStorageAccount}#"
    container_name       = "tfstate"
    key                  = "terraform.tfstate"
  }
}

Debug Output ( Full log )

2019-01-07T12:03:19.1699614Z ##[section]Starting: Terraform apply -auto-approve
2019-01-07T12:03:19.1704298Z ==============================================================================
2019-01-07T12:03:19.1704389Z Task         : Run Terraform
2019-01-07T12:03:19.1704458Z Description  : Run a Terraform on the build agent
2019-01-07T12:03:19.1704510Z Version      : 2.1.0
2019-01-07T12:03:19.1704576Z Author       : Peter Groenewegen - Xpirit
2019-01-07T12:03:19.1704641Z Help         : [More Information](https://pgroene.wordpress.com/2016/06/14/getting-started-with-terraform-on-windows-and-azure/)
2019-01-07T12:03:19.1704715Z ==============================================================================
2019-01-07T12:03:21.9393378Z ##[command]Import-Module -Name C:\Program Files\WindowsPowerShell\Modules\AzureRM\2.1.0\AzureRM.psd1 -Global
2019-01-07T12:03:31.5844819Z ##[warning]The names of some imported commands from the module 'AzureRM.Websites' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-Verb.
2019-01-07T12:03:31.5968257Z ##[warning]The names of some imported commands from the module 'AzureRM' include unapproved verbs that might make them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the Verbose parameter. For a list of approved verbs, type Get-Verb.
2019-01-07T12:03:31.6632465Z ##[command]Import-Module -Name C:\Program Files\WindowsPowerShell\Modules\AzureRM.profile\2.1.0\AzureRM.profile.psd1 -Global
2019-01-07T12:03:31.6992116Z ##[command]Add-AzureRMAccount -ServicePrincipal -Tenant *** -Credential System.Management.Automation.PSCredential
2019-01-07T12:03:33.2039094Z ##[command]Select-AzureRMSubscription -SubscriptionId my-subscriptionre -TenantId ***
2019-01-07T12:03:33.3284952Z Resource group is:  MyResourceGroup
2019-01-07T12:03:33.3292159Z Storage account is: teststorage
2019-01-07T12:03:33.3299314Z Container name is:  tfstate
2019-01-07T12:03:36.5761127Z 
2019-01-07T12:03:36.5761551Z 
2019-01-07T12:03:36.5762766Z CloudBlobContainer : Microsoft.WindowsAzure.Storage.Blob.CloudBlobContainer
2019-01-07T12:03:36.5763019Z Permission         : Microsoft.WindowsAzure.Storage.Blob.BlobContainerPermissions
2019-01-07T12:03:36.5763641Z PublicAccess       : Off
2019-01-07T12:03:36.5763846Z LastModified       : 10/25/2018 2:58:45 PM +00:00
2019-01-07T12:03:36.5764029Z ContinuationToken  : 
2019-01-07T12:03:36.5764206Z Context            : Microsoft.WindowsAzure.Commands.Common.Storage.LazyAzureStorageContext
2019-01-07T12:03:36.5764449Z Name               : tfstate
2019-01-07T12:03:36.5764591Z 
2019-01-07T12:04:00.0090350Z PSPath            : Microsoft.PowerShell.Core\FileSystem::C:\terraform-download
2019-01-07T12:04:00.0090632Z PSParentPath      : Microsoft.PowerShell.Core\FileSystem::C:\
2019-01-07T12:04:00.0090989Z PSChildName       : terraform-download
2019-01-07T12:04:00.0091108Z PSDrive           : C
2019-01-07T12:04:00.0091374Z PSProvider        : Microsoft.PowerShell.Core\FileSystem
2019-01-07T12:04:00.0091712Z PSIsContainer     : True
2019-01-07T12:04:00.0092067Z Name              : terraform-download
2019-01-07T12:04:00.0093098Z FullName          : C:\terraform-download
2019-01-07T12:04:00.0093678Z Parent            : 
2019-01-07T12:04:00.0094538Z Exists            : True
2019-01-07T12:04:00.0094864Z Root              : C:\
2019-01-07T12:04:00.0095058Z Extension         : 
2019-01-07T12:04:00.0095386Z CreationTime      : 1/7/2019 12:03:59 PM
2019-01-07T12:04:00.0095748Z CreationTimeUtc   : 1/7/2019 12:03:59 PM
2019-01-07T12:04:00.0096298Z LastAccessTime    : 1/7/2019 12:03:59 PM
2019-01-07T12:04:00.0097094Z LastAccessTimeUtc : 1/7/2019 12:03:59 PM
2019-01-07T12:04:00.0097517Z LastWriteTime     : 1/7/2019 12:03:59 PM
2019-01-07T12:04:00.0098158Z LastWriteTimeUtc  : 1/7/2019 12:03:59 PM
2019-01-07T12:04:00.0098818Z Attributes        : Directory
2019-01-07T12:04:00.0100633Z Mode              : d-----
2019-01-07T12:04:00.0100695Z BaseName          : terraform-download
2019-01-07T12:04:00.0100809Z Target            : {}
2019-01-07T12:04:00.0101047Z LinkType          : 
2019-01-07T12:04:00.0101221Z 
2019-01-07T12:04:01.0321451Z Terraform version:
2019-01-07T12:04:02.8026679Z Terraform v0.11.11
2019-01-07T12:04:02.8026857Z 
2019-01-07T12:04:03.0875271Z ##[command]"terraform" init -backend-config="arm_subscription_id=xxxxxx-7ec3-42a0-b413-54516796a473" -backend-config="arm_tenant_id=***" -backend-config="arm_client_id=***" -backend-config="arm_client_secret=***" -input=false -no-color
2019-01-07T12:04:03.1777195Z 
2019-01-07T12:04:03.1778178Z Initializing the backend...
2019-01-07T12:04:04.5556438Z 
2019-01-07T12:04:04.5557646Z Successfully configured the backend "azurerm"! Terraform will automatically
2019-01-07T12:04:04.5557917Z use this backend unless the backend configuration changes.
2019-01-07T12:04:05.2487383Z 
2019-01-07T12:04:05.2487713Z Initializing provider plugins...
2019-01-07T12:04:05.2487832Z - Checking for available provider plugins on https://releases.hashicorp.com...
2019-01-07T12:04:05.3262922Z - Downloading plugin for provider "azurerm" (1.20.0)...
2019-01-07T12:04:06.5867568Z 
2019-01-07T12:04:06.5869463Z The following providers do not have any version constraints in configuration,
2019-01-07T12:04:06.5869665Z so the latest version was installed.
2019-01-07T12:04:06.5870199Z 
2019-01-07T12:04:06.5870323Z To prevent automatic upgrades to new major versions that may contain breaking
2019-01-07T12:04:06.5870415Z changes, it is recommended to add version = "..." constraints to the
2019-01-07T12:04:06.5870516Z corresponding provider blocks in configuration, with the constraint strings
2019-01-07T12:04:06.5870968Z suggested below.
2019-01-07T12:04:06.5871035Z 
2019-01-07T12:04:06.5871449Z * provider.azurerm: version = "~> 1.20"
2019-01-07T12:04:06.5871570Z 
2019-01-07T12:04:06.5872054Z Terraform has been successfully initialized!
2019-01-07T12:04:06.5872144Z 
2019-01-07T12:04:06.5872677Z You may now begin working with Terraform. Try running "terraform plan" to see
2019-01-07T12:04:06.5872791Z any changes that are required for your infrastructure. All Terraform commands
2019-01-07T12:04:06.5873357Z should now work.
2019-01-07T12:04:06.5875839Z 
2019-01-07T12:04:06.5876124Z If you ever set or change modules or backend configuration for Terraform,
2019-01-07T12:04:06.5876336Z rerun this command to reinitialize your working directory. If you forget, other
2019-01-07T12:04:06.5876509Z commands will detect it and remind you to do so if necessary.
2019-01-07T12:04:06.7175245Z ##[command]"terraform" apply -auto-approve -input=false -no-color
2019-01-07T12:04:08.1724598Z Acquiring state lock. This may take a few moments...
2019-01-07T12:04:11.4503822Z azurerm_application_insights.etest: Refreshing state... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...components/aa-etest-prod-AppInsights)
2019-01-07T12:04:11.4519172Z azurerm_resource_group.etest: Refreshing state... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...96a473/resourceGroups/E-test-Prod-RG)
2019-01-07T12:04:11.4530073Z azurerm_public_ip.etest: Refreshing state... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-.../publicIPAddresses/aa-etest-prod-pip)
2019-01-07T12:04:11.4537186Z azurerm_app_service_plan.functionapp: Refreshing state... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...a-etest-prod-FunctionApp-ServicePlan)
2019-01-07T12:04:11.4556777Z azurerm_cdn_profile.etest: Refreshing state... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...dn/profiles/aa-etest-prod-CdnProfile)
2019-01-07T12:04:11.4568047Z azurerm_virtual_network.vnet: Refreshing state... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...k/virtualNetworks/aa-etest-prod-vnet)
2019-01-07T12:04:11.4605144Z azurerm_storage_account.etest: Refreshing state... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...e/storageAccounts/aaetestprodstorage)
2019-01-07T12:04:11.4605310Z azurerm_redis_cache.etest: Refreshing state... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...soft.Cache/Redis/aa-etest-prod-redis)
2019-01-07T12:04:11.8760953Z azurerm_function_app.functionapp: Refreshing state... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-....Web/sites/aa-etest-prod-FunctionApp)
2019-01-07T12:04:11.8768876Z azurerm_cdn_endpoint.etest: Refreshing state... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...prod-CdnProfile/endpoints/etest-prod)
2019-01-07T12:04:11.8769027Z azurerm_storage_container.etest: Refreshing state... (ID: https://aaetestprodstorage.blob.core.windows.net/web)
2019-01-07T12:04:11.9332971Z azurerm_subnet.default: Refreshing state... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...od-vnet/subnets/aa-etest-prod-subnet)
2019-01-07T12:04:13.9815217Z azurerm_application_gateway.etest: Refreshing state... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...cationGateways/aa-etest-prod-gateway)
2019-01-07T12:04:20.0233989Z azurerm_application_insights.etest: Destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...components/aa-etest-prod-AppInsights)
2019-01-07T12:04:20.0239372Z azurerm_cdn_endpoint.etest: Destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...prod-CdnProfile/endpoints/etest-prod)
2019-01-07T12:04:20.0241885Z azurerm_application_gateway.etest: Destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...cationGateways/aa-etest-prod-gateway)
2019-01-07T12:04:20.0242071Z azurerm_redis_cache.etest: Destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...soft.Cache/Redis/aa-etest-prod-redis)
2019-01-07T12:04:20.0248982Z azurerm_resource_group.etest: Destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...96a473/resourceGroups/E-test-Prod-RG)
2019-01-07T12:04:22.0922366Z azurerm_application_insights.etest: Destruction complete after 2s
2019-01-07T12:04:30.0254214Z azurerm_resource_group.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...96a473/resourceGroups/E-test-Prod-RG, 10s elapsed)
2019-01-07T12:04:30.0254479Z azurerm_redis_cache.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...soft.Cache/Redis/aa-etest-prod-redis, 10s elapsed)
2019-01-07T12:04:30.0254625Z azurerm_application_gateway.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...cationGateways/aa-etest-prod-gateway, 10s elapsed)
2019-01-07T12:04:30.0254732Z azurerm_cdn_endpoint.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...prod-CdnProfile/endpoints/etest-prod, 10s elapsed)
2019-01-07T12:04:40.0267282Z azurerm_redis_cache.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...soft.Cache/Redis/aa-etest-prod-redis, 20s elapsed)
2019-01-07T12:04:40.0267886Z azurerm_application_gateway.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...cationGateways/aa-etest-prod-gateway, 20s elapsed)
2019-01-07T12:04:40.0269075Z azurerm_resource_group.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...96a473/resourceGroups/E-test-Prod-RG, 20s elapsed)
2019-01-07T12:04:40.0270557Z azurerm_cdn_endpoint.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...prod-CdnProfile/endpoints/etest-prod, 20s elapsed)
2019-01-07T12:04:50.0268955Z azurerm_redis_cache.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...soft.Cache/Redis/aa-etest-prod-redis, 30s elapsed)
2019-01-07T12:04:50.0269470Z azurerm_resource_group.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...96a473/resourceGroups/E-test-Prod-RG, 30s elapsed)
2019-01-07T12:04:50.0270638Z azurerm_cdn_endpoint.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...prod-CdnProfile/endpoints/etest-prod, 30s elapsed)
2019-01-07T12:04:50.0271936Z azurerm_application_gateway.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...cationGateways/aa-etest-prod-gateway, 30s elapsed)
2019-01-07T12:05:00.0278494Z azurerm_redis_cache.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...soft.Cache/Redis/aa-etest-prod-redis, 40s elapsed)
2019-01-07T12:05:00.0279422Z azurerm_resource_group.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...96a473/resourceGroups/E-test-Prod-RG, 40s elapsed)
2019-01-07T12:05:00.0280520Z azurerm_cdn_endpoint.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...prod-CdnProfile/endpoints/etest-prod, 40s elapsed)
2019-01-07T12:05:00.0282098Z azurerm_application_gateway.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...cationGateways/aa-etest-prod-gateway, 40s elapsed)
2019-01-07T12:05:10.0320521Z azurerm_redis_cache.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...soft.Cache/Redis/aa-etest-prod-redis, 50s elapsed)
2019-01-07T12:05:10.0321212Z azurerm_cdn_endpoint.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...prod-CdnProfile/endpoints/etest-prod, 50s elapsed)
2019-01-07T12:05:10.0323380Z azurerm_resource_group.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...96a473/resourceGroups/E-test-Prod-RG, 50s elapsed)
2019-01-07T12:05:20.0320351Z azurerm_cdn_endpoint.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...prod-CdnProfile/endpoints/etest-prod, 1m0s elapsed)
2019-01-07T12:05:20.0320961Z azurerm_redis_cache.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...soft.Cache/Redis/aa-etest-prod-redis, 1m0s elapsed)
2019-01-07T12:05:20.0322570Z azurerm_resource_group.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...96a473/resourceGroups/E-test-Prod-RG, 1m0s elapsed)
2019-01-07T12:05:21.2117737Z azurerm_cdn_endpoint.etest: Destruction complete after 1m1s
2019-01-07T12:05:21.2132569Z azurerm_cdn_profile.etest: Destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...dn/profiles/aa-etest-prod-CdnProfile)
2019-01-07T12:05:30.0329243Z azurerm_redis_cache.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...soft.Cache/Redis/aa-etest-prod-redis, 1m10s elapsed)
2019-01-07T12:05:30.0330584Z azurerm_resource_group.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...96a473/resourceGroups/E-test-Prod-RG, 1m10s elapsed)
2019-01-07T12:05:40.0335890Z azurerm_resource_group.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...96a473/resourceGroups/E-test-Prod-RG, 1m20s elapsed)
2019-01-07T12:05:40.0336473Z azurerm_redis_cache.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...soft.Cache/Redis/aa-etest-prod-redis, 1m20s elapsed)
2019-01-07T12:05:50.0337413Z azurerm_resource_group.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...96a473/resourceGroups/E-test-Prod-RG, 1m30s elapsed)
2019-01-07T12:05:50.0338002Z azurerm_redis_cache.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...soft.Cache/Redis/aa-etest-prod-redis, 1m30s elapsed)
2019-01-07T12:06:00.0340733Z azurerm_resource_group.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...96a473/resourceGroups/E-test-Prod-RG, 1m40s elapsed)
2019-01-07T12:06:00.0341467Z azurerm_redis_cache.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...soft.Cache/Redis/aa-etest-prod-redis, 1m40s elapsed)
2019-01-07T12:06:10.0348853Z azurerm_resource_group.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...96a473/resourceGroups/E-test-Prod-RG, 1m50s elapsed)
2019-01-07T12:06:10.0349409Z azurerm_redis_cache.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...soft.Cache/Redis/aa-etest-prod-redis, 1m50s elapsed)
2019-01-07T12:06:20.0382987Z azurerm_resource_group.etest: Still destroying... (ID: /subscriptions/xxxxxx-7ec3-42a0-b413-...96a473/resourceGroups/E-test-Prod-RG, 2m0s elapsed)

References

None that I am aware of.

Please let me know if there is need for more information to follow up this one

Thanks in advance.

[abiydv]

What did the "terraform plan" output say? That it was going to "update in place" and then went on to "destroy and recreate" the resources? Curious because I don't see any output for terraform plan in the logs you pasted above.

In my experience, it does call out specifically in case there is something it needs to destroy and recreate. Also denotes the change with a "-/+" symbol.

[agrcrobles]

We do not have a terraform plan in our CI script, why might terraform destroy the resources instead of updating them?

[abiydv]

It depends on the resource you are trying to modify/update. Some resources don't allow "updates in place". Ex - In AWS, you can update an instance's security group in place, but if you try to update the AMI for an instance, it will destroy and recreate the instance.

[agrcrobles]

Thanks @abiydv for your reply,
what about resource groups in azure? do you know where I can get that information/documentation?

[abiydv]

I think the easiest way to check would be to run terraform plan. Other than that, I guess Azure's docs might help w.r.t what parameters can be changed once the resource has been created. The parameters which can't be changed after creation would lead terraform to destroy & create.

[mildwonkey]

Hi @agrcrobles, I'm sorry to hear you've experienced this. As @abiydv indicated, you would need to run terraform plan to see what changes terraform plans on making, and why.

You can find the provider documentation here. You will find that there are several resources (in all providers) that require recreation to apply a change instead of update. We suggest never running terraform apply --auto-approve in production for exactly this reason!

Please note that we use GitHub issues for tracking bugs and enhancements rather than for questions. While we may be able to help with certain simple problems here it's generally better to use one of the community forums where there are far more people ready to help, whereas the GitHub issues here are generally monitored only by our few core maintainers.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.