Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support updating customData on Azure RM VM Scaling Sets #61

Closed
hashibot opened this issue Jun 13, 2017 · 6 comments · Fixed by #559
Closed

Support updating customData on Azure RM VM Scaling Sets #61

hashibot opened this issue Jun 13, 2017 · 6 comments · Fixed by #559
Assignees
Labels
enhancement service/vmss Virtual Machine Scale Sets upstream
Milestone

Comments

@hashibot
Copy link

This issue was originally opened by @lmickh as hashicorp/terraform#13547. It was migrated here as part of the provider split. The original body of the issue is below.


Terraform Version

Terraform v0.9.2

Affected Resource(s)

azurerm_virtual_machine_scale_set

Expected Behavior

Allow the update to the customData.

Actual Behavior

Get the following message:

compute.VirtualMachineScaleSetsClient#CreateOrUpdate: Failure responding to request: StatusCode=409 -- Original Error: autorest/azure: Service returned an error. Status=409 Code="PropertyChangeNotAllowed" Message="Changing property 'customData' is not allowed."

Steps to Reproduce

Create an azurerm_virtual_machine_scale_set resource with generated customData in the os_profile.
Generate new customData (change a cloud-init template file or ignition resource for example)
Run terraform apply against the scale set.

Important Factoids

My understanding is that the Azure Compute API was updated to allow for changes to customData. A later version of the api ( > "2016-03-30" possibly "2016-08-30") may need to be used. I tried confirming the specific version of the change, but have had troubles do to the lack of documentation for the Azure API.

@apeschel
Copy link

I have confirmed via ARM template that the latest version of the Compute API 2017-03-30 supports changing customData.

https://docs.microsoft.com/en-us/azure/templates/microsoft.compute/virtualmachinescalesets

@tombuildsstuff
Copy link
Member

@apeschel thanks for confirming this - I've requested that the Azure SDK for Go is upgraded to use this new API version

Thanks!

@apeschel
Copy link

@tombuildsstuff A caveat to be aware of when updating the CustomData for a VMSS -- it will not update on existing hosts, only new hosts. There does not appear to be any way to get waagent to pull the new CustomData, so you will need to reimage existing hosts if you want them to get the updated CustomData. This can be done manually with az vmss reimage.

@tombuildsstuff
Copy link
Member

@apeschel indeed, however there's a ReImageAll method in the SDK which should allow us to (optionally) reimage all of the machines in the set - which I believe should solve this :)

@apeschel
Copy link

@tombuildsstuff The danger with going that route is the VMSS may be a HA cluster of some kind where you don't want all your hosts to go down at once. A rolling reimage would be fine, but I believe ReImageAll just kills and rebuilds everything at once.

@rcarun rcarun added the msft label Oct 11, 2017
@rcarun rcarun added this to the M1 milestone Oct 11, 2017
@rcarun rcarun added M1 service/vmss Virtual Machine Scale Sets and removed msft labels Oct 11, 2017
@tombuildsstuff tombuildsstuff self-assigned this Nov 14, 2017
tombuildsstuff added a commit that referenced this issue Nov 15, 2017
tombuildsstuff added a commit that referenced this issue Nov 15, 2017
sebastus added a commit to sebastus/terraform-provider-azurerm that referenced this issue Dec 8, 2017
* Provision sample for ASP.NET on azure_rm_app_service

* Added vnet datasource

* add identity property to vm

* refactor, tests and docs

* added vnet_peering

* changing to TypeMap

* Updating the Provider block

* Variable consistency and removing unused variables

* Changed to azure_virtual_network, added crash control and added documentation.

* vmss: Support for updating the customData field

Fixes hashicorp#61
Fixes hashicorp#490

* Updating to include hashicorp#559

* Support for Auto Inflating

```
$ acctests azurerm TestAccAzureRMEventHubNamespace_maximumThroughputUnits
=== RUN   TestAccAzureRMEventHubNamespace_maximumThroughputUnits
--- PASS: TestAccAzureRMEventHubNamespace_maximumThroughputUnits (202.41s)
PASS
ok      github.com/terraform-providers/terraform-provider-azurerm/azurerm    202.432s
```

* New Resource: `azurerm_network_watcher`

```
$ acctests azurerm TestAccAzureRMNetworkWatcher_
=== RUN   TestAccAzureRMNetworkWatcher_importBasic
--- PASS: TestAccAzureRMNetworkWatcher_importBasic (75.79s)
=== RUN   TestAccAzureRMNetworkWatcher_importComplete
--- PASS: TestAccAzureRMNetworkWatcher_importComplete (69.85s)
=== RUN   TestAccAzureRMNetworkWatcher_basic
--- PASS: TestAccAzureRMNetworkWatcher_basic (69.62s)
=== RUN   TestAccAzureRMNetworkWatcher_complete
--- PASS: TestAccAzureRMNetworkWatcher_complete (72.16s)
=== RUN   TestAccAzureRMNetworkWatcher_update
--- PASS: TestAccAzureRMNetworkWatcher_update (81.75s)
=== RUN   TestAccAzureRMNetworkWatcher_disappears
--- PASS: TestAccAzureRMNetworkWatcher_disappears (94.38s)
PASS
ok
```

* Updating to include hashicorp#569

* Hotfix: upgrade packages under go-autorest to be v9.4.1.

Intergrate with latest version of go-autorest to read access tokens through new way
customized through environment variable. The old behavior on local shell will be kept.

Notice: for Azure Cloud Shell user, please make sure that they're using latest patched
provider.

* Vendoring the Locks SDK

* New Resource: `azurerm_management_lock`

Note: As the Subscription specific Locks will break other tests; these tests need to be run individually.
As such I've introduced the `TF_ACC_SUBSCRIPTION_PARALLEL_LOCK`  environment variable for this purpose.

Tests pass:

```
$ TF_ACC_SUBSCRIPTION_PARALLEL_LOCK=1 acctests azurerm TestAccAzureRMManagementLock_
=== RUN   TestAccAzureRMManagementLock_importResourceGroupReadOnlyBasic
--- PASS: TestAccAzureRMManagementLock_importResourceGroupReadOnlyBasic (61.52s)
=== RUN   TestAccAzureRMManagementLock_importResourceGroupReadOnlyComplete
--- PASS: TestAccAzureRMManagementLock_importResourceGroupReadOnlyComplete (58.75s)
=== RUN   TestAccAzureRMManagementLock_importResourceGroupCanNotDeleteBasic
--- PASS: TestAccAzureRMManagementLock_importResourceGroupCanNotDeleteBasic (53.38s)
=== RUN   TestAccAzureRMManagementLock_importResourceGroupCanNotDeleteComplete
--- PASS: TestAccAzureRMManagementLock_importResourceGroupCanNotDeleteComplete (46.87s)
=== RUN   TestAccAzureRMManagementLock_importPublicIPCanNotDeleteBasic
--- PASS: TestAccAzureRMManagementLock_importPublicIPCanNotDeleteBasic (80.46s)
=== RUN   TestAccAzureRMManagementLock_importPublicIPReadOnlyBasic
--- PASS: TestAccAzureRMManagementLock_importPublicIPReadOnlyBasic (68.53s)
=== RUN   TestAccAzureRMManagementLock_resourceGroupReadOnlyBasic
--- PASS: TestAccAzureRMManagementLock_resourceGroupReadOnlyBasic (61.24s)
=== RUN   TestAccAzureRMManagementLock_resourceGroupReadOnlyComplete
--- PASS: TestAccAzureRMManagementLock_resourceGroupReadOnlyComplete (64.10s)
=== RUN   TestAccAzureRMManagementLock_resourceGroupCanNotDeleteBasic
--- PASS: TestAccAzureRMManagementLock_resourceGroupCanNotDeleteBasic (72.49s)
=== RUN   TestAccAzureRMManagementLock_resourceGroupCanNotDeleteComplete
--- PASS: TestAccAzureRMManagementLock_resourceGroupCanNotDeleteComplete (113.71s)
=== RUN   TestAccAzureRMManagementLock_publicIPReadOnlyBasic
--- PASS: TestAccAzureRMManagementLock_publicIPReadOnlyBasic (64.05s)
=== RUN   TestAccAzureRMManagementLock_publicIPCanNotDeleteBasic
--- PASS: TestAccAzureRMManagementLock_publicIPCanNotDeleteBasic (94.53s)
=== RUN   TestAccAzureRMManagementLock_subscriptionReadOnlyBasic
--- PASS: TestAccAzureRMManagementLock_subscriptionReadOnlyBasic (17.98s)
=== RUN   TestAccAzureRMManagementLock_subscriptionCanNotDeleteBasic
--- PASS: TestAccAzureRMManagementLock_subscriptionCanNotDeleteBasic (15.20s)
PASS
ok      github.com/terraform-providers/terraform-provider-azurerm/azurerm    872.839s
```

Fixes hashicorp#23

* Updating to include hashicorp#573

* Updating to include hashicorp#571

* Adding validation for the locks name

Tests:
```
$ acctests azurerm TestValidateManagementLockName
=== RUN   TestValidateManagementLockName
--- PASS: TestValidateManagementLockName (0.00s)
PASS
ok      github.com/terraform-providers/terraform-provider-azurerm/azurerm    0.020s
```

* Linting

* Updating to include hashicorp#575

* Updating the changelog for consistency

* removed tabs, used spaces

* add test for issue hashicorp#450

* Updated the way user agent string gets assigned.

* Changed code to make it more readable.

* pr tweaks

* Avoid out of index errors when flattening image data disks.

* Updating to include hashicorp#587

* Updating to include hashicorp#589

* Conditional loading of the Subscription ID / Tenant ID / Environment

* Refactoring the provider block to support determining the TenantID/Environment from the SubscriptionID

Splitting out the authentication logic into a helpers folder
Also adding unit tests for these - which pass:

```
$ go test . -v
=== RUN   TestAzureFindValidAccessTokenForTenant_InvalidDate
--- PASS: TestAzureFindValidAccessTokenForTenant_InvalidDate (0.00s)
=== RUN   TestAzureFindValidAccessTokenForTenant_Expired
2017/11/30 15:02:01 [DEBUG] Token "7cabcf30-8dca-43f9-91e6-fd56dfb8632f" has expired
--- PASS: TestAzureFindValidAccessTokenForTenant_Expired (0.00s)
=== RUN   TestAzureFindValidAccessTokenForTenant_ExpiringIn
--- PASS: TestAzureFindValidAccessTokenForTenant_ExpiringIn (0.00s)
=== RUN   TestAzureFindValidAccessTokenForTenant_InvalidManagementDomain
2017/11/30 15:02:01 [DEBUG] Resource "https://portal.azure.com/" isn't a management domain
--- PASS: TestAzureFindValidAccessTokenForTenant_InvalidManagementDomain (0.00s)
=== RUN   TestAzureFindValidAccessTokenForTenant_DifferentTenant
2017/11/30 15:02:01 [DEBUG] Resource "https://management.core.windows.net/" isn't for the correct Tenant
--- PASS: TestAzureFindValidAccessTokenForTenant_DifferentTenant (0.00s)
=== RUN   TestAzureFindValidAccessTokenForTenant_ValidFromCloudShell
--- PASS: TestAzureFindValidAccessTokenForTenant_ValidFromCloudShell (0.00s)
=== RUN   TestAzureFindValidAccessTokenForTenant_ValidFromAzureCLI
--- PASS: TestAzureFindValidAccessTokenForTenant_ValidFromAzureCLI (0.00s)
=== RUN   TestAzureFindValidAccessTokenForTenant_NoTokens
--- PASS: TestAzureFindValidAccessTokenForTenant_NoTokens (0.00s)
=== RUN   TestAzureCLIProfileFindDefaultSubscription
--- PASS: TestAzureCLIProfileFindDefaultSubscription (0.00s)
=== RUN   TestAzureCLIProfileFindSubscription
--- PASS: TestAzureCLIProfileFindSubscription (0.00s)
=== RUN   TestAzurePopulateSubscriptionFromCLIProfile_Missing
--- PASS: TestAzurePopulateSubscriptionFromCLIProfile_Missing (0.00s)
=== RUN   TestAzurePopulateSubscriptionFromCLIProfile_NoDefault
--- PASS: TestAzurePopulateSubscriptionFromCLIProfile_NoDefault (0.00s)
=== RUN   TestAzurePopulateSubscriptionFromCLIProfile_Default
--- PASS: TestAzurePopulateSubscriptionFromCLIProfile_Default (0.00s)
=== RUN   TestAzurePopulateTenantAndEnvironmentFromCLIProfile_Empty
--- PASS: TestAzurePopulateTenantAndEnvironmentFromCLIProfile_Empty (0.00s)
=== RUN   TestAzurePopulateTenantAndEnvironmentFromCLIProfile_MissingSubscription
--- PASS: TestAzurePopulateTenantAndEnvironmentFromCLIProfile_MissingSubscription (0.00s)
=== RUN   TestAzurePopulateTenantAndEnvironmentFromCLIProfile_PopulateEnvironment
--- PASS: TestAzurePopulateTenantAndEnvironmentFromCLIProfile_PopulateEnvironment (0.00s)
=== RUN   TestAzurePopulateTenantAndEnvironmentFromCLIProfile_NormaliseAndPopulateEnvironment
--- PASS: TestAzurePopulateTenantAndEnvironmentFromCLIProfile_NormaliseAndPopulateEnvironment (0.00s)
=== RUN   TestAzurePopulateTenantAndEnvironmentFromCLIProfile_PopulateTenantId
--- PASS: TestAzurePopulateTenantAndEnvironmentFromCLIProfile_PopulateTenantId (0.00s)
=== RUN   TestAzurePopulateTenantAndEnvironmentFromCLIProfile_Complete
--- PASS: TestAzurePopulateTenantAndEnvironmentFromCLIProfile_Complete (0.00s)
=== RUN   TestAzurePopulateFromAccessToken_Missing
--- PASS: TestAzurePopulateFromAccessToken_Missing (0.00s)
=== RUN   TestAzurePopulateFromAccessToken_Exists
--- PASS: TestAzurePopulateFromAccessToken_Exists (0.00s)
=== RUN   TestAzureEnvironmentNames
--- PASS: TestAzureEnvironmentNames (0.00s)
=== RUN   TestAzureValidateBearerAuth
--- PASS: TestAzureValidateBearerAuth (0.00s)
=== RUN   TestAzureValidateServicePrincipal
--- PASS: TestAzureValidateServicePrincipal (0.00s)
PASS
ok      github.com/terraform-providers/terraform-provider-azurerm/azurerm/helpers/authentication    0.012s
```

* Fixing the build

* Remove the field marked as "Removed" according to hashicorp#572.

* Upgrading to v11.2.2-beta of the Azure SDK for Go

* Updating to include hashicorp#593

* Fixing the Management Lock validation

* Adding a default value for the identity field

* Updating to include hashicorp#482

* Updating to include hashicorp#574

* Adding settings to the hash

Test passes:

```
$ acctests azurerm TestAccAzureRMVirtualMachineScaleSet_extensionUpdate
=== RUN   TestAccAzureRMVirtualMachineScaleSet_extensionUpdate
--- PASS: TestAccAzureRMVirtualMachineScaleSet_extensionUpdate (593.13s)
PASS
ok      github.com/terraform-providers/terraform-provider-azurerm/azurerm    593.153s
```

* Updating to include hashicorp#609

* Local Network Gateways: support for BGP Settings

```
$ acctests azurerm TestAccAzureRMLocalNetworkGateway_
=== RUN   TestAccAzureRMLocalNetworkGateway_importBasic
--- PASS: TestAccAzureRMLocalNetworkGateway_importBasic (82.23s)
=== RUN   TestAccAzureRMLocalNetworkGateway_basic
--- PASS: TestAccAzureRMLocalNetworkGateway_basic (81.29s)
=== RUN   TestAccAzureRMLocalNetworkGateway_disappears
--- PASS: TestAccAzureRMLocalNetworkGateway_disappears (79.17s)
=== RUN   TestAccAzureRMLocalNetworkGateway_bgpSettings
--- PASS: TestAccAzureRMLocalNetworkGateway_bgpSettings (78.70s)
=== RUN   TestAccAzureRMLocalNetworkGateway_bgpSettingsDisable
--- PASS: TestAccAzureRMLocalNetworkGateway_bgpSettingsDisable (96.18s)
=== RUN   TestAccAzureRMLocalNetworkGateway_bgpSettingsEnable
--- PASS: TestAccAzureRMLocalNetworkGateway_bgpSettingsEnable (97.39s)
=== RUN   TestAccAzureRMLocalNetworkGateway_bgpSettingsComplete
--- PASS: TestAccAzureRMLocalNetworkGateway_bgpSettingsComplete (79.68s)
PASS
ok      github.com/terraform-providers/terraform-provider-azurerm/azurerm    594.680s
```

* Refactoring

* Adding an import test for BGP Settings:

```
$ acctests azurerm TestAccAzureRMLocalNetworkGateway_importBGPSettingsComplete
=== RUN   TestAccAzureRMLocalNetworkGateway_importBGPSettingsComplete
--- PASS: TestAccAzureRMLocalNetworkGateway_importBGPSettingsComplete (80.96s)
PASS
ok      github.com/terraform-providers/terraform-provider-azurerm/azurerm    80.987s
```

* Splitting the data source out into it's own step

* Minor refactoring

* Updating to include hashicorp#533

* Exporting the Default Hostname field

* Updating the App Service example to be complete

This removes support for Publishing, since the SCM URL's aren't consistent across Sovereign Clouds (China/Germany/Govt etc)
Switches to using the new `default_site_hostname` field introduced in hashicorp#612 rather than assuming what it is

* Updating to include hashicorp#594

* Updating to include hashicorp#611

* Updating to include hashicorp#612

* Remove leading line break from key_vault_key docs

Leading line break causes page metadata to be ignored.
@ghost
Copy link

ghost commented Apr 1, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!

@hashicorp hashicorp locked and limited conversation to collaborators Apr 1, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement service/vmss Virtual Machine Scale Sets upstream
Projects
None yet
4 participants