Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make Location case-insensitive #1809

Closed
thecodetinker opened this issue Jul 6, 2022 · 8 comments · Fixed by #2281
Closed

Make Location case-insensitive #1809

thecodetinker opened this issue Jul 6, 2022 · 8 comments · Fixed by #2281
Assignees
Labels
impact/usability Something that impacts users' ability to use the product easily and intuitively kind/enhancement Improvements or new features resolution/fixed This issue was fixed
Milestone

Comments

@thecodetinker
Copy link

thecodetinker commented Jul 6, 2022

What happened?

I am trying to update one of our environments, and for some reason the case of the Location has changed from NorthEurope to northeurope. This is an Input<string> based off the ResourceGroup whose Location is actually specified as NorthEurope. It's not clear why the case has changed in Pulumi's eyes (edit: I've put a possible reason in the comment, and updated the code sample).

Nonetheless, now Pulumi wants to Replace the storage account which is... inconvenient.

image

Steps to reproduce

Unclear, but along the lines of

var resourceGroup = new ResourceGroup("ResourceGroup", new ResourceGroupArgs
{
    Location = "NorthEurope",
    ResourceGroupName = "RGName"
};

var storageAccount = new StorageAccount("StorageAccount", new StorageAccountArgs
{
   AccountName = accountName,
   Location = "NorthEurope",
   ResourceGroupName = resourceGroup.Name
};

Then change to

var resourceGroup = new ResourceGroup("ResourceGroup", new ResourceGroupArgs
{
    Location = "NorthEurope",
    ResourceGroupName = "RGName"
};

var storageAccount = new StorageAccount("StorageAccount", new StorageAccountArgs
{
   AccountName = accountName,
   Location = resourceGroup.Location, // Update here
   ResourceGroupName = resourceGroup.Name
};

Expected Behavior

As the Azure Region is the same, I expect that Pulumi would not make a change here.

Actual Behavior

Pulumi replaces the storage account.

Versions used

This is using the Pulumi Automation API so not sure how to get the 'about' information. Nuget packages are:

Pulumi: 3.25.1
AzureNative: 1.60

Additional context

Case sensitivity feels like a disaster waiting to happen. I've had a lot of issues with importing existing environments due to Azure's own inconsistency in URNs (resourcegroup vs resourceGroup being quite a prevalent one). See #854

Now this is a problem too. It would be really helpful if Pulumi could be more lenient when it comes to differences in case only.

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

@thecodetinker thecodetinker added kind/bug Some behavior is incorrect or out of spec needs-triage Needs attention from the triage team labels Jul 6, 2022
@thecodetinker
Copy link
Author

I can further see in the state file that the Resource Group's Location case differs between the Input and Output

"inputs": {
      "location": "WestEurope",
      "resourceGroupName": "IXS-WESTEUROPE-JUP001-PRIMARY"
}

"outputs": {
    "id": "/subscriptions/21863b6c-XXXX/resourceGroups/IXS-WESTEUROPE-JUP001-PRIMARY",
    "location": "westeurope",
}

So I can see that a simple refactoring to use ResourceGroup.Location as an argument for the Storage Account location, rather than the given configuration string, might provoke the above behaviour.

@thecodetinker
Copy link
Author

Aha thought I, I'll just update the stack. But still Pulumi is wanting to replace (MI this time) even though there's no change.

image

NB I changed the file directly in the Blob storage, rather than import/export. Would that make any difference?

@danielrbradley
Copy link
Member

danielrbradley commented Jul 6, 2022

Hi @thecodetinker thanks for the detailed writeup. Yes, the provider doesn't currently have knowledge of which fields are case-insensitive, but it would be a great addition to enhance the usibility of the provider – along with the other issue you referenced around URN case sensitivity.

The core reason behind a lot of this behaviour is that the value of the output is based on whatever the Azure API returns back to us. A similar issue would happen during a refresh too - where the value being read might vary from your original input.

I think the reason that your last example highlights the property as a replacement is because during preview it only knows that the value is going to change; it makes a guess as to the value it's likely to change to, but until it actually does the call to the service for the first resource it doesn't know for certain what value will come back for the output – and therefore the preview flags it as a possible change and therefore a possible replace. Doing these changes in separate steps might be helpful to see the change one at a time.

Another approach which might be helpful it to refactor the code for all resources to reference location from the same simple variable which reduces the complexities which might arise from outputs having the possibility of changing:

var location = "NorthEurope";
var resourceGroup = new ResourceGroup("ResourceGroup", new ResourceGroupArgs
{
    Location = location,
    ResourceGroupName = "RGName"
});

var storageAccount = new StorageAccount("StorageAccount", new StorageAccountArgs
{
   AccountName = accountName,
   Location = location,
   ResourceGroupName = resourceGroup.Name
});

@danielrbradley danielrbradley added impact/usability Something that impacts users' ability to use the product easily and intuitively kind/enhancement Improvements or new features and removed kind/bug Some behavior is incorrect or out of spec needs-triage Needs attention from the triage team labels Jul 6, 2022
@thecodetinker
Copy link
Author

Hi @danielrbradley thanks for the quick response. I understand it's hard to make any general assumptions about whether or not a change of case is really a change - for example it would be in most text fields. However I wonder if it would be possible to put in place one or two exceptions for cases like this?

I can see now that I'm actually in this scenario precisely because of a code refactoring along the lines you suggest. We are migrating from TF to Pulumi so importing resources. The locations are all imported as lower case, so any that were defined in Pulumi as coming from the Config (which was PascalCase) did not import. Resources whose location was defined as coming from resourceGroup.Location output field were fine. So I just changed everything to use the latter and was then able to successfully import pre-existing environments.

All was good until we've come to update environments that were built in Pulumi from the start, and now it's wanting to delete half the resources and recreate them due to what seemed to be a fairly innocuous code change.

One final thing - the preview was indeed telling the truth, approving the change does result in the resources getting deleted, even after updating the state file to use lower case.

@danielrbradley
Copy link
Member

Yup, I agree this would be worth special case handling for this field given it applies to all resources. We're currently deep into a few other issues right now, but we'll keep this one in the backlog to look at further.

@danielrbradley danielrbradley changed the title Pulumi replaced a Storage Account due to case difference Make Location case-insensitive Jul 7, 2022
@danielrbradley danielrbradley added the help-wanted We'd love your contributions on this issue label Jul 7, 2022
@thecodetinker
Copy link
Author

Thanks, I'd appreciate that. One final thing: I thought I'd figured out it was because of the dependency change so I updated the propertyDependencies section in the state file too (based off a before/after run). Still no luck, sadly.

@thecodetinker
Copy link
Author

Ok one final thing, I think I've worked around this by setting options.IgnoreChanges = "location". It feels quite grotty, but in practice should be fine because the only reason to update this would be if the RG location changes, in which case everything gets replaced anyway 👍

@jbrezina
Copy link

jbrezina commented Mar 2, 2023

I created some resources using azure-native provider, I configured the location for the provider, now I changed the way of creating the resources so I set the location directly on the resource (it is not taken from the provider configuration). Pulumi now wants to replace the resource because of the location changed from UK South to uksouth even though the provider was before also configured to uksouth.
I tried to export the stack, change the location manually and import it back, but the problem remains. When I refresh the stack, it also returns back to UK South.
So I understand that it depends on what the Azure API returns back. I wanted to point out it is not only case sensitivity. It would be great if the values were always mapped to RunLocation. In other ways it might be issue of DisplayName vs Name of the location. You can get the complete list using the following command:
az account list-locations -o table
Which returns these at the moment:

DisplayName               Name                 RegionalDisplayName
------------------------  -------------------  -------------------------------------
East US                   eastus               (US) East US
East US 2                 eastus2              (US) East US 2
South Central US          southcentralus       (US) South Central US
West US 2                 westus2              (US) West US 2
West US 3                 westus3              (US) West US 3
Australia East            australiaeast        (Asia Pacific) Australia East
Southeast Asia            southeastasia        (Asia Pacific) Southeast Asia
North Europe              northeurope          (Europe) North Europe
Sweden Central            swedencentral        (Europe) Sweden Central
UK South                  uksouth              (Europe) UK South
West Europe               westeurope           (Europe) West Europe
Central US                centralus            (US) Central US
South Africa North        southafricanorth     (Africa) South Africa North
Central India             centralindia         (Asia Pacific) Central India
East Asia                 eastasia             (Asia Pacific) East Asia
Japan East                japaneast            (Asia Pacific) Japan East
Korea Central             koreacentral         (Asia Pacific) Korea Central
Canada Central            canadacentral        (Canada) Canada Central
France Central            francecentral        (Europe) France Central
Germany West Central      germanywestcentral   (Europe) Germany West Central
Norway East               norwayeast           (Europe) Norway East
Switzerland North         switzerlandnorth     (Europe) Switzerland North
UAE North                 uaenorth             (Middle East) UAE North
Brazil South              brazilsouth          (South America) Brazil South
East US 2 EUAP            eastus2euap          (US) East US 2 EUAP
Qatar Central             qatarcentral         (Middle East) Qatar Central
Central US (Stage)        centralusstage       (US) Central US (Stage)
East US (Stage)           eastusstage          (US) East US (Stage)
East US 2 (Stage)         eastus2stage         (US) East US 2 (Stage)
North Central US (Stage)  northcentralusstage  (US) North Central US (Stage)
South Central US (Stage)  southcentralusstage  (US) South Central US (Stage)
West US (Stage)           westusstage          (US) West US (Stage)
West US 2 (Stage)         westus2stage         (US) West US 2 (Stage)
Asia                      asia                 Asia
Asia Pacific              asiapacific          Asia Pacific
Australia                 australia            Australia
Brazil                    brazil               Brazil
Canada                    canada               Canada
Europe                    europe               Europe
France                    france               France
Germany                   germany              Germany
Global                    global               Global
India                     india                India
Japan                     japan                Japan
Korea                     korea                Korea
Norway                    norway               Norway
Singapore                 singapore            Singapore
South Africa              southafrica          South Africa
Switzerland               switzerland          Switzerland
United Arab Emirates      uae                  United Arab Emirates
United Kingdom            uk                   United Kingdom
United States             unitedstates         United States
United States EUAP        unitedstateseuap     United States EUAP
East Asia (Stage)         eastasiastage        (Asia Pacific) East Asia (Stage)
Southeast Asia (Stage)    southeastasiastage   (Asia Pacific) Southeast Asia (Stage)
East US STG               eastusstg            (US) East US STG
North Central US          northcentralus       (US) North Central US
West US                   westus               (US) West US
Jio India West            jioindiawest         (Asia Pacific) Jio India West
Central US EUAP           centraluseuap        (US) Central US EUAP
West Central US           westcentralus        (US) West Central US
South Africa West         southafricawest      (Africa) South Africa West
Australia Central         australiacentral     (Asia Pacific) Australia Central
Australia Central 2       australiacentral2    (Asia Pacific) Australia Central 2
Australia Southeast       australiasoutheast   (Asia Pacific) Australia Southeast
Japan West                japanwest            (Asia Pacific) Japan West
Jio India Central         jioindiacentral      (Asia Pacific) Jio India Central
Korea South               koreasouth           (Asia Pacific) Korea South
South India               southindia           (Asia Pacific) South India
West India                westindia            (Asia Pacific) West India
Canada East               canadaeast           (Canada) Canada East
France South              francesouth          (Europe) France South
Germany North             germanynorth         (Europe) Germany North
Norway West               norwaywest           (Europe) Norway West
Switzerland West          switzerlandwest      (Europe) Switzerland West
UK West                   ukwest               (Europe) UK West
UAE Central               uaecentral           (Middle East) UAE Central
Brazil Southeast          brazilsoutheast      (South America) Brazil Southeast

mikhailshilkov added a commit that referenced this issue Mar 3, 2023
)

All `location` properties in Azure are case- and space- insensitive, so
both "West Europe" and "westeurope" are valid values representing the
same region.

This PR adjusts our diff calculation logic with this special rule. We
get a diff calculated by vanilla platform diffing, then we check if
there is an Update to the property called `location`, and if so, it
normalizes both values and compares those. Any non-significant changes
are ignored.

Fix #1809
Fix #1005
@pulumi-bot pulumi-bot added the resolution/fixed This issue was fixed label Mar 3, 2023
@mikhailshilkov mikhailshilkov added this to the 0.85 milestone Mar 3, 2023
@mikhailshilkov mikhailshilkov removed the help-wanted We'd love your contributions on this issue label Mar 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
impact/usability Something that impacts users' ability to use the product easily and intuitively kind/enhancement Improvements or new features resolution/fixed This issue was fixed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants