azurerm_role_assignment: Retry replication lag error for cross-tenant usage

[walkline]

Community Note

• Please vote on this PR by adding a 👍 reaction to the original PR to help the community and maintainers prioritize for review
• Please do not leave "+1" or "me too" comments, they generate extra noise for PR followers and do not help prioritize for review

Description

When working in a cross-tenant setup, there is a chance to encounter the following error on role assignment:

 Error: authorization.RoleAssignmentsClient#Create: Failure responding to request: StatusCode=403 -- Original Error: autorest/azure: Service returned an error. Status=403 Code="LinkedAuthorizationFailed" Message="The client 'xxxxxx-xxxx-xxxx-xxxx-xxxxxxx' with object id 'xxxxxx-xxxx-xxxx-xxxx-xxxxxxx' has permission to perform action 'Microsoft.Authorization/roleAssignments/write' on scope '/subscriptions/xxxxxx-xxxx-xxxx-xxxx-xxxxxxx/resourceGroups/xxxxxx/providers/Microsoft.ContainerRegistry/registries/xxxxxx/providers/Microsoft.Authorization/roleAssignments/xxxxxx-xxxx-xxxx-xxxx-xxxxxxx'; however, it does not have permission to perform action(s) 'Microsoft.ManagedIdentity/userAssignedIdentities/write' on the linked scope(s) '/subscriptions/xxxxxx-xxx-xxxx-xxx-xxxxxxx/resourceGroups/xxxxxx/providers/Microsoft.ManagedIdentity/userAssignedIdentities/xxxxxxx' (respectively) or the linked scope(s) are invalid."

In most cases, you may encounter this issue when you run the terraform apply command for the first time. However, when you rerun the terraform apply command after some time, it finishes successfully.

The terraform code in my case looks like this:

resource "azurerm_kubernetes_cluster" "cluster" {
# ...
}

resource "azurerm_container_registry" "registry" {
# ...
}

resource "azurerm_role_assignment" "k8s_to_acr" {
  scope                = azurerm_container_registry.registry.id
  role_definition_name = "AcrPull"
  principal_id         = azurerm_kubernetes_cluster.cluster.kubelet_identity[0].object_id
  skip_service_principal_aad_check = true
  delegated_managed_identity_resource_id =  azurerm_kubernetes_cluster.cluster.kubelet_identity[0].user_assigned_identity_id
  depends_on = [azurerm_container_registry.registry, azurerm_kubernetes_cluster.cluster]
}

With suggested code we will retry LinkedAuthorizationFailed error, if user provided delegated_managed_identity_resource_id (cross tenant usage) and set skip_service_principal_aad_check = true.

PR Checklist

• I have followed the guidelines in our Contributing Documentation.
• I have checked to ensure there aren't other open Pull Requests for the same update/change.
• I have checked if my changes close any open issues. If so please include appropriate closing keywords below.
• I have updated/added Documentation as required written in a helpful and kind way to assist users that may be unfamiliar with the resource / data source.
• I have used a meaningful PR title to help maintainers and other users understand this change and help prevent duplicate work.
  For example: “resource_name_here - description of change e.g. adding property new_property_name_here”

Changes to existing Resource / Data Source

• I have added an explanation of what my changes do and why I'd like you to include them (This may be covered by linking to an issue above, but may benefit from additional explanation).
• I have written new tests for my resource or datasource changes & updated any relevent documentation.
• I have successfully run tests with my changes locally. If not, please provide details on testing challenges that prevented you running the tests.
• (For changes that include a state migration only). I have manually tested the migration path between relevant versions of the provider.

Testing

• My submission includes Test coverage as described in the Contribution Guide and the tests pass. (if this is not possible for any reason, please include details of why you did or could not add test coverage)

I didn't find any tests for errors retrying.

Change Log

Below please provide what should go into the changelog (if anything) conforming to the Changelog Format documented here.

• azurerm_role_assignment - retry replication lag error for cross-tenant usage

This is a (please select all that apply):

• Bug Fix
• New Feature (ie adding a service, resource, or data source)
• Enhancement
• Breaking Change

Related Issue(s)

Fixes

Note

If this PR changes meaningfully during the course of review please update the title and description as required.

[katbyte]

Thanks @walkline - LGTM 🏕️