Terraform apply errors with 404 when creating multiple auth rules for event hubs

[danieldekock-wintech]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureRM Provider) Version

Terraform v0.12.15
provider.azurerm v1.36.0

Affected Resource(s)

• azurerm_eventhub_authorization_rule

Terraform Configuration Files

provider "azurerm" {
  version = "=1.36.0"
}

resource "azurerm_resource_group" "rg-debug" {
  name = "rg-debug"
  location = "North Europe"
}

resource "azurerm_eventhub_namespace" "ehns-debug" {
  name = "ehns-debug"
  location = "North Europe"
  resource_group_name = azurerm_resource_group.rg-debug.name
  sku = "Standard"
  capacity = 1
}

resource "azurerm_eventhub" "eh-debug" {
  name = "eh-debug"
  namespace_name = azurerm_eventhub_namespace.ehns-debug.name
  resource_group_name = azurerm_resource_group.rg-debug.name
  partition_count = 2
  message_retention = 1
}

variable "auth-rules" {
  type    = list(string)
  default = ["rule1", "rule2", "rule3", "rule4", "rule5"]
}

resource "azurerm_eventhub_authorization_rule" "eh-rule" {
  for_each = toset(var.auth-rules)
  name = each.key
  namespace_name = azurerm_eventhub_namespace.ehns-debug.name
  eventhub_name = azurerm_eventhub.eh-debug.name
  resource_group_name = azurerm_resource_group.rg-debug.name
  listen = false
  send = true
  manage = false
}

Debug Output

https://gist.github.com/danieldekock-wintech/32e94d5f90def9372951bfef963aa1d5

Panic Output

None

Expected Behavior

Terraform apply should create all the authorization rules for the event hub

Actual Behavior

Terraform apply does not create all authorization rules and errors with a 404 that the authorization rules doesn't exists. e.g.

Error: eventhub.EventHubsClient#GetAuthorizationRule: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="NotFound" Message="The requested resource rule4 does not exist. CorrelationId: 9d7b8524-4
7a5-eabe-bd57-fe3960951970"

Steps to Reproduce

1. terraform apply

Important Factoids

N/A

References

A similar issue has been reported for resource "azurerm_notification_hub_authorization_rule"
#4087

• #0000

[tombuildsstuff]

@ravulachetan #4087 fixed this in the Notification Hubs resource, not in the EventHub resources - which is what this issue is (open) tracking

[ravulachetan]

Thanks @tombuildsstuff for the clarification.
Is there any estimate on when this will be resolved for event hub?

[nickmhankins]

@tombuildsstuff We are also running into this issue. It seems to happen intermittently. Is there any current workaround for this?

Error: eventhub.EventHubsClient#GetAuthorizationRule: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="NotFound" Message="The requested resource ListenSendSharedAccessKey does not exist"

  on eventhubs.tf line 23, in resource "azurerm_eventhub_authorization_rule" "listener_accesskey":

  23: resource azurerm_eventhub_authorization_rule listener_accesskey {

Error: eventhub.EventHubsClient#GetAuthorizationRule: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="NotFound" Message="The requested resource SendSharedAccessKey does not exist"


  on eventhubs.tf line 35, in resource "azurerm_eventhub_authorization_rule" "sender_accesskey":

  35: resource azurerm_eventhub_authorization_rule sender_accesskey {

[lukazeta]

Is there any current workaround for this?

since there is no retry mechanism, I've added ugly 5min sleep in between rules that appear to work(I did see this error with lower time on occasion).

provisioner "local-exec" {
  command = "echo sleep 300s; sleep 300"
}

[lukazeta]

I've eventually hit this error again with sleep. Thats when I noticed that all the rules are being deployed at once. I've chained them using dependencies instead and so far didnt hit this again.

[tombuildsstuff]

Ultimately this'll be fixed via a lock when creating/updating/deleting EventHub (& Namespace) Auth Rules - as per #4087 - but in the interim it should be possible to use Terraform's parallelism feature to work around this (although clearly that's not ideal for speed reasons)

[nickmhankins]

@tombuildsstuff I've submitted a pull request to fix the issue here:
#6701

Thanks

[ttjackott]

Would this also affect azurerm_servicebus_queue_authorization_rule resources? We are seeing the same error when creating multiple of these.

[nickmhankins]

@jacksinclairaudley No, just event_hub. I'd open a separate issue for servicebus auth rules.

[ghost]

This has been released in version 2.9.0 of the provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading. As an example:

provider "azurerm" {
    version = "~> 2.9.0"
}
# ... other configuration ...

[AntonChernysh]

this still doesn't work in 2.10.0

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!