Handling Azure data-plane readiness after resource creation #4448
Replies: 2 comments 1 reply
|
Hi @corwynlevettWTW, Unfortunately OpenTofu's view of what's going on in the remote system is limited to what a provider reports, and so if a provider reports that something has been successfully created then OpenTofu will assume that to be true and proceed with applying anything that depends on it. If there's a delay after a successful API call before something becomes usable then the provider is the only component that can really do anything to compensate for that. I'm not very familiar with Azure in particular, but I know from experience that the AWS provider tends to have special logic built into it where either the creation of something polls until the provider is able to successfully read it back from the API or the downstream things that depend on it have their own polling behavior to wait for the parent object to become available instead of immediately failing with an error. It sounds like the Azure provider features you're using here don't have similar logic to compensate for the remote API's eventual consistency. Since only the provider knows how to interact directly with the remote system, without that built-in handling unfortunately just introducing hard-coded delays into the configuration (e.g. using the It seems like a variant of the problem you discussed here was previously reported in hashicorp/terraform-provider-azurerm#4445 but closed without any response from the maintainers. Some of the commenters on hashicorp/terraform-provider-azurerm#3939 might've been seeing problems similar to yours too, but that issue seems to be describing a wide variety of different problems. I'm afraid I don't have anything good to suggest beyond seeing if this is something that can be fixed upstream in the provider itself. Given the wide variety of failure modes you described though, this specific API seems like it's quite challenging to support robustly in an OpenTofu/Terraform provider and perhaps the only way the provider could handle this is to keep retrying for a while until it either succeeds or reaches a timeout. 😞 |
|
Thanks very much for the detailed response, we'll investigate this further at the provider level and look at the existing approaches to polling and retrying until the resource is actually usable, and see whether there's an opportunity to contribute or work with the relevant provider maintainers on a solution. It's useful to understand that this is something the provider is better placed to handle than OpenTofu itself. Thanks again for taking the time to explain this and point us in the right direction :) |
Uh oh!
There was an error while loading. Please reload this page.
Hi Everyone,
I'm looking for some guidance on an issue we've encountered when provisioning Azure infrastructure with OpenTofu.
As an example, we have scenarios where we create an Azure Storage Account and then need to perform an operation against its data plane as part of the same deployment.
The challenge we've found is that there can be a period where OpenTofu/Azure reports the Storage Account resource as successfully provisioned, but the data plane isn't yet ready to accept requests.
If we then immediately attempt to perform an operation against the Storage Account, we can receive errors such as,
azurerm_storage_tableresource fails with:After waiting for a period of time, the exact same operation succeeds without any configuration changes.
From our investigation, this appears to be an eventual-consistency/readiness issue rather than an actual provisioning failure. The Azure control plane considers the resource provisioned, but the data plane and/or associated networking isn't necessarily ready at the same point in time. At the moment, we've had to introduce polling/retry logic to wait for the resource to become genuinely usable before continuing.
We've experimented with approaches such as explicit waits and external scripts, but these can be brittle because the amount of time required isn't deterministic. We've therefore currently encapsulated this behaviour in a reusable module which waits for the resource to become accessible before proceeding.
This works for us, but we're conscious that this is essentially a workaround, and we'd prefer to use a more native OpenTofu mechanism if one exists. We'd be interested to hear whether this is a known issue for others working with Azure and OpenTofu, and whether there's a recommended or more native way of handling this kind of control-plane/data-plane readiness gap.
We're particularly interested to know if this is something that has been discussed for future OpenTofu capabilities, as we'd ideally like to replace our current workaround with a more native solution if one becomes available.
Thanks in advance for any guidance.
All reactions