Error updating {{resource}}: The update_mask in the Update{{resource}}Request must be set

[hspens]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

Terraform v1.5.7
hashicorp/google 5.0.0

Affected Resource(s)

• google_pubsub_topic
  edit: Updated with the following resources as I ran into similar issues with those as well
• google_pubsub_subscription
• google_certificate_manager_certificate_map
• google_certificate_manager_dns_authorization
• google_certificate_manager_certificate

Terraform Configuration Files

Upgrading to the 5.0.0 provider caused the issue. No manual change involved.

Terraform wants to do the following:

  # module.<name>.google_pubsub_topic.topic will be updated in-place
!   resource "google_pubsub_topic" "topic" {
        id               = "projects/<cluster-id>/topics/<topic>"
        name             = <topic>
!       terraform_labels = {
+           "service" = "my-service"
+           "team"    = "my-team"
        }
        # (4 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

We aren't using any default_labels on the provider. Only labels on the terraform resource.

Debug Output

Apply fails with:

Error: Error updating Topic "projects/<project-id>/topics/<topic>": googleapi: Error 400: The update_mask in the UpdateTopicRequest must be set, and must contain a non-empty paths list.

  with module.<name>.google_pubsub_topic.topic,
  on .terraform/modules/<name>/topic.tf line 2, in resource "google_pubsub_topic" "topic":
   2: resource "google_pubsub_topic" "topic" {

Panic Output

Expected Behavior

Apply should have been successful.

Actual Behavior

Apply failed due to update_mask being empty on the UpdateTopicRequest. Looking at the go code for this it seems like there was a recent addition in the resource_pubsub_topic.go file: 3e00cfb#diff-48620e991ba2fc65e445167722a633eb4f8b7707be51c2cd8cee9404152b393cR457-R460

which removes the hasChanged check for labels and adds one for effective_labels instead. I don't see anything about terraform_labels there but that might not trigger labels to be added to the updateMask?

I'm not seeing any effective_labels in my terraform plan either so I don't really see anything that would lead to labels being added to the updateMask.

Steps to Reproduce

1. upgrade google provider from 4.80.0 -> 5.0.0
2. terraform apply

Important Factoids

References

[oprudkyi]

the same with subscriptions

Error updating Subscription ... googleapi: Error 400: The update_mask in the UpdateSubscriptionRequest must be set, and must contain a non-empty paths list.

in plan

   ~ resource "google_pubsub_subscription" "sub" {
       ~ terraform_labels             = {
           + "field" = "ttt"
         }

[hspens]

It seems like terraform thinks there are resources to update and fails on the first apply with the errors describe above (both for topics and subscriptions). Every new apply "syncs" a new resource type. When all types have synced it will report No Changes.

[oprudkyi]

@hspens thank you. the workaround works but in some cases I had to run apply few times to fix all subscriptions

[zli82016]

Sorry for the problem. I will look at it.

[rileykarson]

I think these APIs are rejecting spurious updates- @zli82016 can probably guard making the update call on a nonzero number of entries being added to the update mask. Most of these spurious calls should be a no-op as that's the default / standard behaviour for GCP APIs, but some will reject the call as y'all have found.

terraform_labels is updated on upgrade for any resource with labels- we use it to record the resource labels and default labels, to correctly handle the case where a default label is removed. Deltas against that field are duplicated against effective_labels in order to display the final state of the resource's labels after any apply. That update is prompting Terraform to think there's an update needed which we're later determining nothing is changing based on the values in effective_labels, resulting in an empty update, but still making the now-spurious call to the API.

---

Before we ship a fix, the following workarounds you can:

• Apply these updates individually, as even a failure will correctly amend the terraform_labels field
• Make some other change- such as adding a new label or modifying the existing one to avoid making a spurious call
• (dangerous) Update your state file so that labels and terraform_labels share the same values

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.