Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add operation retry for exceeded quota group OperationReadGroup #8746

Conversation

modular-magician
Copy link
Collaborator

Fixes #8655

The error we need to retry on:

{
  "error": {
    "code": 403,
    "message": "Quota exceeded for quota group 'OperationReadGroup' and limit 'Operation read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:<>'.",
    "errors": [
      {
        "message": "Quota exceeded for quota group 'OperationReadGroup' and limit 'Operation read requests per 100 seconds' of service 'compute.googleapis.com' for consumer 'project_number:<>'.",
        "domain": "usageLimits",
        "reason": "rateLimitExceeded"
      }
    ],
    "status": "PERMISSION_DENIED"
  }

Currently only GCE has a quota group called OperationReadGroup, but the compute operations use CommonRefreshFunc. Rather than rewrite compute operations to use a non-common refresh function, I just added another retry predicate to the existing one.

Tested by spamming operation reads. Sorry GCE SRE's...

Release Note Template for Downstream PRs (will be copied)

compute: fixed an issue where exceeding the operation rate limit would fail without retrying

Derived from GoogleCloudPlatform/magic-modules#4599

…icorp#4599)

Signed-off-by: Modular Magician <magic-modules@google.com>
@ghost
Copy link

ghost commented Apr 22, 2021

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!

@ghost ghost locked as resolved and limited conversation to collaborators Apr 22, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Terraform state leaks when the GCE Operation API(s) rate limit is exceeded
1 participant