Skip to content

CLI: remove ResourceExhausted from retriable gRPC codes #3053

@migmartri

Description

@migmartri

Background

The CLI's retry logic in app/cli/cmd/errors.go currently treats codes.ResourceExhausted as a retriable gRPC error:

retriableCodes := []codes.Code{
    codes.Unavailable,
    codes.Internal,
    codes.ResourceExhausted,
    codes.DeadlineExceeded,
}

Problem

ResourceExhausted typically signals rate limiting or quota exhaustion. Blanket-retrying it is problematic:

  • Hard quotas don't self-recover. Retrying a monthly/daily quota limit just spins the retry loop until the 3-minute max elapsed time expires, wasting work.
  • It can amplify server load. If the server is already overloaded, additional retries make the situation worse.
  • gRPC guidance is to only retry ResourceExhausted when the server explicitly signals it's safe (via RetryInfo metadata with a retry_delay), rather than retrying unconditionally.

Proposed change

Remove codes.ResourceExhausted from the retriable set. A follow-up can add RetryInfo-aware handling if we want to honor server-indicated rate-limit backoffs explicitly.

Scope

  • app/cli/cmd/errors.go

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions