Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FAILED_PRECONDITION - quota restrictions #480

Closed
eggsy84 opened this issue Jan 8, 2024 · 7 comments
Closed

FAILED_PRECONDITION - quota restrictions #480

eggsy84 opened this issue Jan 8, 2024 · 7 comments
Labels
bug Something isn't working

Comments

@eggsy84
Copy link

eggsy84 commented Jan 8, 2024

TL;DR

Experiencing the following error when authenticating with a JSON service account

ERROR: (gcloud.run.deploy) FAILED_PRECONDITION: failed precondition: due to quota restrictions, cannot run builds in this region. Please contact support

I've tried manually from my terminal via executing

gcloud auth activate-service-account --key-file="same_service_account_saved_in_github.json"

Followed by:

gcloud run deploy some-service-name --source ./  --platform managed --format json --region europe-west2 --project some-project-name

Running directly in terminal it seems to be submitting the build fine.

I've checked all my quotas and everything looks ok so the error seems to be incorrect.

Wondering if I'm missing something?

Expected behavior

Expect gcloud build to be submitted successfully for cloud run

Observed behavior

ERROR: (gcloud.run.deploy) FAILED_PRECONDITION: failed precondition: due to quota restrictions, cannot run builds in this region. Please contact support

Action YAML

# Re-usable workflow format called from a main workflow

name: "deploy"
on:
  workflow_call:
    inputs:
      project:
        required: true
        type: string
      service_name:
        required: true
        type: string
      region:
        required: true
        type: string
    secrets:
      service_account_json:
        required: true

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Check out ${{github.ref_name}} branch
        uses: actions/checkout@v4
      - name: Authenticate with GCP
        id: auth
        uses: "google-github-actions/auth@v2"
        with:
          credentials_json: "${{ secrets.service_account_json }}"
      - name: Cloud run deploy
        uses: "google-github-actions/deploy-cloudrun@v2"
        with:
          service: ${{ inputs.service_name }}
          region: ${{ inputs.region }}
          source: ./
          project_id: ${{ inputs.project }}

Log output

with:
    service: some-service
    region: europe-west2
    source: ./
    project_id: some-project
    skip_default_labels: false
    no_traffic: false
  env:
    CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE: /home/runner/work/repo/repo/gha-creds-65661635efb6ab57.json
    GOOGLE_APPLICATION_CREDENTIALS: /home/runner/work/repo/repo/gha-creds-65661635efb6ab57.json
    GOOGLE_GHA_CREDS_PATH: /home/runner/work/repo/repo/gha-creds-65661635efb6ab57.json
    CLOUDSDK_CORE_PROJECT: some-project
    CLOUDSDK_PROJECT: some-project
    GCLOUD_PROJECT: some-project
    GCP_PROJECT: some-project
    GOOGLE_CLOUD_PROJECT: some-project
/usr/bin/tar xz --warning=no-unknown-keyword --overwrite -C /home/runner/work/_temp/c4167636-ff6e-49d1-a1ab-d249c50a15b9 -f /home/runner/work/_temp/2efd0bde-8bd3-45ab-90a2-10d1229b15b2
Successfully authenticated
Running: gcloud run deploy some-service --quiet --source ./ --update-labels managed-by=github-actions,commit-sha=f1b09e73d687a6914ce8cb6112345 --platform managed --format json --region europe-west2 --project some-project

Error: google-github-actions/deploy-cloudrun failed with: failed to execute gcloud command `gcloud run deploy some-service --quiet --source ./ --update-labels managed-by=github-actions,commit-sha=f1b09e73d687a6914ce8cb6112345 --platform managed --format json --region europe-west2 --project some-project`: This command is equivalent to running `gcloud builds submit --tag [IMAGE] ./` and `gcloud run deploy some-service --image [IMAGE]`

Building using Dockerfile and deploying container to Cloud Run service [some-service] in project [some-project] region [europe-west2]
Building and deploying new service...
Uploading sources.............done
failed
Deployment failed
ERROR: (gcloud.run.deploy) FAILED_PRECONDITION: failed precondition: due to quota restrictions, cannot run builds in this region. Please contact support

Additional information

No response

@eggsy84 eggsy84 added the bug Something isn't working label Jan 8, 2024
@sethvargo
Copy link
Member

Hi @eggsy84 - as the error message says, you've exhausted your quota. You can contact Google Cloud support to request a quota increase.

We only support the GitHub Action for deploying to Cloud Run. You must work with the Google Cloud support organization for quota issues.

@eggsy84
Copy link
Author

eggsy84 commented Jan 9, 2024

Hi @sethvargo

Thanks for reviewing and coming back to me. In terms of the quota, would it be right that the exact same authentication and gcloud run deploy commands work manually if I use them on my terminal?

I would have thought that the quota would also be thrown as an error there if it was truly a quota issue?

Are there some other quotas that the action is using because all my cloud run quotas seem to be ok (above 0)

@eggsy84
Copy link
Author

eggsy84 commented Jan 9, 2024

@eggsy84
Copy link
Author

eggsy84 commented Jan 9, 2024

@sethvargo Have found that if you pin the GCP CLI version then the GitHub actions work successfully so I think this might be a bug with the upstream CLI tool rather than this particular GitHub action

Reference material
https://stackoverflow.com/questions/77678578/google-cloud-run-failed-precondition-due-to-quota-restrictions-but-real-quot/77787728#77787728

Pinning CLI version

      - name: Cloud run deploy
        uses: "google-github-actions/deploy-cloudrun@v2"
        with:
          service: ${{ inputs.service_name }}
          region: ${{ inputs.region }}
          source: ./
          project_id: ${{ inputs.project }}
          gcloud_version: "455.0.0"

@sethvargo
Copy link
Member

Hi @eggsy84 - thanks for the feedback. Are you able to confirm the specific version where this breaks? Does 455 work but 456 does not? Here's a list of all the versions: https://cloud.google.com/sdk/docs/release-notes

@eggsy84
Copy link
Author

eggsy84 commented Jan 9, 2024

Hey @sethvargo just had a play - I tested with the version after 455.0.0, version 456.0.0, and it failed

      - name: Cloud run deploy
        uses: "google-github-actions/deploy-cloudrun@v2"
        with:
          service: ${{ inputs.service_name }}
          region: ${{ inputs.region }}
          source: ./
          project_id: ${{ inputs.project }}
          gcloud_version: "456.0.0"

resulted in

ERROR: (gcloud.run.deploy) FAILED_PRECONDITION: failed precondition: due to quota restrictions, cannot run builds in this region. Please contact support

So would assume that versions post 455 break as well?

@sethvargo
Copy link
Member

Hi @eggsy84 - the team identified the cause of this bug and is working on a fix to be included in the next release of gcloud.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Development

No branches or pull requests

2 participants