Deploy fails when using org policy constraints/gcp.resourceLocations

[chilversc]

Environment info

firebase-tools: 7.12.1
Platform: Ubuntu 18.04

Steps to reproduce

• Create a GCP project and enable the organisation policy "constraints/gcp.resourceLocations".
• Ensure that us-central1 is not included in the list of allowed regions.
• Try to deploy a cloud function using the firebase tool that is set to use a valid region.

Expected behavior

The function should be deployed.

Actual behavior

The function fails to deploy with the following error:

"error": {
  "code": 400,
  "message": "The request has violated one or more Org Policies. Please refer to the respective violations for more information."
  "status": "FAILED_PRECONDITION" 
  "details\": [
    {
      "@type": "type.googleapis.com/google.rpc.PreconditionFailure",
      "violations": [
        {
          "type": "constraints/gcp.resourceLocations",
          "subject": "orgpolicy:projects/<project>",
          "description": "Constraint constraints/gcp.resourceLocations violated for projects/<project> attempting GenerateUploadUrlActionV1 with location set to us-central1. See https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints for more information."
        }
      ]
    }
  ]
}

Source

This issue comes from the following lines which hard code the region as us-central1 when generating the upload URL for the cloud functions.

firebase-tools/src/gcp/cloudfunctions.js

Line 275 in 5d06001

DEFAULT_REGION: "us-central1",

firebase-tools/src/deploy/functions/deploy.js

Line 11 in 78f11b6

var GCP_REGION = gcp.cloudfunctions.DEFAULT_REGION;

firebase-tools/src/deploy/functions/deploy.js

Line 18 in 78f11b6

.generateUploadUrl(context.projectId, GCP_REGION)

[google-oss-bot]

This issue does not seem to follow the issue template. Make sure you provide all the required information.

[samtstern]

@chilversc thanks for the detailed report! Could you provide a few more pieces of information:

1. Can you show the source of the function you're trying to deploy so we can see how the region is configured?
2. Could you show the entire output from --debug so we can see what HTTP requests are being made and which one returns the error?

[chilversc]

The function is defined as standard:

exports.helloWorld = functions.region('europe-west2')
  .https.onReceive((req, res) => res.status(200).send('Hello World'));

Debug output shows its the call to generateUploadUrl because its trying to use us-central1:

$ firebase deploy --only functions --debug
[2020-02-14T17:08:21.140Z] ----------------------------------------------------------------------
[2020-02-14T17:08:21.142Z] Command:       /usr/bin/node /usr/bin/firebase deploy --only functions --debug
[2020-02-14T17:08:21.142Z] CLI Version:   7.12.1
[2020-02-14T17:08:21.142Z] Platform:      linux
[2020-02-14T17:08:21.142Z] Node Version:  v12.14.0
[2020-02-14T17:08:21.142Z] Time:          Fri Feb 14 2020 17:08:21 GMT+0000 (Greenwich Mean Time)
[2020-02-14T17:08:21.143Z] ----------------------------------------------------------------------
[2020-02-14T17:08:21.143Z] 
[2020-02-14T17:08:21.146Z] > command requires scopes: ["email","openid","https://www.googleapis.com/auth/cloudplatformprojects.readonly","https://www.googleapis.com/auth/firebase","https://www.googleapis.com/auth/cloud-platform"]
[2020-02-14T17:08:21.147Z] > authorizing via signed-in user
[2020-02-14T17:08:21.147Z] [iam] checking project <project> for permissions ["cloudfunctions.functions.create","cloudfunctions.functions.delete","cloudfunctions.functions.get","cloudfunctions.functions.list","cloudfunctions.functions.update","cloudfunctions.operations.get","firebase.projects.get"]
[2020-02-14T17:08:21.148Z] >>> HTTP REQUEST POST https://cloudresourcemanager.googleapis.com/v1/projects/<project>:testIamPermissions  
 permissions=[cloudfunctions.functions.create, cloudfunctions.functions.delete, cloudfunctions.functions.get, cloudfunctions.functions.list, cloudfunctions.functions.update, cloudfunctions.operations.get, firebase.projects.get]
[2020-02-14T17:08:21.903Z] <<< HTTP RESPONSE 200 content-type=application/json; charset=UTF-8, vary=X-Origin, Referer, Origin,Accept-Encoding, date=Fri, 14 Feb 2020 17:08:21 GMT, server=ESF, cache-control=private, x-xss-protection=0, x-frame-options=SAMEORIGIN, x-content-type-options=nosniff, server-timing=gfet4t7; dur=657, alt-svc=quic=":443"; ma=2592000; v="46,43",h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000, accept-ranges=none, transfer-encoding=chunked

=== Deploying to '<project>'...

i  deploying functions
Running command: npm --prefix "$RESOURCE_DIR" run lint

> functions@ lint ./functions
> eslint .

✔  functions: Finished running predeploy script.
[2020-02-14T17:08:23.514Z] > [functions] package.json contents: {
  "name": "functions",
  "description": "Cloud Functions for Firebase",
  "scripts": {
    "lint": "eslint .",
    "serve": "firebase serve --only functions",
    "shell": "firebase functions:shell",
    "start": "npm run shell",
    "deploy": "firebase deploy --only functions",
    "logs": "firebase functions:log"
  },
  "engines": {
    "node": "8"
  },
  "dependencies": {
    "@js-joda/core": "^2.0.0",
    "cookie-parser": "^1.4.4",
    "firebase-admin": "^8.6.0",
    "firebase-functions": "^3.3.0"
  },
  "devDependencies": {
    "eslint": "^5.12.0",
    "eslint-plugin-promise": "^4.0.1",
    "firebase-functions-test": "^0.1.6"
  },
  "private": true
}
i  functions: ensuring necessary APIs are enabled...
[2020-02-14T17:08:23.515Z] >>> HTTP REQUEST GET https://serviceusage.googleapis.com/v1/projects/<project>/services/cloudfunctions.googleapis.com  
 
[2020-02-14T17:08:23.515Z] >>> HTTP REQUEST GET https://serviceusage.googleapis.com/v1/projects/<project>/services/runtimeconfig.googleapis.com  
 
[2020-02-14T17:08:24.361Z] <<< HTTP RESPONSE 200 content-type=application/json; charset=UTF-8, vary=X-Origin, Referer, Origin,Accept-Encoding, date=Fri, 14 Feb 2020 17:08:24 GMT, server=ESF, cache-control=private, x-xss-protection=0, x-frame-options=SAMEORIGIN, x-content-type-options=nosniff, alt-svc=quic=":443"; ma=2592000; v="46,43",h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000, accept-ranges=none, transfer-encoding=chunked
[2020-02-14T17:08:24.373Z] <<< HTTP RESPONSE 200 content-type=application/json; charset=UTF-8, vary=X-Origin, Referer, Origin,Accept-Encoding, date=Fri, 14 Feb 2020 17:08:24 GMT, server=ESF, cache-control=private, x-xss-protection=0, x-frame-options=SAMEORIGIN, x-content-type-options=nosniff, alt-svc=quic=":443"; ma=2592000; v="46,43",h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000, accept-ranges=none, transfer-encoding=chunked
✔  functions: all necessary APIs are enabled
[2020-02-14T17:08:24.376Z] >>> HTTP REQUEST GET https://firebase.googleapis.com/v1beta1/projects/<project>/adminSdkConfig  
 
[2020-02-14T17:08:24.766Z] <<< HTTP RESPONSE 200 content-type=application/json; charset=UTF-8, vary=X-Origin, Referer, Origin,Accept-Encoding, date=Fri, 14 Feb 2020 17:08:24 GMT, server=ESF, cache-control=private, x-xss-protection=0, x-frame-options=SAMEORIGIN, x-content-type-options=nosniff, alt-svc=quic=":443"; ma=2592000; v="46,43",h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000, accept-ranges=none, transfer-encoding=chunked
i  functions: preparing functions directory for uploading...
[2020-02-14T17:08:24.770Z] >>> HTTP REQUEST GET https://runtimeconfig.googleapis.com/v1beta1/projects/<project>/configs  
 
[2020-02-14T17:08:25.687Z] <<< HTTP RESPONSE 200 content-type=application/json; charset=UTF-8, vary=X-Origin, Referer, Origin,Accept-Encoding, date=Fri, 14 Feb 2020 17:08:25 GMT, server=ESF, cache-control=private, x-xss-protection=0, x-frame-options=SAMEORIGIN, x-content-type-options=nosniff, alt-svc=quic=":443"; ma=2592000; v="46,43",h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000, accept-ranges=none, transfer-encoding=chunked
i  functions: packaged functions (42.64 KB) for uploading
[2020-02-14T17:08:25.983Z] >>> HTTP REQUEST POST https://cloudfunctions.googleapis.com/v1/projects/<project>/locations/us-central1/functions:generateUploadUrl  
 
[2020-02-14T17:08:26.404Z] <<< HTTP RESPONSE 400 vary=X-Origin, Referer, Origin,Accept-Encoding, content-type=application/json; charset=UTF-8, date=Fri, 14 Feb 2020 17:08:26 GMT, server=ESF, cache-control=private, x-xss-protection=0, x-frame-options=SAMEORIGIN, x-content-type-options=nosniff, alt-svc=quic=":443"; ma=2592000; v="46,43",h3-Q050=":443"; ma=2592000,h3-Q049=":443"; ma=2592000,h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000, accept-ranges=none, transfer-encoding=chunked
[2020-02-14T17:08:26.405Z] <<< HTTP RESPONSE BODY {
  "error": {
    "code": 400,
    "message": "The request has violated one or more Org Policies. Please refer to the respective violations for more information.",
    "status": "FAILED_PRECONDITION",
    "details": [
      {
        "@type": "type.googleapis.com/google.rpc.PreconditionFailure",
        "violations": [
          {
            "type": "constraints/gcp.resourceLocations",
            "subject": "orgpolicy:projects/<project>",
            "description": "Constraint constraints/gcp.resourceLocations violated for projects/<project> attempting GenerateUploadUrlActionV1 with location set to us-central1. See https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints for more information."
          }
        ]
      }
    ]
  }
}



There was an issue deploying your functions. Verify that your project has a Google App Engine instance setup at https://console.cloud.google.com/appengine and try again. If this issue persists, please contact support.
⚠  functions: Upload Error: HTTP Error: 400, The request has violated one or more Org Policies. Please refer to the respective violations for more information.

Error: HTTP Error: 400, The request has violated one or more Org Policies. Please refer to the respective violations for more information.
[2020-02-14T17:08:26.482Z] Error Context: {
  "body": {
    "error": {
      "code": 400,
      "message": "The request has violated one or more Org Policies. Please refer to the respective violations for more information.",
      "status": "FAILED_PRECONDITION",
      "details": [
        {
          "@type": "type.googleapis.com/google.rpc.PreconditionFailure",
          "violations": [
            {
              "type": "constraints/gcp.resourceLocations",
              "subject": "orgpolicy:projects/<project>",
              "description": "Constraint constraints/gcp.resourceLocations violated for projects/<project> attempting GenerateUploadUrlActionV1 with location set to us-central1. See https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints for more information."
            }
          ]
        }
      ]
    }
  },
  "response": {
    "statusCode": 400,
    "body": "{\n  \"error\": {\n    \"code\": 400,\n    \"message\": \"The request has violated one or more Org Policies. Please refer to the respective violations for more information.\",\n    \"status\": \"FAILED_PRECONDITION\",\n    \"details\": [\n      {\n        \"@type\": \"type.googleapis.com/google.rpc.PreconditionFailure\",\n        \"violations\": [\n          {\n            \"type\": \"constraints/gcp.resourceLocations\",\n            \"subject\": \"orgpolicy:projects/<project>\",\n            \"description\": \"Constraint constraints/gcp.resourceLocations violated for projects/<project> attempting GenerateUploadUrlActionV1 with location set to us-central1. See https://cloud.google.com/resource-manager/docs/organization-policy/org-policy-constraints for more information.\"\n          }\n        ]\n      }\n    ]\n  }\n}\n",
    "headers": {
      "vary": "X-Origin, Referer, Origin,Accept-Encoding",
      "content-type": "application/json; charset=UTF-8",
      "date": "Fri, 14 Feb 2020 17:08:26 GMT",
      "server": "ESF",
      "cache-control": "private",
      "x-xss-protection": "0",
      "x-frame-options": "SAMEORIGIN",
      "x-content-type-options": "nosniff",
      "alt-svc": "quic=\":443\"; ma=2592000; v=\"46,43\",h3-Q050=\":443\"; ma=2592000,h3-Q049=\":443\"; ma=2592000,h3-Q048=\":443\"; ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000",
      "accept-ranges": "none",
      "transfer-encoding": "chunked"
    },
    "request": {
      "uri": {
        "protocol": "https:",
        "slashes": true,
        "auth": null,
        "host": "cloudfunctions.googleapis.com",
        "port": 443,
        "hostname": "cloudfunctions.googleapis.com",
        "hash": null,
        "search": null,
        "query": null,
        "pathname": "/v1/projects/<project>/locations/us-central1/functions:generateUploadUrl",
        "path": "/v1/projects/<project>/locations/us-central1/functions:generateUploadUrl",
        "href": "https://cloudfunctions.googleapis.com/v1/projects/<project>/locations/us-central1/functions:generateUploadUrl"
      },
      "method": "POST"
    }
  }
}

Having trouble? Try firebase [command] --help

[samtstern]

@chilversc thanks! So this is the request that's failing (as you suspected):

[2020-02-14T17:08:25.983Z] >>> HTTP REQUEST POST https://cloudfunctions.googleapis.com/v1/projects/<project>/locations/us-central1/functions:generateUploadUrl  

Assigning to @joehan who I think will know what we should be doing differently.

[joehan]

Hey @chilversc, thanks for finding this and providing such a clear and detailed report! This definitely seems like something that we overlooked, and we should create some way to ensure that we use a working region for generateUploadUrl.

My initial thought is to grab a region from one of the function triggers and use that for the generateUploadUrl call, but I want to run that approach by the team first to see if its the right choice. Either way, I'll update this issue when we have a good solution.

[Mistic92]

Hi, we just encountered the same issue. I have restricted region to all in EU but we can't deploy function to europe-west1 which is weird.

There was an issue deploying your functions. Verify that your project has a Google App Engine instance setup at https://console.cloud.google.com/appengine and try again. If this issue persists, please contact support.
!  functions: Upload Error: HTTP Error: 400, The request has violated one or more Org Policies. Please refer to the respective violations for more information.

I see ther this call to us-central1 which is prohibited.

"error": {
    "code": 400,
    "message": "The request has violated one or more Org Policies. Please refer to the respective violations for more information.",
    "status": "FAILED_PRECONDITION",
    "details": [
      {
        "@type": "type.googleapis.com/google.rpc.PreconditionFailure",
        "violations": [
          {
            "type": "constraints/gcp.resourceLocations",
            "subject": "orgpolicy:projects/yoshai-XXXXXX",
            "description": "Constraint constraints/gcp.resourceLocations violated for projects/yoshai-XXXXXXX attempting GenerateUploadUrlActionV1 with location set to us-central1. See https://cloud.google.com/resource-manager/do
cs/organization-policy/org-policy-constraints for more information."
          }
        ]
      }

Do you have any update for this issue?

[demianjaeger-oviva]

Thanks @joehan for fixing this !

We are now able to do: export FIREBASE_FUNCTIONS_UPLOAD_REGION="europe-west1"

see e881f1e

[joehan]

Thanks for pointing this out! As mentioned, you can now configure this to any region you want::
export FIREBASE_FUNCTIONS_UPLOAD_REGION="europe-west1"