Tutorial of @app.route('/') with authentication

[amirbtb]

I'm trying to setup a Cloud Function with multiple routes. The endpoint must be private but I don't really know much about authentication in Google Cloud.
I've spend a lot of time reading Goblet docs Authentication Topic and all mentioned Google Docs + intensive research but I'm not sure how I should proceed : Do I have to setup something else in GCP ? (IAP, 0Auth) ?

[anovis]

It depends what is making the request. The most common use case is either a service or user using a service account. In that case you would add the following to your goblet security definitions

     "securityDefinitions": {
                "service-account": {
                    "authorizationUrl": "",
                    "flow": "implicit",
                    "type": "oauth2",
                    "x-google-audiences": "SERVICE_ACCOUNT_NAME@PROJECT.iam.gserviceaccount.com",
                    "x-google-issuer": "SERVICE_ACCOUNT_NAME@PROJECT.iam.gserviceaccount.com",
                    "x-google-jwks_uri": "https://www.googleapis.com/robot/v1/metadata/x509/securetoken@system.gserviceaccount.com"
                },
}

then you would generate a token with the service account and attach to your request "Authorization: Bearer ${TOKEN}"

another option if you are using firebase auth on a web or mobile frontend is to simply pass that token and configure your security definition using the firebase client id.

 securityDefinitions:
    firebase:
      authorizationUrl: ""
      flow: "implicit"
      type: "oauth2"
      # Replace YOUR-PROJECT-ID with your project ID
      x-google-issuer: "https://securetoken.google.com/YOUR-PROJECT-ID"
      x-google-jwks_uri: "https://www.googleapis.com/service_accounts/v1/metadata/x509/securetoken@system.gserviceaccount.com"
      x-google-audiences: "YOUR-PROJECT-ID"

i haven't used the other methods, but they should work similarly with the only difference being the token in the request Authorization header.

[amirbtb]

Let's say the project where I created my service account is my-gcp-project and the service account name is my-service-account
This is my security definition

"securityDefinitions":{
        "service-account": {
            "authorizationUrl": "",
            "flow": "implicit",
            "type": "oauth2",
            "x-google-audiences": "my-service-account@my-gcp-project.iam.gserviceaccount.com",
            "x-google-issuer": "my-service-account@my-gcp-project.iam.gserviceaccount.com",
            "x-google-jwks_uri": "https://www.googleapis.com/service_accounts/v1/jwk/my-service-account@my-gcp-project.iam.gserviceaccount.com"
        }
    }

The deployment is successful and the API config is in status Completed in the API Gateway Console. For the sake of the explanation, let's say the deployed API Gateway URL is https://goblet-my-function-26nu6048.ew.gateway.dev

Here is the code of my local Flask App which I use to test the API Gateway. I found this code here GoogleCloudPlatform/python-docs-samples

utils.py

import time

import google.auth.crypt
import google.auth.jwt

import requests

def generate_jwt(
    service_account_keyfile,
    service_account_email,
    expiry_length=3600
):
    """Generates a signed JSON Web Token using a Google API Service Account."""
    iat = time.time()
    exp = iat + 3600
    payload = {
            'iss': service_account_email,
            'sub': service_account_email,
            'aud': service_account_email,
            'iat': iat,
            'exp': exp
    }
    additional_headers = {
        "alg": "RS256",
        "typ": "JWT"
    }
    signer = google.auth.crypt.RSASigner.from_service_account_file(service_account_keyfile)
    jwt = google.auth.jwt.encode(
        signer=signer,
        payload=payload, 
        header=additional_headers
    )
    return jwt


def make_jwt_request(signed_jwt, url):
    """Makes an authorized request to the endpoint"""
    headers = {
        'Authorization': 'Bearer {}'.format(signed_jwt.decode('utf-8')),
        'content-type': 'application/json'
    }
    response = requests.get(url, headers=headers)
    response.raise_for_status()
    return response

app.py

from flask import Flask
from requests.api import get
from .utils import make_jwt_request, generate_jwt
from .settings import APIGATEWAY_URL, SERVICE_ACCOUNT_EMAIL, SERVICE_ACCOUNT_KEYFILE

app = Flask(__name__)

@app.route("/")
def main():
    signed_jwt = generate_jwt(
        service_account_keyfile=SERVICE_ACCOUNT_KEYFILE,
        service_account_email=SERVICE_ACCOUNT_EMAIL
    )
    response = make_jwt_request(
        signed_jwt=signed_jwt,
        url=APIGATEWAY_URL
    )
    return response.text

if __name__ == '__main__':
    app.run()

settings.py

import os

APIGATEWAY_URL = "https://goblet-my-function-26nu6048.ew.gateway.dev" # URL of the API Gateway
SERVICE_ACCOUNT_EMAIL = "my-service-account@my-gcp-project.iam.gserviceaccount.com" # Email of the service account
SERVICE_ACCOUNT_KEYFILE = os.environ.get('GOOGLE_APPLICATION_CREDENTIALS') # Path to the 'my-service-account' key file

When I run the app, I am redirected to a Google sign in page instead of being authorized. Is it normal ?

[anovis]

thats what api gateway will return if the auth is failing i beleive.

i am not sure exactly why yours is failing, but below is how we are generating the bearer token in python without a service account key .

def generate_jwt_payload(
        sa_email,
        audience,
        expiry_length=7600):
    """Generates a signed JSON Web Token using a Google API Service Account."""

    now = int(time.time())

    payload = {
        'iat': now,
        "exp": now + expiry_length,
        'iss': sa_email,
        'aud': audience,
        'sub': sa_email,
        'email': sa_email
    }
    return payload


def generate_service_account_jwt_tokens(service_account_name):
    # gets oauth token for service account...requires the service account to have iam.serviceAccounts.getAccessToken permissions
    credentials = GoogleCredentials.get_application_default()
    service = discovery.build('iamcredentials', 'v1', credentials=credentials)
    body = {
        "payload": json.dumps(generate_jwt_payload(service_account_name, service_account_name))
    }
    encoded_sa = urllib.parse.quote_plus(service_account_name)
    resp = service.projects().serviceAccounts().signJwt(name=f"projects/-/serviceAccounts/{encoded_sa}",
                                                        body=body).execute()
    access_jwt_token = credentials.access_token
    signed_jwt_token = resp["signedJwt"]
    return access_jwt_token, signed_jwt_token

from the cli you can also try running gcloud auth print-access-token --impersonate-service-account="NAME@PROJECT.iam.gserviceaccount.com" to generate your token and add to your request in postman or CURL

[amirbtb]

All of this is really confusing... The process is really different depending on the library used. According to google.oauth2.service_account documentation :

This profile uses a JWT to acquire an OAuth 2.0 access token. The JWT is used in place of the usual authorization token returned during the standard OAuth 2.0 Authorization Code grant. The JWT is only used for this purpose, as the acquired access token is used as the bearer token when making requests using these credentials.

This profile differs from normal OAuth 2.0 profile because no user consent step is required. The use of the private key allows this profile to assert identity directly.

This profile also differs from the google.auth.jwt authentication because the JWT credentials use the JWT directly as the bearer token. This profile instead only uses the JWT to obtain an OAuth 2.0 access token. The obtained OAuth 2.0 access token is used as the bearer token.

So If I understand correctly, if you are using a service account :

• Option 1 : use google.oauth2.service_account to get a signed JWT that you can use to get a OAuth 2.0 access token ; then use this token as the bearer token to call API Gateway
• Option 2 : use google.auth.jwt to get a signed JWT that you can use directly as the bearer token to call API Gateway.

But I think that the Open API spec used in the securityDefinitions of config.json (see example in previous post )is expecting a JWT Token in the header, so I cannot use Option 1.

I'm trying the second option.
I believe the solution you suggested in your last post falls in the first option, right ? I couldn't make it work (didn't try the gcloud command though).

[amirbtb]

After reading IAM Documentation : REST Resource: projects.serviceAccounts, I have a couple of remarks about the piece of code you suggested here :

• signJwt is deprecated
• you should avoid using the wild character - in f"projects/-/serviceAccounts/{encoded_sa}" ; according to the documentation, a better syntax would be f"projects/{PROJECT_ID}/serviceAccounts/{encoded_sa}". This syntax is for Signing JWTs with a client library

I'm sure that I only partially understand the problem, my remarks are only meant to get a better understand of the messy Google APIs.

[anovis]

@amirbtb thats good to know. they must have deprecated the signJwt recently, which is annoying since that was the easiest way I could figure out without having to use a key locally. I wish GCP had more robust documentation, since goblet in this case is just a pass through for auth handled by api gateway.

I guess they still prefer using a private key to sign the jwts. This is essentially what i was doing with signJwt, but instead of having the private key downloaded I was making an api call.

in terms of option1 and option2 all tokens should be provided as an Authorization: Bearer header.

Another option is to create an api key , but it is not recommended since it is a static key so is vulnerable to man-in-the-middle-attacks

I am curious though why your initial code was not working as expected.

lets try using a different route say "/test" to make sure its not just the "/" thats not working. and then try printing out the
signed_jwt token and use postman or curl and see what you get.

[anovis]

ah. Looked at the depreciation note @amirbtb and they write Note: This method is deprecated. Use the signJwt method in the IAM Service Account Credentials API instead. If you currently use this method, see the migration guide for instructions.

So this is the new api endpoint they are supporting.
It should be similar to my code before. It essentially does the same thing as with the key, but instead of using a key, it using this api endpoint to sign it. This way when you deploy your code you don't need to include an api key, but instead can just call this gcp endpoint

[anovis]

I will work on getting an example together and add documentation, since it is not clear at all from GCP and I spent way too much time getting it to work before.

[anovis]

just tested and worked with the following config .Not sure if it matters, but i used https://www.googleapis.com/service_accounts/v1/metadata/x509/ in the jwks_uri.

and this is the gcp api endpoint used for reference and no key required.

"securityDefinitions":{
        "service-account": {
            "authorizationUrl": "",
            "flow": "implicit",
            "type": "oauth2",
            "x-google-audiences": "my-service-account@my-gcp-project.iam.gserviceaccount.com",
            "x-google-issuer": "my-service-account@my-gcp-project.iam.gserviceaccount.com",
            "x-google-jwks_uri":"https://www.googleapis.com/service_accounts/v1/metadata/x509/my-service-account@my-gcp-project.iam.gserviceaccount.com"
        }
    }

import time
import json 
import urllib.parse
import requests
from oauth2client.client import GoogleCredentials
from googleapiclient import discovery

def generate_jwt_payload(service_account_email):
    """Generates jwt payload"""
    now = int(time.time())
    payload = {
        'iat': now,
        "exp": now + 3600,
        'iss': service_account_email,
        'aud':  service_account_email,
        'sub': service_account_email,
        'email': service_account_email
    }
    return payload

def get_jwt(service_account_email):
    """Generate a signed JSON Web Token using a Google API Service Account."""

    credentials = GoogleCredentials.get_application_default()
    service = discovery.build('iamcredentials', 'v1', credentials=credentials)
    body = {
        "payload": json.dumps(generate_jwt_payload(service_account_email))
    }
    encoded_sa = urllib.parse.quote_plus(service_account_email)
    resp = service.projects().serviceAccounts().signJwt(name=f"projects/-/serviceAccounts/{encoded_sa}", body=body).execute()
    return resp["signedJwt"]


def make_jwt_request(service_account_email, url):
    """Makes an authorized request to the endpoint"""
    signed_jwt = get_jwt(service_account_email)
    headers = {
        'Authorization': 'Bearer {}'.format(signed_jwt),
        'content-type': 'application/json'
    }
    response = requests.get(url, headers=headers)
    return response


if __name__ == '__main__':
    print(make_jwt_request("SERVICE_ACCOUNT_EMAIL","GATEWAY/PATH").text)

[amirbtb]

I'm getting this error :

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 2073, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1518, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1516, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1502, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
  File "/src/docs_client/app.py", line 10, in main
    response = make_jwt_request(SERVICE_ACCOUNT_EMAIL,GATEWAY_PATH)
  File "/src/docs_client/utils.py", line 36, in make_jwt_request
    signed_jwt = get_jwt(service_account_email)
  File "/src/docs_client/utils.py", line 30, in get_jwt
    resp = service.projects().serviceAccounts().signJwt(name=f"projects/-/serviceAccounts/{encoded_sa}", body=body).execute()
  File "/usr/local/lib/python3.8/dist-packages/googleapiclient/_helpers.py", line 131, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/googleapiclient/http.py", line 937, in execute
    raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 403 when requesting https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/my-service-account%my-project.iam.gserviceaccount.com:signJwt?alt=json returned "The caller does not have permission". Details: "The caller does not have permission">

I'm using a service account that has the following roles (and others not mentioned) :

• ApiGateway Admin
• ApiGateway Viewer
• Cloud Functions Admin
• Cloud Functions Invoker
• Service Account Token Creator

The keyfile path of the service account is exported as GOOGLE_APPLICATION_CREDENTIALS environment variable + I use the email of the service account as parameter for service_account_email argument in make_jwt_request().
I created a dedicated service account for the cloud function and passed it toserviceAccountEmail in config.json. It has no role as it is only meant to return static html.

I'm I missing something ?

[anovis]

hmm so you are running this app locally using a service account passed into GOOGLE_APPLICATION_CREDENTIALS and calling the deployed cloudfunction with a service account with no roles.

You should only need the roles/iam.serviceAccountTokenCreator permission to make the api call to generate the signedJwt.

What could be happening is it is using your personal credentials and not the service account. can you verify if you have the roles/iam.serviceAccountTokenCreator role.

in my script i generate the credentials using credentials = GoogleCredentials.get_application_default() which grabs my personal credentials locally, but will grab the credentials of the service account in the cloudfunction or other gcp service if that code is run there.

[amirbtb]

I added roles/iam.serviceAccountTokenCreator to my roles. I'm still getting the error. Since I can see the service account (the one declared with GOOGLE_APPLICATION_CREDENTIALS environment variable) in the error, I guess the credentials retrieved are the right ones. It is a mystery.

On another lead : Did you setup something in particular related to OAuth in your project ? I didn't setup anything. Do you think that it can be related ?

[anovis]

Hmm I don't think i set anything else up. I did run gcloud auth application-default login and then did not use the service account key or GOOGLE_APPLICATION_CREDENTIALS.

The script is actually going to use your credentials locally to call the gcp endpoint to then sign the jwt with the service account so that may be why it was in the logs.

The only other permission that i have needed in other cases is the roles/iam.serviceaccountuser but i don't think it is applicable on this case.

[amirbtb]

Maybe my issue is related to CORS since I use an index.html that needs to call a local endpoint that expose a .json file. I'm gonna look into that.
Thank you for your help in this issue !

[anovis]

that could be it. you can set cors=True in the endpoint to enable cors. https://anovis.github.io/goblet/docs/build/html/topics.html#cors

@app.route('/custom_backend', cors=True)
def home():
    return "enable cors"