403 errors when deploying aggregation-service

[yanghuang1028]

Hi team,

I’m trying to set up our deployment environment. But I encountered this error. Could you please help to look at it ? Thanks a lot !!!

These are the roles of our service accounts. Do I need to add some additional role permissions?
our projectId: ecs-1709881683838

Error: Error creating function: googleapi: Error 403: Could not create Cloud Run service dev-us-west2-worker-scale-in. Permission ‘iam.serviceaccounts.actAs’ denied on service account [worker-sa-aggregation-service@microsites-sa.iam.gserviceaccount.com](mailto:worker-sa-aggregation-service@microsites-sa.iam.gserviceaccount.com) (or it may not exist).
│
│   with module.job_service.module.autoscaling.google_cloudfunctions2_function.worker_scale_in_cloudfunction,
│   on ../../coordinator-services-and-shared-libraries/operator/terraform/gcp/modules/autoscaling/workerscalein.tf line 35, in resource “google_cloudfunctions2_function” “worker_scale_in_cloudfunction”:
│   35: resource “google_cloudfunctions2_function” “worker_scale_in_cloudfunction” {
│
╵
╷
│ Error: Error creating function: googleapi: Error 403: Could not create Cloud Run service dev-us-west2-frontend-service. Permission ‘iam.serviceaccounts.actAs’ denied on service account [dev-frontend@microsites-sa.iam.gserviceaccount.com](mailto:dev-frontend@microsites-sa.iam.gserviceaccount.com) (or it may not exist).
│
│   with module.job_service.module.frontend.google_cloudfunctions2_function.frontend_service_cloudfunction,
│   on ../../coordinator-services-and-shared-libraries/operator/terraform/gcp/modules/frontend/main.tf line 43, in resource “google_cloudfunctions2_function” “frontend_service_cloudfunction”:
│   43: resource “google_cloudfunctions2_function” “frontend_service_cloudfunction” {
│
╵
╷
│ Error: Error creating instance template: googleapi: Error 409: The resource ‘projects/ecs-1709881683838/global/instanceTemplates/dev-collector’ already exists, alreadyExists
│
│   with module.job_service.module.worker.google_compute_instance_template.collector,
│   on ../../coordinator-services-and-shared-libraries/operator/terraform/gcp/modules/worker/collector.tf line 49, in resource “google_compute_instance_template” “collector”:
│   49: resource “google_compute_instance_template” “collector” {

[maybellineboon]

Hi @yanghuang1028 ,

Thanks for reaching out. Just to check, did you use the terraform script when deploying your aggregation service deploy and worker?

You will need to run this when creating/updating your worker so that they are provided with the appropriate access. You can update the section below in the <repository>/terraform/gcp/environments/adtech_setup folder with file adtech_setup.auto.tfvars.

# [1] Uncomment below lines if you like Terraform grant needed permissions to
# pre-existing service accounts
# deploy_service_account_email = "<YourDeployServiceAccountName>@<ProjectID>.iam.gserviceaccount.com"
# worker_service_account_email = "<YourWorkerServiceAccountName>@<ProjectID>.iam.gserviceaccount.com"

[yanghuang1028]

Hi maybellineboon,
I did use the terraform script to update my deploy & worker accounts. But our situation is a bit complex. Due to safety, our company does not allow our project to create a service account(but terraform need to create a frontend account), so we have to use a global project(microsites-sa. it's owned by our company's GCP team) to create a frontend account in advance for the terraform script to use.

As a result, I change the main.tf of the path "terraform/gcp/modules/frontend" so that terraform can run properly. I don't know if this change caused this problem.

our project_id: ecs-1709881683838

our service accounts:

Here are my changes in "terraform/gcp/modules/frontend/main.tf"
click to see the pr details:

If convenient, please help to look at this! I've been stuck at this step for a long time...

Thanks anyway!!

[hostirosti]

Hi @yanghuang1028,

it looks like you're running into the issue described here: https://stackoverflow.com/questions/68579808/how-to-solve-error-creating-service-googleapi-error-403-permission-iam-servi

The service account / user you use to run terraform needs permission to actAs the service account attached to the Cloud Run Service. --> you need to give that service account / user you use to run terraform the ServiceAccountUser permission.

[yanghuang1028]

Hi @maybellineboon @hostirosti ,

I used the service account "sa-aggregation-service@microsites-sa.iam.gserviceaccount.com" to run the terraform script, and actually the service accounts I used all have the ServiceAccountUser permission. However, it still threw the Permission 'iam.serviceaccounts.actAs' error, which is a bit wired...

[hostirosti]

Hi @yanghuang1028,

can you try to grant that permission for sa-aggregation-service@microsites-sa.iam.gserviceaccount.com directly on the 2 service accounts (worker-sa-aggregation-service@microsites-sa.iam.gserviceaccount.com and dev-frontend@microsites-sa.iam.gserviceaccount.com). You find the permission tab on the service account details page itself

Where you can grant access to specific principals:

I also noticed your gcloud config points to a different project (ecs-1709881683838). Is that intended?
You can update this with gcloud config set project microsites-sa

[yanghuang1028]

Hi @hostirosti ,
Because our company eBay.inc does not allow us to change account permissions ourselves, I asked our own GCP team to grant that permission again. But it still does not work for me ...

ecs-1709881683838 is our exact project id, while "microsites-sa" is a public project which all service accounts in our company are created under this project for managment easily.

[hostirosti]

Hi @yanghuang1028, you can ask for a consultation meeting through your partner manager contact on Google side. They'll schedule one.

To my knowledge, the service accounts used as part of the deployment need to be created in the project you deploy the aggregation service to. I did some non-extensive testing in this area in the past and was unsuccessful to use a service account created in a different project in some parts of the aggregation service (notably the worker service account, and I assume same for the frontend service account used by Cloud Run).

It looks like the way your company likes to organize / manage service accounts is not supported by GCP to run aggregation service.

[yanghuang1028]

Hi @hostirosti,

okay, thanks a lot !!

I'll contact with our partner manager to discuss about this. really thanks!!