Cloud Run Service outputs are missing after creation

[mikhailshilkov]

The URL is missing in the outputs, and refresh sometimes brings changes: https://github.com/pulumi/pulumi-gcp-native/runs/2297186491?check_suite_focus=true#step:20:126

[steren]

This is expected from the perspective of the Cloud Run API, which matches the Knative API spec. It is expected for clients to poll the resource and look for updates under status. The URL might be missng at first and only be populated once the URL has been attributed to the service.

[ColeSiegelTR]

I'm running into this problem, I expected that using the output value from status would provide me the URL but it is always undefined. I've been trying to work around it by running a gcloud CLI or SDK command to retrieve the status after deployment. I'm not sure how I would go about implementing this though, as I need to create other Pulumi resources afterward in the same stack using that URL. Any suggestions @mikhailshilkov ?

I was hoping to use it like the tutorial here https://www.pulumi.com/blog/google-cloud-run-serverless-containers/

[mikhailshilkov]

@ColeSiegelTR Do the outputs appear if you run pulumi refresh?

[ColeSiegelTR]

@mikhailshilkov No, that whole status section is missing on pulumi refresh also

 ~   └─ google-native:run/v1:Service                            cloudrun_service                                                                 updated     [diff: ~metadata,status]
 
Outputs:

    service: {
        apiVersion: "serving.knative.dev/v1"
        id        : "v1/projects/xxx/locations/us-east4/services/xxx"
        kind      : "Service"
        location  : "us-east4"
        metadata  : {
            annotations      : {
                run.googleapis.com/ingress      : "all"
                serving.knative.dev/creator     : "xxx"
                serving.knative.dev/lastModifier: "xxx"
            }
            creationTimestamp: "2021-11-04T15:59:14.860761Z"
            generation       : 1
            labels           : {
                cloud.googleapis.com/location: "us-east4"
            }
            name             : "xxx"
            namespace        : "xxxx"
            resourceVersion  : "AAXP+Ju3Gtk"
            selfLink         : "/apis/serving.knative.dev/v1/namespaces/xxxx/services/xxx"
            uid              : "b0a765ea-d82d-4e62-b9eb-5b455672702a"
        }
        project   : "xxx"
        spec      : {
            template: {
                metadata: {
                    annotations: {
                        autoscaling.knative.dev/maxScale: "1"
                    }
                }
                spec    : {
                    containerConcurrency: 1
                    containers          : [
                        [0]: {
                            env         : [
                                [0]: {
                                    name : "EXTRACT_API_URL_BASE"
                                    value: "xxx"
                                }
                                [1]: {
                                    name : "EXTRACT_GCS_PATH_STAGING"
                                    value: "xxx"
                                }
                                [2]: {
                                    name : "EXTRACT_GCS_PATH_RESULTS"
                                    value: "xxx"
                                }
                                [3]: {
                                    name : "EXTRACT_BQ_TABLE_RAW"
                                    value: ""
                                }
                                [4]: {
                                    name : "EXTRACT_BQ_TABLE_RESULTS"
                                    value: ""
                                }
                                [5]: {
                                    name : "EXTRACT_API_USERNAME_FILE"
                                    value: ""
                                }
                                [6]: {
                                    name : "EXTRACT_API_PASSWORD_FILE"
                                    value: ""
                                }
                            ]
                            image       : "gcr.io/xxx/xxx:latest"
                            ports       : [
                                [0]: {
                                    containerPort: 8080
                                    name         : "http1"
                                }
                            ]
                            resources   : {
                                limits: {
                                    cpu   : "1"
                                    memory: "4Gi"
                                }
                            }
                            volumeMounts: [
                                [0]: {
                                    mountPath: "/run/secrets/extract/username"
                                    name     : "xxx"
                                    readOnly : true
                                }
                                [1]: {
                                    mountPath: "/run/secrets/extract/password"
                                    name     : "xxx"
                                    readOnly : true
                                }
                            ]
                        }
                    ]
                    serviceAccountName  : "xxx@xxx.iam.gserviceaccount.com"
                    timeoutSeconds      : 3600
                    volumes             : [
                        [0]: {
                            name  : "xxx"
                            secret: {
                                items     : [
                                    [0]: {
                                        key : "latest"
                                        path: "file"
                                    }
                                ]
                                secretName: "xxx"
                            }
                        }
                        [1]: {
                            name  : "xxx"
                            secret: {
                                items     : [
                                    [0]: {
                                        key : "latest"
                                        path: "file"
                                    }
                                ]
                                secretName: "xxx"
                            }
                        }
                    ]
                }
            }
            traffic : [
                [0]: {
                    latestRevision: true
                    percent       : 100
                }
            ]
        }
        urn       : "xxx"
    }

[mikhailshilkov]

This is surprising. I've just run this example: https://github.com/pulumi/pulumi-google-native/blob/master/examples/cloudrun-ts/index.ts and got

"apiVersion": "serving.knative.dev/v1",
"kind": "Service",
"metadata": {
    "annotations": {
        "run.googleapis.com/ingress": "all",
        "run.googleapis.com/ingress-status": "all"
    },
    "creationTimestamp": "2021-11-05T14:33:07.236749Z",
    "generation": 1,
    "labels": {
        "cloud.googleapis.com/location": "us-central1"
    },
    "name": "run-123",
    "namespace": "921927215178",
    "resourceVersion": "AAXQC4YLkFQ",
    "selfLink": "/apis/serving.knative.dev/v1/namespaces/921927215178/services/run-123",
    "uid": "b8fdc2d0-8669-4742-8427-78fc4bc12b6f"
},
"spec": {
    "template": {
        "metadata": {
            "annotations": {
                "autoscaling.knative.dev/maxScale": "100"
            }
        },
        "spec": {
            "containerConcurrency": 80,
            "containers": [
                {
                    "image": "gcr.io/cloudrun/hello",
                    "ports": [
                        {
                            "containerPort": 8080,
                            "name": "http1"
                        }
                    ],
                    "resources": {
                        "limits": {
                            "cpu": "1000m",
                            "memory": "512Mi"
                        }
                    }
                }
            ],
            "timeoutSeconds": 300
        }
    },
    "traffic": [
        {
            "latestRevision": true,
            "percent": 100
        }
    ]
},
"status": {
    "address": {
        "url": "https://run-123-q5wszdxb2a-uc.a.run.app"
    },
    "conditions": [
        {
            "lastTransitionTime": "2021-11-05T14:33:15.675732Z",
            "status": "True",
            "type": "Ready"
        },
        {
            "lastTransitionTime": "2021-11-05T14:33:15.303386Z",
            "status": "True",
            "type": "ConfigurationsReady"
        },
        {
            "lastTransitionTime": "2021-11-05T14:33:15.675732Z",
            "status": "True",
            "type": "RoutesReady"
        }
    ],
    "latestCreatedRevisionName": "run-123-2lrgz",
    "latestReadyRevisionName": "run-123-2lrgz",
    "observedGeneration": 1,
    "traffic": [
        {
            "latestRevision": true,
            "percent": 100,
            "revisionName": "run-123-2lrgz"
        }
    ],
    "url": "https://run-123-q5wszdxb2a-uc.a.run.app"
}

Could you try this simple example?

[ColeSiegelTR]

That is strange... even with that exact example I don't get the status back in the output with everything else. I can retrieve the status by API, so I don't think it's an access issue. Any suggestions?

edit @mikhailshilkov

I tested a bunch more, sometimes it shows the output URL but its very inconsistent. I think the state management might be messed up for this resource? One possible cause, I realized that often the service gets re-created simply because some auto-generated properties have changed - maybe because it's newly replaced it doesn't have a status anymore? In any case, I'm definitely not able to create a cloud run service and another resource which uses status.url in the same stack - even if I run pulumi refresh. The reference to the URL is empty and I end up with errors google-native:cloudscheduler/v1:Job (schedule_job): error: error sending request: googleapi: Error 400: URL cannot be empty. Please check the entry below and note the replaced [diff: -status~metadata] maybe it has something to do with the root cause.

 +-  └─ google-native:run/v1:Service                            cloudrun_service                                                                 replaced     [diff: -status~metadata]
 
Outputs:
  ~ serviceDetails: {
        apiVersion: "serving.knative.dev/v1"
        id        : "v1/projects/x/locations/us-east4/services/x-content-x-x"
        kind      : "Service"
        location  : "us-east4"
      ~ metadata  : {
            annotations      : {
                run.googleapis.com/ingress      : "all"
                serving.knative.dev/creator     : "cole.siegel@x.com"
                serving.knative.dev/lastModifier: "cole.siegel@x.com"
            }
          ~ creationTimestamp: "2021-11-05T20:24:46.910184Z" => "2021-11-05T20:29:14.486875Z"
            generation       : 1
            labels           : {
                cloud.googleapis.com/location: "us-east4"
            }
            name             : "idt-content-expedia-extract"
            namespace        : "262388767350"
          ~ resourceVersion  : "AAXQEG8uQug" => "AAXQEH8hKFs"
            selfLink         : "/apis/serving.knative.dev/v1/namespaces/262388767350/services/x-content-x-x"
          ~ uid              : "e8042aea-e4b1-49a8-bf55-747180259070" => "0f769e69-acd7-4bb8-989b-e44974a70198"
        }

[ColeSiegelTR]

In the mean time, I've been able to work-around this issue using the code below to fetch the service after it is created.

cloudRunService.id.apply(id => {
    const scheduler = setScheduler([cloudScheduler, service]);
});

export async function setScheduler(dependencies: pulumi.Resource[]) {
    const getService = await gcp.run.v1.getService({
        location: config.region,
        project: config.project.id,
        serviceId: config.service.name
    });

    return new gcp.cloudscheduler.v1.Job('schedule_job', {
        location: config.region,
        project: config.project.id,
        name: `projects/${config.project.id}/locations/${config.region}/jobs/${config.name}`,
        schedule: config.schedule,
        httpTarget: {
            httpMethod: 'GET',
            uri: getService.status.url,
            oidcToken: {
                serviceAccountEmail: `${config.serviceAccount.name}@${config.project.id}.iam.gserviceaccount.com`,
                audience: getService.status.url
            }
        }
    }, {
        dependsOn: dependencies
    });
}

[viveklak]

I believe this should be fixed in #411 (0.18.1). Please reopen if this issue persists.

[vschettino-asimov]

Hey! I'm getting the following error message when I try to pulumi refresh an stack containing a google-native:compute/v1:SslCertificate:

    error: Preview failed: selfLink property "selfLink" not found

Perhaps the issue is related to this one?

[hcharley]

@vschettino-asimov I found this ticket as well after searching for a similar issue with a pulumi preview containing a stack with a google-native:sqladmin/v1beta4:Database:

  Diagnostics:
    google-native:sqladmin/v1beta4:Database (my-database):
      error: Preview failed: selfLink property "selfLink" not found
   
    pulumi:pulumi:Stack (proj-stack):
      error: preview failed
   
  
  
   stderr: 
   err?: