Overview

This code implements an approch to invoke a dataflow application through Cloud Run. Cloud Run accepts container images built with any tool capable of building container images, as long as they respect the container contract.In particular, your code must listen for HTTP requests on the port defined by the PORT environment variable. This PORT environment variable is automatically injected by Cloud Run into your container.

We can build docker image locally. To build your container image using Docker, use the follwing command:

Build and test image locally

export PROJECT_ID=[PROJECT-ID]
export  IMAGE=[IMAGE]
docker build . --tag gcr.io/${PROJECT-ID}/${IMAGE}

Download the service account for the dataflow appliction and set GOOGLE_APPLICATION_CREDENTIALS

export GOOGLE_APPLICATION_CREDENTIALS=/tmp/sa.json

Run locally and test

PORT=8080 && docker run \
 -p 8080:${PORT} \
 -e PORT=${PORT} \
 -e GOOGLE_APPLICATION_CREDENTIALS=/tmp/[SA_FOR_CR].json \
 -v $GOOGLE_APPLICATION_CREDENTIALS:/tmp/[SA_FOR_CR].json:ro \
   gcr.io/${PROJECT_ID}/${IMAGE}

Prepare the input parameters and test the dataflow application locally. Set all the variables defined in payload_template.json as environment locally and then generate required payload.

$  ( echo "cat <<EOF" ; cat payload_template.json ; echo EOF ) | sh >> payload.json

Run the following command. It prepares the input to invoke the REST endpoint. The value in messageId and publishTime are random values and will be generated by pubsub when integrated with PubSub.

$ payload_json=`cat payload.json`
$ (echo $payload_json | base64) >> payload_base64
$ printf -v data '{"message": 
        { "data": "%s", 
        "messageId": "12", 
        "publishTime": "201912312"
        }
      }' `cat payload_base64`

Finally, we can invoke the endpoint as shown below.

curl -X POST \
  http://localhost:8080/ \
  -H 'content-type: application/json' \
  -d "$data"

Deploy the Cloud Run

Push the code to Google container registry

gcloud auth configure-docker
docker push gcr.io/${PROJECT_ID}/${IMAGE}

And finally deploy the Cloud Run

gcloud run deploy demo-app \
--image gcr.io/${PROJECT_ID}/${IMAGE} \
--service-account cloud-run-sa@${PROJECT_ID}.iam.gserviceaccount.com \
--memory 500M \
--platform managed \
--no-allow-unauthenticated \
--region us-central1

Similarly, we can invoke the REST service using the Cloud Run Endpoint endpoint. We can configure a pubsub topic to push message to Cloud Run Endpoint. Please follow this article to hook PubSub to invoke Cloud Run. The following blog post explains the integration of cloud Run, Cloud PusbSub and Cloud scheduler.