Skip to content

Latest commit

 

History

History
185 lines (140 loc) · 6.21 KB

README.md

File metadata and controls

185 lines (140 loc) · 6.21 KB
Raw

This sample shows how to configure the GCP PubSub event source. This event source is most useful as a bridge from other GCP services, such as Cloud Storage, IoT Core and Cloud Scheduler.

Prerequisites

  1. Create a Google Cloud project and install the gcloud CLI and run gcloud auth login. This sample will use a mix of gcloud and kubectl commands. The rest of the sample assumes that you've set the $PROJECT_ID environment variable to your Google Cloud project id, and also set your project ID as default using gcloud config set project $PROJECT_ID.

  2. Setup Knative Serving

  3. Setup Knative Eventing. In addition, install the GCP PubSub event source from release-gcppubsub.yaml:

    kubectl apply --filename kubectl apply --filename https://github.com/knative/eventing-sources/releases/download/v0.3.0/release-gcppubsub.yaml

  4. Enable the Cloud Pub/Sub API on your project:

    gcloud services enable pubsub.googleapis.com

  5. Create a GCP Service Account. This sample creates one service account for both registration and receiving messages, but you can also create a separate service account for receiving messages if you want additional privilege separation.

    1. Create a new service account named knative-source with the following command:

      gcloud iam service-accounts create knative-source

    2. Give that Service Account the Pub/Sub Editor role on your GCP project:

      gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member=serviceAccount:knative-source@$PROJECT_ID.iam.gserviceaccount.com \
  --role roles/pubsub.editor

    3. Download a new JSON private key for that Service Account. Be sure not to check this key into source control!

      gcloud iam service-accounts keys create knative-source.json \
  --iam-account=knative-source@$PROJECT_ID.iam.gserviceaccount.com

    4. Create two secrets on the kubernetes cluster with the downloaded key:

      # Note that the first secret may already have been created when installing
# Knative Eventing. The following command will overwrite it. If you don't
# want to overwrite it, then skip this command.
kubectl -n knative-sources create secret generic gcppubsub-source-key --from-file=key.json=knative-source.json --dry-run -o yaml | kubectl apply --filename -

# The second secret should not already exist, so just try to create it.
kubectl -n default create secret generic google-cloud-key --from-file=key.json=knative-source.json

      gcppubsub-source-key and key.json are pre-configured values in the controller-manager StatefulSet which manages your Eventing sources.

      google-cloud-key and key.json are pre-configured values in gcp-pubsub-source.yaml.

Deployment

  1. Create a Channel. This example creates a Channel called pubsub-test which uses the in-memory provisioner, with the following definition:

    apiVersion: eventing.knative.dev/v1alpha1
kind: Channel
metadata:
  name: pubsub-test
spec:
  provisioner:
    apiVersion: eventing.knative.dev/v1alpha1
    kind: ClusterChannelProvisioner
    name: in-memory-channel

    If you're in the samples directory, you can apply the channel.yaml file:

    kubectl apply --filename channel.yaml

  2. Create a GCP PubSub Topic. If you change its name (testing), you also need to update the topic in the gcp-pubsub-source.yaml file:

    gcloud pubsub topics create testing

  3. Replace the MY_GCP_PROJECT placeholder in gcp-pubsub-source.yaml and apply it.

    If you're in the samples directory, you can replace MY_GCP_PROJECT and apply in one command:

     sed "s/MY_GCP_PROJECT/$PROJECT_ID/g" gcp-pubsub-source.yaml | \
     kubectl apply --filename -

    If you are replacing MY_GCP_PROJECT manually, then make sure you apply the resulting YAML:

    kubectl apply --filename gcp-pubsub-source.yaml

  4. Create a function and subscribe it to the pubsub-test channel:

    kubectl apply --filename subscriber.yaml

Publish

Publish messages to your GCP PubSub Topic:

gcloud pubsub topics publish testing --message="Hello world"

Verify

We will verify that the published message was sent into the Knative eventing system by looking at what is downstream of the GcpPubSubSource. If you deployed the Subscriber, then continue using this section. If not, then you will need to look downstream yourself.

  1. We need to wait for the downstream pods to get started and receive our event, wait 60 seconds.

    • You can check the status of the downstream pods with:

      kubectl get pods --selector serving.knative.dev/service=message-dumper

      You should see at least one.

  2. Inspect the logs of the subscriber:

    kubectl logs --selector serving.knative.dev/service=message-dumper -c user-container

You should see log lines similar to:

{
  "ID": "284375451531353",
  "Data": "SGVsbG8sIHdvcmxk",
  "Attributes": null,
  "PublishTime": "2018-10-31T00:00:00.00Z"
}

The log message is a dump of the message sent by GCP PubSub. In particular, if you base-64 decode the Data field, you should see the sent message:

echo "SGVsbG8sIHdvcmxk" | base64 --decode

Results in: "Hello world"

For more information about the format of the message, see the PubsubMessage documentation.