-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Add tutorial for writing an event source - easy way #2494
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
093a1c1
ac3781b
ffe62e6
09c5102
8ebfd4d
87d1f91
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| FROM node:10 | ||
|
|
||
| # Create app directory | ||
| WORKDIR /usr/src/app | ||
|
|
||
| # Install app dependencies | ||
| # A wildcard is used to ensure both package.json AND package-lock.json are copied | ||
| # where available (npm@5+) | ||
| COPY package*.json ./ | ||
|
|
||
| RUN npm install | ||
| # If you are building your code for production | ||
| # RUN npm ci --only=production | ||
|
|
||
| # Bundle app source | ||
| COPY . . | ||
|
|
||
| EXPOSE 8080 | ||
| CMD [ "node", "index.js" ] | ||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,298 @@ | ||||||
| # Introduction | ||||||
|
|
||||||
| As stated in [tutorial on writing a Source with a Receive Adapter](../writing-receive-adapter-source/README.md), there are multiple ways to | ||||||
| create event sources. The way in that tutorial is to create an independent event source that has its own CRD. | ||||||
|
|
||||||
| In this tutorial though, you will build an event source in Javascript and use it with | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
|
||||||
| [ContainerSource](../../../eventing/sources/README.md#meta-sources) and / or [SinkBinding](../../../eventing/sources/README.md#meta-sources). | ||||||
|
|
||||||
| [ContainerSource](../../../eventing/sources/README.md#meta-sources) is an easy way to turn any dispatcher container into an Event Source. | ||||||
| Similarly, another option is using [SinkBinding](../../../eventing/sources/README.md#meta-sources) | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I thought we were recommending SinkBinding over ContainerSource at this point?
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added a sentence about the preferred way in #2512 |
||||||
| which provides a framework for injecting environment variables into any Kubernetes resource which has a `spec.template` that looks like a Pod (aka PodSpecable). | ||||||
|
|
||||||
| Code for this tutorial is available [here](./). | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think this link will work correctly on the website; but you can put in |
||||||
|
|
||||||
| # Bootstrapping | ||||||
|
|
||||||
| Create the project and add the dependencies: | ||||||
| ```bash | ||||||
| npm init | ||||||
| npm install cloudevents-sdk --save | ||||||
| ``` | ||||||
|
|
||||||
| ## Making use of ContainerSource | ||||||
|
|
||||||
| `ContainerSource` and `SinkBinding` both work by injecting environment variables to an application. | ||||||
| Injected environment variables at minimum contain the URL of a sink that will receive events. | ||||||
|
|
||||||
| Following example emits an event to the sink every 1000 milliseconds. | ||||||
| The sink URL to post the events will be made available to the application via the `K_SINK` environment variable by `ContainerSource`. | ||||||
|
|
||||||
| ```javascript | ||||||
| // File - index.js | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You may want to use the
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Let's do that later if necessary. The code I embedded in the tutorial is a bit shorter and more focused than the code in the files. |
||||||
|
|
||||||
| const v1 = require("cloudevents-sdk/v1"); | ||||||
|
|
||||||
| let sinkUrl = process.env['K_SINK']; | ||||||
|
|
||||||
| console.log("Sink URL is " + sinkUrl); | ||||||
|
|
||||||
| let config = { | ||||||
| method: "POST", | ||||||
| url: sinkUrl | ||||||
| }; | ||||||
|
|
||||||
| // The binding instance | ||||||
| let binding = new v1.BinaryHTTPEmitter(config); | ||||||
|
|
||||||
| let eventIndex = 0; | ||||||
| setInterval(function () { | ||||||
| console.log("Emitting event #" + ++eventIndex); | ||||||
|
|
||||||
| // create the event | ||||||
| let myevent = v1.event() | ||||||
| .id('your-event-id') | ||||||
| .type("your.event.source.type") | ||||||
| .source("urn:event:from:your-api/resource/123") | ||||||
| .dataContentType("application/json") | ||||||
| .data({"hello": "World " + eventIndex}); | ||||||
|
|
||||||
| // Emit the event | ||||||
| binding.emit(myevent) | ||||||
| .then(response => { | ||||||
| // Treat the response | ||||||
| console.log("Event posted successfully"); | ||||||
| console.log(response.data); | ||||||
| }) | ||||||
| .catch(err => { | ||||||
| // Deal with errors | ||||||
| console.log("Error during event post"); | ||||||
| console.error(err); | ||||||
| }); | ||||||
| }, 1000); | ||||||
| ``` | ||||||
|
|
||||||
| ```dockerfile | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. (Ditto) |
||||||
| # File - Dockerfile | ||||||
|
|
||||||
| FROM node:10 | ||||||
| WORKDIR /usr/src/app | ||||||
| COPY package*.json ./ | ||||||
| RUN npm install | ||||||
| COPY . . | ||||||
| EXPOSE 8080 | ||||||
| CMD [ "node", "index.js" ] | ||||||
|
|
||||||
| ``` | ||||||
|
|
||||||
| Build and push the image: | ||||||
| ```bash | ||||||
| docker build . -t path/to/image/registry/node-knative-heartbeat-source:v1 | ||||||
| docker push path/to/image/registry/node-knative-heartbeat-source:v1 | ||||||
| ``` | ||||||
|
|
||||||
| Create the event display service which simply logs any cloudevents posted to it. | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do you want to use Actually, that may not be necessary if this is always a fixed (published) image. Is
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||||||
| ```bash | ||||||
| cat <<EOS |kubectl apply -f - | ||||||
| --- | ||||||
| apiVersion: serving.knative.dev/v1 | ||||||
| kind: Service | ||||||
| metadata: | ||||||
| name: event-display | ||||||
| spec: | ||||||
| template: | ||||||
| spec: | ||||||
| containers: | ||||||
| - image: docker.io/aliok/event_display-864884f202126ec3150c5fcef437d90c@sha256:93cb4dcda8fee80a1f68662ae6bf20301471b046ede628f3c3f94f39752fbe08 | ||||||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. At the moment there's a bug in the latest published image of The bug is fixed but image is not published yet. Thus, I reference my own image here. This is to be updated later. @slinkydeveloper maybe you can add a link to the bug?
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not sure we have a link for that bug, but it was fixed between sdk-go RC1 and RC2 |
||||||
| EOS | ||||||
| ``` | ||||||
|
|
||||||
| Create the `ContainerSource`: | ||||||
| ```bash | ||||||
| cat <<EOS |kubectl apply -f - | ||||||
| --- | ||||||
| apiVersion: sources.knative.dev/v1alpha2 | ||||||
| kind: ContainerSource | ||||||
| metadata: | ||||||
| name: test-heartbeats | ||||||
| spec: | ||||||
| template: | ||||||
| spec: | ||||||
| containers: | ||||||
| - image: path/to/image/registry/node-knative-heartbeat-source:v1 | ||||||
| name: heartbeats | ||||||
| sink: | ||||||
| ref: | ||||||
| apiVersion: serving.knative.dev/v1 | ||||||
| kind: Service | ||||||
| name: event-display | ||||||
| EOS | ||||||
| ``` | ||||||
|
|
||||||
| Check the logs of the event display service. You will see a new message is pushed every second: | ||||||
| ```bash | ||||||
| $ kubectl logs event-display-dpplv-deployment-67c9949cf9-bvjvk -c user-container | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
This avoids need to look up the name of the pods before reading the logs.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. +1 |
||||||
|
|
||||||
| ☁️ cloudevents.Event | ||||||
| Validation: valid | ||||||
| Context Attributes, | ||||||
| specversion: 1.0 | ||||||
| type: your.event.source.type | ||||||
| source: urn:event:from:your-api/resource/123 | ||||||
| id: your-event-id | ||||||
| datacontenttype: application/json | ||||||
| Data, | ||||||
| { | ||||||
| "hello": "World 1" | ||||||
| } | ||||||
| ``` | ||||||
|
|
||||||
| If you are interested in to see what is injected into the event source, check the logs of it: | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
Not sure if you want to have the code dump out all the environment variables just for pedagogical purposes. |
||||||
| ```bash | ||||||
| $ kubectl logs test-heartbeats-deployment-7575c888c7-85w5t | ||||||
|
|
||||||
| Sink URL is http://event-display.default.svc.cluster.local | ||||||
| Emitting event #1 | ||||||
| Emitting event #2 | ||||||
| Event posted successfully | ||||||
| Event posted successfully | ||||||
| ``` | ||||||
|
|
||||||
| Please note that the example code above is using _Binary_ mode for CloudEvents. | ||||||
| Simply change | ||||||
| ```javascript | ||||||
| let binding = new v1.BinaryHTTPEmitter(config); | ||||||
| ``` | ||||||
| with | ||||||
| ```javascript | ||||||
| let binding = new v1.StructuredHTTPEmitter(config); | ||||||
| ``` | ||||||
| to employ structured mode. | ||||||
|
|
||||||
| However, binary mode should be used in most of the cases as: | ||||||
| - It is faster in terms of serialization and deserialization | ||||||
| - It works better with cloudevents-aware proxies (like Knative Channels) can simply check the header instead of parsing the payload | ||||||
|
|
||||||
| ## Making use of SinkBinding | ||||||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think we're recommending SinkBinding going forward, so I'd move this above ContainerSource (or maybe prefer to just document SinkBinding). @n3wscott to verify |
||||||
|
|
||||||
| `SinkBinding` is a more powerful way of making any Kubernetes resource an event source. | ||||||
|
|
||||||
| `ContainerSource` will create the container for your event source's image and it will be `ContainerSource` responsibility to manage the container. | ||||||
|
|
||||||
| `SinkBinding` though, will not create any containers. It will inject the sink information to the already existing Kubernetes resources. | ||||||
| This is a more flexible approach as you can use any Kubernetes `PodSpecable` as an event source, such as `Deployment`, `Job`, `Knative Service`, `DaemonSet` etc. | ||||||
|
|
||||||
| We don't need any code changes in our source for making it work with `SinkBinding`. | ||||||
|
|
||||||
| Create the event display as in the section before: | ||||||
| ```bash | ||||||
| cat <<EOS |kubectl apply -f - | ||||||
| --- | ||||||
| apiVersion: serving.knative.dev/v1 | ||||||
| kind: Service | ||||||
| metadata: | ||||||
| name: event-display | ||||||
| spec: | ||||||
| template: | ||||||
| spec: | ||||||
| containers: | ||||||
| - image: docker.io/aliok/event_display-864884f202126ec3150c5fcef437d90c@sha256:93cb4dcda8fee80a1f68662ae6bf20301471b046ede628f3c3f94f39752fbe08 | ||||||
| EOS | ||||||
| ``` | ||||||
|
|
||||||
| Create a Kubernetes deployment that runs the event source: | ||||||
| ```bash | ||||||
| cat <<EOS |kubectl apply -f - | ||||||
| --- | ||||||
| apiVersion: apps/v1 | ||||||
| kind: Deployment | ||||||
| metadata: | ||||||
| name: node-heartbeats-deployment | ||||||
| labels: | ||||||
| app: node-heartbeats | ||||||
| spec: | ||||||
| replicas: 2 | ||||||
| selector: | ||||||
| matchLabels: | ||||||
| app: node-heartbeats | ||||||
| template: | ||||||
| metadata: | ||||||
| labels: | ||||||
| app: node-heartbeats | ||||||
| spec: | ||||||
| containers: | ||||||
| - name: node-heartbeats | ||||||
| image: path/to/image/registry/node-knative-heartbeat-source:v1 | ||||||
aliok marked this conversation as resolved.
Show resolved
Hide resolved
|
||||||
| ports: | ||||||
| - containerPort: 8080 | ||||||
| EOS | ||||||
| ``` | ||||||
|
|
||||||
| As the `SinkBinding` is not created yet, `K_SINK` environment variable is not yet injected and the event source will complain about that. | ||||||
|
|
||||||
| ``` | ||||||
| $ kubectl logs node-heartbeats-deployment-9ffbb644b-llkzk | ||||||
|
|
||||||
| Sink URL is undefined | ||||||
| Emitting event #1 | ||||||
| Error during event post | ||||||
| TypeError [ERR_INVALID_ARG_TYPE]: The "url" argument must be of type string. Received type undefined | ||||||
| ``` | ||||||
|
|
||||||
| Create the `SinkBinding`: | ||||||
| ```bash | ||||||
| cat <<EOS |kubectl apply -f - | ||||||
| --- | ||||||
| apiVersion: sources.knative.dev/v1alpha1 | ||||||
| kind: SinkBinding | ||||||
| metadata: | ||||||
| name: bind-node-heartbeat | ||||||
| spec: | ||||||
| subject: | ||||||
| apiVersion: apps/v1 | ||||||
| kind: Deployment | ||||||
| selector: | ||||||
| matchLabels: | ||||||
| app: node-heartbeats | ||||||
| sink: | ||||||
| ref: | ||||||
| apiVersion: serving.knative.dev/v1 | ||||||
| kind: Service | ||||||
| name: event-display | ||||||
| EOS | ||||||
| ``` | ||||||
|
|
||||||
|
|
||||||
| You will see the pods are recreated and this time the `K_SINK` environment variable is injected. | ||||||
|
|
||||||
| Also note that since the `replicas` is set to 2, there will be 2 pods that are posting events to the sink. | ||||||
|
|
||||||
| ```bash | ||||||
| $ kubectl logs event-display-dpplv-deployment-67c9949cf9-bvjvk -c user-container | ||||||
|
|
||||||
| ☁️ cloudevents.Event | ||||||
| Validation: valid | ||||||
| Context Attributes, | ||||||
| specversion: 1.0 | ||||||
| type: your.event.source.type | ||||||
| source: urn:event:from:your-api/resource/123 | ||||||
| id: your-event-id | ||||||
| datacontenttype: application/json | ||||||
| Data, | ||||||
| { | ||||||
| "hello": "World 1" | ||||||
| } | ||||||
|
|
||||||
| ☁️ cloudevents.Event | ||||||
| Validation: valid | ||||||
| Context Attributes, | ||||||
| specversion: 1.0 | ||||||
| type: your.event.source.type | ||||||
| source: urn:event:from:your-api/resource/123 | ||||||
| id: your-event-id | ||||||
| datacontenttype: application/json | ||||||
| Data, | ||||||
| { | ||||||
| "hello": "World 1" | ||||||
| } | ||||||
| ``` | ||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| --- | ||
| title: "Writing an Event Source using the easy way" | ||
| linkTitle: "Event Source - easy way" | ||
| weight: 10 | ||
| type: "docs" | ||
| --- | ||
|
|
||
| {{% readfile file="README.md" %}} |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,58 @@ | ||
| const v1 = require("cloudevents-sdk/v1"); | ||
|
|
||
| let sinkUrl = process.env['K_SINK']; | ||
|
|
||
| console.log("Sink URL is " + sinkUrl); | ||
|
|
||
| let config = { | ||
| method: "POST", | ||
| url: sinkUrl | ||
| }; | ||
|
|
||
| // The binding instance | ||
| let binding = new v1.BinaryHTTPEmitter(config); | ||
|
|
||
| let eventIndex = 0; | ||
| setInterval(function () { | ||
| console.log("Emitting event #" + ++eventIndex); | ||
|
|
||
| // create the event | ||
| let myevent = v1.event() | ||
| .id('your-event-id') | ||
| .type("your.event.source.type") | ||
| .source("urn:event:from:your-api/resource/123") | ||
| .dataContentType("application/json") | ||
| .data({"hello": "World " + eventIndex}); | ||
|
|
||
| // Emit the event | ||
| binding.emit(myevent) | ||
| .then(response => { | ||
| // Treat the response | ||
| console.log("Event posted successfully"); | ||
| console.log(response.data); | ||
| }) | ||
| .catch(err => { | ||
| // Deal with errors | ||
| console.log("Error during event post"); | ||
| console.error(err); | ||
| }); | ||
| }, 1000); | ||
|
|
||
| registerGracefulExit(); | ||
|
|
||
| function registerGracefulExit() { | ||
| let logExit = function () { | ||
| console.log("Exiting"); | ||
| process.exit(); | ||
| }; | ||
|
|
||
| // handle graceful exit | ||
| //do something when app is closing | ||
| process.on('exit', logExit); | ||
| //catches ctrl+c event | ||
| process.on('SIGINT', logExit); | ||
| process.on('SIGTERM', logExit); | ||
| // catches "kill pid" (for example: nodemon restart) | ||
| process.on('SIGUSR1', logExit); | ||
| process.on('SIGUSR2', logExit); | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we're using node:10, this would be later than npm@5, right?
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Deleted in #2512