diff --git a/docs/eventing/broker-trigger.md b/docs/eventing/broker-trigger.md index ae605f07450..31e7266bab5 100644 --- a/docs/eventing/broker-trigger.md +++ b/docs/eventing/broker-trigger.md @@ -15,27 +15,63 @@ a Broker, all metadata other than the CloudEvent is stripped away (e.g. unless set as a CloudEvent attribute, there is no concept of how this event entered the Broker). +There can be different classes of Brokers providing different kinds +of semantics around durability of events, performance, etc. The Broker that is +part of the Knative Eventing repo is used for these examples, it uses Knative +[Channels](./channels/) for delivering events. You can read more details about +[Channel Based Broker](./channel-based-broker.md). Simple example showing a `Broker` +where the configuration is specified in a `ConfigMap` config-br-default-channel, +which uses `InMemoryChannel`: + Example: ```yaml -apiVersion: eventing.knative.dev/v1alpha1 +apiVersion: eventing.knative.dev/v1beta1 +kind: Broker +metadata: + name: default +spec: + # Configuration specific to this broker. + config: + apiVersion: v1 + kind: ConfigMap + name: config-br-default-channel + namespace: knative-eventing +``` + +More complex example, showing the same `Broker` as above +but with failed events being delivered to Knative Service called `dlq-service` + +```yaml +apiVersion: eventing.knative.dev/v1beta1 kind: Broker metadata: name: default spec: - channelTemplateSpec: - apiVersion: messaging.knative.dev/v1alpha1 - kind: InMemoryChannel + # Configuration specific to this broker. + config: + apiVersion: v1 + kind: ConfigMap + name: config-br-default-channel + namespace: knative-eventing + # Where to deliver Events that failed to be processed. + delivery: + deadLetterSink: + ref: + apiVersion: serving.knative.dev/v1 + kind: Service + name: dlq-service ``` ## Trigger A Trigger represents a desire to subscribe to events from a specific Broker. -Example: +Simple example which will receive all the events from the given (`default`) broker and +deliver them to Knative Serving service `my-service`: ```yaml -apiVersion: eventing.knative.dev/v1alpha1 +apiVersion: eventing.knative.dev/v1beta1 kind: Trigger metadata: name: my-service-trigger @@ -57,7 +93,7 @@ Note that we only support exact matching on string values. Example: ```yaml -apiVersion: eventing.knative.dev/v1alpha1 +apiVersion: eventing.knative.dev/v1beta1 kind: Trigger metadata: name: my-service-trigger @@ -77,146 +113,17 @@ spec: The example above filters events from the `default` Broker that are of type `dev.knative.foo.bar` AND have the extension `myextension` with the value `my-extension-value`. -## Usage - -### Channel - -`Broker`s use their `spec.channelTemplateSpec` to create their internal -[Channels](./channels/), which dictate the durability guarantees of events sent -to that `Broker`. If `spec.channelTemplateSpec` is not specified, then the -[default channel](./channels/default-channels.md) for their namespace is used. - -#### Setup - -Have a `Channel` CRD installed and set as the default channel for the namespace -you are interested in. For development, the -[InMemoryChannel](https://github.com/knative/eventing/tree/{{< branch >}}/config/channels/in-memory-channel) -is normally used. - -#### Changing - -**Note** changing the `Channel` of a running `Broker` will lose all in-flight -events. - -If you want to change which `Channel` is used by a given `Broker`, then -determine if the `spec.channelTemplateSpec` is specified or not. - -If `spec.channelTemplateSpec` is specified: - -1. Delete the `Broker`. -1. Create the `Broker` with the updated `spec.channelTemplateSpec`. - -If `spec.channelTemplateSpec` is not specified: - -1. Change the - [default channel](./channels/default-channels.md#setting-the-default-channel-configuration) - for the namespace that `Broker` is in. -1. Delete and recreate the `Broker`. - -### Broker - -There are two ways to create a Broker: - -- [namespace annotation](#annotation) -- [manual setup](#manual-setup) - -Normally the [namespace annotation](#annotation) is used to do this setup. - -#### Annotation - -The easiest way to get started, is to annotate your namespace (replace `default` -with the desired namespace): - -```shell -kubectl label namespace default knative-eventing-injection=enabled -``` - -This should automatically create a `Broker` named `default` in the `default` -namespace. - -```shell -kubectl -n default get broker default -``` - -_NOTE_ `Broker`s created due to annotation will not be removed if you remove the -annotation. For example, if you annotate the namespace, which will then create -the `Broker` as described above. If you now remove the annotation, the `Broker` -will not be removed, you have to manually delete it. - -For example, to delete the injected Broker from the foo namespace: - -```shell -kubectl -n foo delete broker default -``` - -#### Manual Setup - -In order to setup a `Broker` manually, we must first create the required -`ServiceAccount`s and give them the proper RBAC permissions. This setup is -required once per namespace. These instructions will use the `default` -namespace, but you can replace it with any namespace you want to install a -`Broker` into. - -Create the `ServiceAccount` objects. - -```shell -kubectl -n default create serviceaccount eventing-broker-ingress -kubectl -n default create serviceaccount eventing-broker-filter -``` - -Then give them the needed RBAC permissions: - -```shell -kubectl -n default create rolebinding eventing-broker-ingress \ - --clusterrole=eventing-broker-ingress \ - --serviceaccount=default:eventing-broker-ingress -kubectl -n default create rolebinding eventing-broker-filter \ - --clusterrole=eventing-broker-filter \ - --serviceaccount=default:eventing-broker-filter -``` +## Complete end-to-end example -Note that these commands each use three different objects, all named -`eventing-broker-ingress` or `eventing-broker-filter`. The `ClusterRole` is -installed with Knative Eventing -[here](https://github.com/knative/eventing/blob/master/config/200-broker-clusterrole.yaml). -The `ServiceAccount` was created two commands prior. The `RoleBinding` is -created with this command. - -Create RBAC permissions granting access to shared configmaps for logging, -tracing, and metrics configuration. - -_These commands assume the shared Knative Eventing components are installed in -the `knative-eventing` namespace. If you installed the shared Knative Eventing -components in a different namespace, replace `knative-eventing` with the name of -that namespace._ - -```shell -kubectl -n knative-eventing create rolebinding eventing-config-reader-default-eventing-broker-ingress \ - --clusterrole=eventing-config-reader \ - --serviceaccount=default:eventing-broker-ingress -kubectl -n knative-eventing create rolebinding eventing-config-reader-default-eventing-broker-filter \ - --clusterrole=eventing-config-reader \ - --serviceaccount=default:eventing-broker-filter -``` - -Now we can create the `Broker`. Note that this example uses the name `default`, -but could be replaced by any other valid name. +### Broker setup -```shell -cat << EOF | kubectl apply -f - -apiVersion: eventing.knative.dev/v1alpha1 -kind: Broker -metadata: - namespace: default - name: default -EOF -``` +We assume that you have installed a Broker in namespace `default`. If you haven't done that +yet, [install it from here](./channel-based-broker.md). ### Subscriber -Now create a function to receive those events. This document will assume the -following manifest describing a Knative Service is created, but it could be -anything that is `Addressable`. +Create a function to receive events. This document uses a Knative Service, but +it could be anything that is [Callable](https://github.com/knative/eventing/blob/master/docs/spec/interfaces.md). ```yaml apiVersion: serving.knative.dev/v1 @@ -229,17 +136,18 @@ spec: spec: containers: - # This corresponds to - # https://github.com/knative/eventing-contrib/blob/v0.2.1/cmd/message_dumper/dumper.go. - image: gcr.io/knative-releases/github.com/knative/eventing-sources/cmd/message_dumper@sha256:ab5391755f11a5821e7263686564b3c3cd5348522f5b31509963afb269ddcd63 + # https://github.com/knative/eventing-contrib/tree/master/cmd/event_display + - image: gcr.io/knative-releases/knative.dev/eventing-contrib/cmd/event_display@sha256:a214514d6ba674d7393ec8448dd272472b2956207acb3f83152d3071f0ab1911 ``` ### Trigger -Create a `Trigger` using the following manifest that sends only events of a -particular type to `my-service`: +Create a `Trigger` that sends only events of a particular type to the subscriber +created above (`my-service`). For this example, we use Ping Source, and it +emits events types `dev.knative.sources.ping`. ```yaml -apiVersion: eventing.knative.dev/v1alpha1 +apiVersion: eventing.knative.dev/v1beta1 kind: Trigger metadata: name: my-service-trigger @@ -247,7 +155,7 @@ metadata: spec: filter: attributes: - type: dev.knative.foo.bar + type: dev.knative.sources.ping subscriber: ref: apiVersion: serving.knative.dev/v1 @@ -263,7 +171,7 @@ unspecified. The Webhook will default the YAML above to: ```yaml -apiVersion: eventing.knative.dev/v1alpha1 +apiVersion: eventing.knative.dev/v1beta1 kind: Trigger metadata: name: my-service-trigger @@ -272,7 +180,7 @@ spec: broker: default # Defaulted by the Webhook. filter: attributes: - type: dev.knative.foo.bar + type: dev.knative.sources.ping subscriber: ref: apiVersion: serving.knative.dev/v1 @@ -283,62 +191,24 @@ spec: You can make multiple `Trigger`s on the same `Broker` corresponding to different types, sources (or any other CloudEvents attribute), and subscribers. -### Source - -Now have something emit an event of the correct type (`dev.knative.foo.bar`) -into the `Broker`. We can either do this manually or with a normal Knative -Source. - -#### Manual - -The `Broker`'s address is well known, it will always be -`-broker..svc.`. In our case, it is -`default-broker.default.svc.cluster.local`. +### Emitting Events using Ping Source -While SSHed into a `Pod` and run: - -```shell -curl -v "http://default-broker.default.svc.cluster.local/" \ - -X POST \ - -H "X-B3-Flags: 1" \ - -H "CE-SpecVersion: 0.2" \ - -H "CE-Type: dev.knative.foo.bar" \ - -H "CE-Time: 2018-04-05T03:56:24Z" \ - -H "CE-ID: 45a8b444-3213-4758-be3f-540bf93f85ff" \ - -H "CE-Source: dev.knative.example" \ - -H 'Content-Type: application/json' \ - -d '{ "much": "wow" }' -``` - -#### Knative Source - -Provide the Knative Source the `default` `Broker` as its sink (note you'll need -to use ko apply -f .yaml to create it): +Knative Eventing comes with a [Ping Source](./samples/ping-source/README.md) which +emits an event on a configured schedule. For this we'll configure it to emit +events once a minute, saying, yes, you guessed it `Hello World!`. ```yaml -apiVersion: sources.eventing.knative.dev/v1alpha1 -kind: ContainerSource +apiVersion: sources.knative.dev/v1alpha1 +kind: PingSource metadata: - name: heartbeats-sender + name: test-ping-source spec: - template: - spec: - containers: - - image: github.com/knative/eventing-contrib/cmd/heartbeats/ - name: heartbeats-sender - args: - - --eventType=dev.knative.foo.bar - env: - - name: POD_NAME - valueFrom: - fieldRef: - fieldPath: metadata.name - - name: POD_NAMESPACE - valueFrom: - fieldRef: - fieldPath: metadata.namespace + schedule: "*/1 * * * *" + data: '{"message": "Hello world!"}' sink: - apiVersion: eventing.knative.dev/v1alpha1 - kind: Broker - name: default + ref: + # Deliver events to Broker. + apiVersion: eventing.knative.dev/v1alpha1 + kind: Broker + name: default ``` diff --git a/docs/eventing/channel-based-broker.md b/docs/eventing/channel-based-broker.md new file mode 100644 index 00000000000..d3f9e6710c4 --- /dev/null +++ b/docs/eventing/channel-based-broker.md @@ -0,0 +1,215 @@ +--- +title: "Channel Based Broker" +weight: 30 +type: "docs" +--- + +# Channel Based Broker + +Knative provides a `Broker` implementation that uses [Channels](./channels/) for +event routing. You will need to have a Channel provider installed, for example +InMemoryChannel (for development purposes), Kafka, Nats, etc. You can choose from +list of [available channels](https://knative.dev/docs/eventing/channels/channels-crds/) + +Once you have decided which Channel(s) you want to use and have installed them, you +can configure the Broker by controlling which Channel(s) are used. You can choose +this as a cluster level default, by namespace or by a specific Broker. These are +configured by a `config-br-defaults` `ConfigMap` in knative-eventing namespace. + +Here's an example of a configuration that uses Kafka channel for all the +Brokers except namespace `test-broker-6` which uses InMemoryChannels. First +define the `ConfigMap`s to describe how the Channels of each type are created: + +```yaml +# Define how InMemoryChannels are created +apiVersion: v1 +kind: ConfigMap +metadata: + namespace: knative-eventing + name: imc-channel +data: + channelTemplateSpec: | + apiVersion: messaging.knative.dev/v1alpha1 + kind: InMemoryChannel +``` + +```yaml +# Define how Kafka channels are created. Note we specify +# extra parameters that are particular to Kakfa Channels, namely +# numPartitions as well as replicationFactor. +apiVersion: v1 +kind: ConfigMap +metadata: + name: kafka-channel + namespace: knative-eventing +data: + channelTemplateSpec: | + apiVersion: messaging.knative.dev/v1alpha1 + kind: KafkaChannel + spec: | + numPartitions: 3 + replicationFactor: 1 +``` + +```yaml +kind: ConfigMap +apiVersion: v1 +metadata: + name: config-br-defaults + namespace: knative-eventing +data: + default-br-config: | + clusterDefault: + apiVersion: v1 + kind: ConfigMap + name: imc-channel + namespace: knative-eventing + namespaceDefaults: + test-broker-6: + apiVersion: v1 + kind: ConfigMap + name: kafka-channel + namespace: knative-eventing +``` + + +## Installing Broker by Annotation + +The easiest way to get Broker installed, is to annotate your namespace +(replace `default` with the desired namespace): + +```shell +kubectl label namespace default knative-eventing-injection=enabled +``` + +This will automatically create a `Broker` named `default` in the `default` +namespace. As per above configuration, it would be configured to use Kafka +channels. + +```shell +kubectl -n default get broker default +``` + +_NOTE_ `Broker`s created due to annotation will not be removed if you remove the +annotation. For example, if you annotate the namespace, which will then create +the `Broker` as described above. If you now remove the annotation, the `Broker` +will not be removed, you have to manually delete it. + +For example, to delete the injected Broker from the foo namespace: + +```shell +kubectl -n foo delete broker default +``` + +## Installing Broker Manually + +In order to setup a `Broker` manually, we must first create the required +`ServiceAccount`s and give them the proper RBAC permissions. This setup is +required once per namespace. These instructions will use the `default` +namespace, but you can replace it with any namespace you want to install a +`Broker` into. + +Create the `ServiceAccount` objects. + +```shell +kubectl -n default create serviceaccount eventing-broker-ingress +kubectl -n default create serviceaccount eventing-broker-filter +``` + +Then give them the needed RBAC permissions: + +```shell +kubectl -n default create rolebinding eventing-broker-ingress \ + --clusterrole=eventing-broker-ingress \ + --serviceaccount=default:eventing-broker-ingress +kubectl -n default create rolebinding eventing-broker-filter \ + --clusterrole=eventing-broker-filter \ + --serviceaccount=default:eventing-broker-filter +``` + +Note that these commands each use three different objects, all named +`eventing-broker-ingress` or `eventing-broker-filter`. The `ClusterRole` is +installed with Knative Eventing +[here](https://github.com/knative/eventing/blob/master/config/200-broker-clusterrole.yaml). +The `ServiceAccount` was created two commands prior. The `RoleBinding` is +created with this command. + +Create RBAC permissions granting access to shared configmaps for logging, +tracing, and metrics configuration. + +_These commands assume the shared Knative Eventing components are installed in +the `knative-eventing` namespace. If you installed the shared Knative Eventing +components in a different namespace, replace `knative-eventing` with the name of +that namespace._ + +```shell +kubectl -n knative-eventing create rolebinding eventing-config-reader-default-eventing-broker-ingress \ + --clusterrole=eventing-config-reader \ + --serviceaccount=default:eventing-broker-ingress +kubectl -n knative-eventing create rolebinding eventing-config-reader-default-eventing-broker-filter \ + --clusterrole=eventing-config-reader \ + --serviceaccount=default:eventing-broker-filter +``` + +Now we can create the `Broker`. Note that this example uses the name `default`, +but could be replaced by any other valid name. This example uses the defaults +for Channel that we configured above in our `config-br-defaults` `ConfigMap`, +and hence would use Kafka Channels. + +```shell +cat << EOF | kubectl apply -f - +apiVersion: eventing.knative.dev/v1beta1 +kind: Broker +metadata: + namespace: default + name: default +EOF +``` + +If you wanted to explicitly specify the Configuration of the Broker, you could do +so by using spec.config, like so (manually overriding it to use InMemoryChannels). + +```shell +cat << EOF | kubectl apply -f - +apiVersion: eventing.knative.dev/v1beta1 +kind: Broker +metadata: + namespace: default + name: broker-2 +spec: + config: + apiVersion: v1 + kind: ConfigMap + name: imc-channel + namespace: knative-eventing +EOF +``` + +## Handling failed event delivery + +Broker allows you to specify what to do in the case if failed event delivery, say +you have a consumer (function for example) that's failing to process the event. +You can use `delivery` for it: + +```yaml +apiVersion: eventing.knative.dev/v1beta1 +kind: Broker +metadata: + name: default +spec: + config: + apiVersion: v1 + kind: ConfigMap + namespace: knative-eventing + name: imc-channel + # Deliver failed events here + delivery: + deadLetterSink: + ref: + apiVersion: serving.knative.dev/v1 + kind: Service + name: error-handler +``` + +You can find out more about delivery spec details [here](https://knative.dev/docs/eventing/event-delivery/). + diff --git a/docs/eventing/channels/default-channels.md b/docs/eventing/channels/default-channels.md index cf1bb88e021..79ee22becfc 100644 --- a/docs/eventing/channels/default-channels.md +++ b/docs/eventing/channels/default-channels.md @@ -11,16 +11,15 @@ be created without specifying an underlying implementation. This is useful for users that do not care about the properties a particular channel provides (e.g., ordering, persistence, etc.), but are rather fine with using the implementation selected by the the operator. The operator controls the default -settings via a `ConfigMap`. For example, when `Brokers` or `Sequences` are -created, an operator can configure a default channel to use for their underlying -channel-based implementations. +settings via a `ConfigMap`. For example, an operator can configure which channels +to use when `Channel Based Brokers` or `Sequences` are created. Even though this default channel mechanism aims to ease the usability of the system, users can still create their own channels directly by instantiating one of the supported channel objects (e.g., `InMemoryChannel`, `KafkaChannel`, etc.). -## Setting the default channel configuration +## Setting the default channel configuration for messaging layer The default channel configuration is specified in the `ConfigMap` named `default-ch-webhook` in the `knative-eventing` namespace. This `ConfigMap` may @@ -107,6 +106,6 @@ underlying `KafkaChannel` instead (i.e., the default for that namespace). ## Defaults only apply on object creation -Defaults are applied by the webhook only on `Channel`, `Broker`, or `Sequence` +Defaults are applied by the webhook only on `Channel`, or `Sequence` creation. If the default settings change, the new defaults will only apply to newly-created channels, brokers, or sequences. Existing ones will not change.