Add Apache Kafka Broker

docs/eventing/broker/alternate/README.md

@@ -6,3 +6,7 @@ implementations provided by Knative Eventing.
 
 ## GCP Broker
 The GCP Broker is optimized for running in GCP. For more details, refer to the [doc](https://github.com/google/knative-gcp/blob/master/docs/install/install-gcp-broker.md).
+
+## Apache Kafka Broker
+
+For information about the Apache Kafka broker, see [link](../kafka-broker.md).

docs/eventing/broker/kafka-broker.md

@@ -0,0 +1,212 @@
+---
+title: "Apache Kafka Broker"
+weight: 30
+type: "docs"
+---
+
+The Apache Kafka Broker is a native Broker implementation, that reduces
+network hops, supports any Kafka version, and has a better integration
+with Apache Kafka for the Knative Broker and Trigger model.
+
+Notable features are:
+
+- Control plane High Availability
+- Horizontally scalable data plane
+- [Extensively configurable](#Kafka-Producer-and-Consumer-configurations)
+- Ordered delivery of events based on [CloudEvents partitioning extension](https://github.com/cloudevents/spec/blob/master/extensions/partitioning.md)
+- Support any Kafka version, see [compatibility matrix](https://cwiki.apache.org/confluence/display/KAFKA/Compatibility+Matrix)
+
+## Prerequisites
+
+[Knative Eventing installation](./../../install/any-kubernetes-cluster.md#installing-the-eventing-component).
+
+## Installation
+
+1. Install the Kafka broker by entering the following command:
+
+    ```bash
+    kubectl apply --filename {{< artifact org="knative-sandbox" repo="eventing-kafka-broker" file="eventing-kafka-broker.yaml" >}}
+    ```
+
+2. Verify that `kafka-broker-controller`, `kafka-broker-receiver` and `kafka-broker-dispatcher` are running,
+by entering the following command:
+
+    ```bash
+    kubectl get deployments.apps -n knative-eventing
+    ```
+
+    Example output:
+
+    ```bash
+    NAME                           READY   UP-TO-DATE   AVAILABLE   AGE
+    eventing-controller            1/1     1            1           10s
+    eventing-webhook               1/1     1            1           9s
+    kafka-broker-controller        1/1     1            1           3s
+    kafka-broker-dispatcher        1/1     1            1           4s
+    kafka-broker-receiver          1/1     1            1           5s
+    ```
+
+## Create a Kafka Broker
+
+A Kafka Broker object looks like this:
+
+```yaml
+apiVersion: eventing.knative.dev/v1
+kind: Broker
+metadata:
+  annotations:
+    # case-sensitive
+    eventing.knative.dev/broker.class: Kafka
+  name: default
+  namespace: default
+spec:
+  # Configuration specific to this broker.
+  config:
+    apiVersion: v1
+    kind: ConfigMap
+    name: kafka-broker-config
+    namespace: knative-eventing
+  # Optional dead letter sink, you can specify either:
+  #  - deadLetterSink.ref, which is a reference to a Callable
+  #  - deadLetterSink.uri, which is an absolute URI to a Callable (It can potentially be out of the Kubernetes cluster)
+  delivery:
+    deadLetterSink:
+      ref:
+        apiVersion: serving.knative.dev/v1
+        kind: Service
+        name: dlq-service
+```
+
+`spec.config` should reference any `ConfigMap` that looks like the following:
+
+```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: kafka-broker-config
+  namespace: knative-eventing
+data:
+  # Number of topic partitions
+  default.topic.partitions: "10"
+  # Replication factor of topic messages.
+  default.topic.replication.factor: "1"
+  # A comma separated list of bootstrap servers. (It can be in or out the k8s cluster)
+  bootstrap.servers: "my-cluster-kafka-bootstrap.kafka:9092"
+```
+
+The above `ConfigMap` is installed in the cluster. You can edit
+the configuration or create a new one with the same values
+depending on your needs.
+
+## Set as default broker implementation
+
+To set the Kafka broker as the default implementation for all brokers in the Knative deployment,
+you can apply global settings by modifying the `config-br-defaults` ConfigMap in the `knative-eventing` namespace.
+
+This allows you to avoid configuring individual or per-namespace settings for each broker,
+such as `metadata.annotations.eventing.knative.dev/broker.class` or `spec.config`.
+
+<<<<<<< HEAD
+The following YAML is an example of a config-br-defaults ConfigMap using Kafka broker as the default implementation.
+=======
+This allows you to avoid configuring individual or per-namespace settings for each broker, such as `metadata.annotations.eventing.knative.dev/broker.class` or
+`spec.config`.
+>>>>>>> 26ab7c8d... lint
+
+```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: config-br-defaults
+  namespace: knative-eventing
+data:
+  default-br-config: |
+    clusterDefault:
+      brokerClass: Kafka
+      apiVersion: v1
+      kind: ConfigMap
+      name: kafka-broker-config
+      namespace: knative-eventing
+    namespaceDefaults:
+      namespace1:
+        brokerClass: Kafka
+        apiVersion: v1
+        kind: ConfigMap
+        name: kafka-broker-config
+        namespace: knative-eventing
+      namespace2:
+        brokerClass: Kafka
+        apiVersion: v1
+        kind: ConfigMap
+        name: kafka-broker-config
+        namespace: knative-eventing
+```
+
+## Kafka Producer and Consumer configurations
+
+Knative exposes all available Kafka producer and consumer configurations that can be modified to suit your workloads.
+
+You can change these configurations by modifying the `config-kafka-broker-data-plane` `ConfigMap` in
+the `knative-eventing` namespace.
+
+Documentation for the settings available in this `ConfigMap` is available on the
+[Apache Kafka website](https://kafka.apache.org/documentation/),
+in particular, [Producer configurations](https://kafka.apache.org/documentation/#producerconfigs)
+and [Consumer configurations](https://kafka.apache.org/documentation/#consumerconfigs).
+
+## Enable debug logging for data plane components
+
+The following YAML shows the default logging configuration for data plane components, that is created during the
+installation step:
+
+```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: kafka-broker-config-logging
+  namespace: knative-eventing
+data:
+  config.xml: |
+    <configuration>
+      <appender name="jsonConsoleAppender" class="ch.qos.logback.core.ConsoleAppender">
+        <encoder class="net.logstash.logback.encoder.LogstashEncoder"/>
+      </appender>
+      <root level="INFO">
+        <appender-ref ref="jsonConsoleAppender"/>
+      </root>
+    </configuration>
+```
+
+To change the logging level to `DEBUG`, you need to: 
+
+1. Apply the following `kafka-broker-config-logging` `ConfigMap` or replace `level="INFO"` with `level="DEBUG"` to the
+`ConfigMap` `kafka-broker-config-logging`:
+
+    ```yaml
+    apiVersion: v1
+    kind: ConfigMap
+    metadata:
+      name: kafka-broker-config-logging
+      namespace: knative-eventing
+    data:
+      config.xml: |
+        <configuration>
+          <appender name="jsonConsoleAppender" class="ch.qos.logback.core.ConsoleAppender">
+            <encoder class="net.logstash.logback.encoder.LogstashEncoder"/>
+          </appender>
+          <root level="DEBUG">
+            <appender-ref ref="jsonConsoleAppender"/>
+          </root>
+        </configuration>
+    ```
+
+2. Restart the `kafka-broker-receiver` and the `kafka-broker-dispatcher`, by entering the following commands:
+
+    ```bash
+    kubectl rollout restart deployment -n knative-eventing kafka-broker-receiver
+    kubectl rollout restart deployment -n knative-eventing kafka-broker-dispatcher
+    ```
+
+### Additional information
+
+- To report bugs or add feature requests, open an issue in the [eventing-kafka-broker repository](https://github.com/knative-sandbox/eventing-kafka-broker).

docs/install/any-kubernetes-cluster.md

@@ -605,8 +605,19 @@ kubectl apply --filename {{< artifact repo="eventing" file="in-memory-channel.ya
 1. Install a Broker (eventing) layer:
 
       <!-- This indentation is important for things to render properly. -->
-
    {{< tabs name="eventing_brokers" default="MT-Channel-based" >}}
+   {{% tab name="Apache Kafka Broker" %}}
+   {{< feature-state version="v0.17" state="alpha" >}}
+The following command installs the Apache Kafka broker, and runs event routing in a system namespace,
+`knative-eventing`, by default.
+
+```bash
+kubectl apply --filename {{< artifact org="knative-sandbox" repo="eventing-kafka-broker" file="eventing-kafka-broker.yaml" >}}
+```
+
+For more information, see the [Kafka Broker](./../eventing/broker/kafka-broker.md) documentation.
+{{< /tab >}}
+
    {{% tab name="MT-Channel-based" %}}
    {{< feature-state version="v0.16" state="stable" >}}