From 9024779cea88fd2c2b62cc1d27ca9d08f4c7c906 Mon Sep 17 00:00:00 2001 From: Evan Anderson Date: Tue, 7 Oct 2025 13:25:38 -0700 Subject: [PATCH] Re-arrange navigation to fill out 'Administration' topic --- docs/versioned/.nav.yml | 191 +++++++++--------- docs/versioned/administer/README.md | 13 ++ .../broker-types/kafka-broker/README.md | 180 +---------------- .../broker-types/rabbitmq-broker/README.md | 30 +-- .../install/eventing/kafka-install.md | 169 ++++++++++++++++ .../install/eventing/rabbitmq-install.md | 46 +++++ .../serving/knative-kubernetes-services.md | 90 --------- 7 files changed, 326 insertions(+), 393 deletions(-) create mode 100644 docs/versioned/administer/README.md create mode 100644 docs/versioned/install/eventing/kafka-install.md create mode 100644 docs/versioned/install/eventing/rabbitmq-install.md delete mode 100644 docs/versioned/serving/knative-kubernetes-services.md diff --git a/docs/versioned/.nav.yml b/docs/versioned/.nav.yml index b84cebf74cb..4088a923bc3 100644 --- a/docs/versioned/.nav.yml +++ b/docs/versioned/.nav.yml @@ -60,12 +60,7 @@ nav: - Overview: serving/README.md - Architecture: serving/architecture.md - Request Flow: serving/request-flow.md - - Resources: - - Revisions: - - About Revisions: serving/revisions/README.md - - Administrator configuration options: serving/revisions/revision-admin-config-options.md - - Developer configuration options: serving/revisions/revision-developer-config-options.md - # add other resource types + - Converting a Kubernetes Deployment to a Knative Service: serving/convert-deployment-to-knative-service.md - Autoscaling: - About autoscaling: serving/autoscaling/README.md - Supported autoscaler types: serving/autoscaling/autoscaler-types.md @@ -94,6 +89,9 @@ nav: - Configuring HTTP: serving/services/http-protocol.md - Performing Dry Run: serving/dryrun.md - Volume Support: serving/services/storage.md + - Revisions: + - About Revisions: serving/revisions/README.md + - Garbage Collection: serving/revisions/revision-developer-config-options.md - Traffic management: serving/traffic-management.md - Configuring gradual rollout of traffic to Revisions: serving/rolling-out-latest-revision.md - Tag resolution: serving/tag-resolution.md @@ -102,27 +100,6 @@ nav: - About load balancing: serving/load-balancing/README.md - Configuring target burst capacity: serving/load-balancing/target-burst-capacity.md - Configuring Activator capacity: serving/load-balancing/activator-capacity.md - # Serving - admin docs - - Administrator Topics: - - Kubernetes services: serving/knative-kubernetes-services.md - - Enabling requests to Knative services when additional authorization policies are enabled: serving/istio-authorization.md - - Exclude namespaces from the Knative webhook: serving/webhook-customizations.md - - Configuring high-availability components: serving/config-ha.md - - Configuring the ingress gateway: serving/setting-up-custom-ingress-gateway.md - - Configuring domain names: serving/using-a-custom-domain.md - - Converting a Kubernetes Deployment to a Knative Service: serving/convert-deployment-to-knative-service.md - - Extending Queue Proxy image with QPOptions: serving/queue-extensions.md - - Serving configuration: - - Configure Deployment resources: serving/configuration/deployment.md - - Configuring gradual rollout of traffic to Revisions: serving/configuration/rolling-out-latest-revision-configmap.md - - Feature and extension flags: serving/configuration/feature-flags.md - - Configure the Defaults ConfigMap: serving/configuration/config-defaults.md - - Serving encryption configuration: - - Overview: serving/encryption/encryption-overview.md - - Configure cert-manager integration: serving/encryption/configure-certmanager-integration.md - - Configure external domain encryption: serving/encryption/external-domain-tls.md - - Configure cluster-local domain encryption: serving/encryption/cluster-local-domain-tls.md - - Configure Knative system-internal encryption: serving/encryption/system-internal-tls.md # Serving - Application Security - Application security: - About Security-Guard: serving/app-security/security-guard-about.md @@ -131,10 +108,6 @@ nav: - Security-Guard example alerts: serving/app-security/security-guard-example-alerts.md # Serving - observability - Observability: - - Request traces: serving/observability/accessing-traces.md - - Collecting logs: serving/observability/logging/collecting-logs.md - - Configuring logging: serving/observability/logging/config-logging.md - - Configuring Request logging: serving/observability/logging/request-logging.md - Collecting metrics: serving/observability/metrics/collecting-metrics.md - Metrics Reference: serving/observability/metrics/serving-metrics.md # Serving - troubleshooting @@ -158,7 +131,6 @@ nav: - Channel based Broker: eventing/brokers/broker-types/channel-based-broker/README.md - Apache Kafka: - About Apache Kafka Broker: eventing/brokers/broker-types/kafka-broker/README.md - - Configuring Kafka features: eventing/brokers/broker-types/kafka-broker/configuring-kafka-features.md - RabbitMQ Broker: eventing/brokers/broker-types/rabbitmq-broker/README.md - Creating a Broker: eventing/brokers/create-broker.md - Developer configuration options: eventing/brokers/broker-developer-config-options.md @@ -241,35 +213,10 @@ nav: - Knative EventMesh Backstage Plugin: eventing/event-registry/eventmesh-backstage-plugin.md - Sugar controller: eventing/sugar/README.md - Debugging: eventing/troubleshooting/README.md - - Administrator topics: - # Eventing config - - Eventing configuration: - - Configure Broker defaults: eventing/configuration/broker-configuration.md - - Configure Channel defaults: eventing/configuration/channel-configuration.md - - Configure Apache Kafka Channel defaults: eventing/configuration/kafka-channel-configuration.md - - Configure event source defaults: eventing/configuration/sources-configuration.md - - Configure Sugar Controller: eventing/configuration/sugar-configuration.md - - Configure KEDA Autoscaling of Knative Kafka Resources: eventing/configuration/keda-configuration.md # Eventing - observability - Observability: - - Accessing CloudEvent traces: eventing/observability/accessing-traces.md - - Collecting logs: eventing/observability/logging/collecting-logs.md - - Configuring logging: eventing/observability/logging/config-logging.md - Collecting metrics: eventing/observability/metrics/collecting-metrics.md - Metrics Reference: eventing/observability/metrics/eventing-metrics.md - - Features: - - About Eventing features: eventing/features/README.md - - DeliverySpec.Timeout field: eventing/features/delivery-timeout.md - - DeliverySpec.RetryAfterMax field: eventing/features/delivery-retryafter.md - - New APIServerSource Filters: eventing/features/new-apiserversource-filters.md - - KReference.Group field: eventing/features/kreference-group.md - - Knative reference mapping: eventing/features/kreference-mapping.md - - EventType auto creation: eventing/features/eventtype-auto-creation.md - - Transport Encryption: eventing/features/transport-encryption.md - - Sender Identity: eventing/features/sender-identity.md - - Authorization: eventing/features/authorization.md - - Eventing with Istio: eventing/features/istio-integration.md - - Cross Namespace Event Links: eventing/features/cross-namespace-event-links.md - FAQ: eventing/faq/README.md # Eventing reference docs - Reference: @@ -286,42 +233,101 @@ nav: # Administration ############################################################################### - Administration: - # TODO: add a general overview of Knative for administrators - - Installing Knative: install/README.md - # YAML Installation - - Install Knative with YAML: - - About YAML-based installation: install/yaml-install/README.md - - Install Knative Serving: - - Install Serving with YAML: install/yaml-install/serving/install-serving-with-yaml.md - - Knative Serving installation files: install/yaml-install/serving/serving-installation-files.md - - Install Knative Eventing: - - Install Eventing with YAML: install/yaml-install/eventing/install-eventing-with-yaml.md - - Knative Eventing installation files: install/yaml-install/eventing/eventing-installation-files.md - # Operator Installation - - Install with Knative Operator: - - Install by using the Knative Operator: install/operator/knative-with-operators.md - - Install by using the Knative Operator CLI plugin: install/operator/knative-with-operator-cli.md - - Configuring Knative using the Operator: install/operator/configuring-with-operator.md - - Configuring Knative Serving CRDs: install/operator/configuring-serving-cr.md - - Configuring Knative Eventing CRDs: install/operator/configuring-eventing-cr.md - # Advanced options for Serving - - Install advanced options: - # Istio Installation - - Install Istio for Knative: install/installing-istio.md - # Cert-manager Installation - - Install cert-manager: install/installing-cert-manager.md - # Backstage plugin - - Install Knative Backstage plugin: install/installing-backstage-plugins.md - # Vendor docs - - Using a Knative-based offering: install/knative-offerings.md + - Overview: administer/README.md + - Installing Knative: + - Overview: install/README.md + # YAML Installation + - Install Knative with YAML: + - About YAML-based installation: install/yaml-install/README.md + - Install Knative Serving: + - Install Serving with YAML: install/yaml-install/serving/install-serving-with-yaml.md + - Knative Serving installation files: install/yaml-install/serving/serving-installation-files.md + - Install Knative Eventing: + - Install Eventing with YAML: install/yaml-install/eventing/install-eventing-with-yaml.md + - Knative Eventing installation files: install/yaml-install/eventing/eventing-installation-files.md + # Operator Installation + - Install with Knative Operator: + - Install by using the Knative Operator: install/operator/knative-with-operators.md + - Install by using the Knative Operator CLI plugin: install/operator/knative-with-operator-cli.md + - Configuring Knative using the Operator: install/operator/configuring-with-operator.md + - Configuring Knative Serving CRDs: install/operator/configuring-serving-cr.md + - Configuring Knative Eventing CRDs: install/operator/configuring-eventing-cr.md + - Installing plugins: + - Install Istio for Knative: install/installing-istio.md + # TODO: docs for kourier, contour, gateway-api + - Install Kafka for Knative: install/eventing/kafka-install.md + - Install RabbitMQ for Knative: install/eventing/rabbitmq-install.md + # N.B. this duplicates an "eventing" topic above, cross-referenced here. + - Eventing Channels: eventing/channels/channels-crds.md + - Integrations: + - Install cert-manager: install/installing-cert-manager.md + - Install Knative Backstage plugin: install/installing-backstage-plugins.md + # Vendor list + - Using a Knative-based offering: install/knative-offerings.md + - Configuring Knative: + # TODO: add a doc on editing ConfigMaps + - Configure high-availability components: serving/config-ha.md + - Exclude namespaces from the Knative webhook: serving/webhook-customizations.md + - Networking Options: + - Configure the ingress gateway: serving/setting-up-custom-ingress-gateway.md + - Configure domain names: serving/using-a-custom-domain.md + - Istio Authorization: serving/istio-authorization.md + - Extending Queue Proxy image with QPOptions: serving/queue-extensions.md + - Serving configuration: + - Configure Deployment resources: serving/configuration/deployment.md + - Configure gradual rollout of traffic to Revisions: serving/configuration/rolling-out-latest-revision-configmap.md + - Config Revision Garbage Collection: serving/revisions/revision-admin-config-options.md + - Configure the Defaults ConfigMap: serving/configuration/config-defaults.md + - Serving encryption configuration: + - Overview: serving/encryption/encryption-overview.md + - Configure cert-manager integration: serving/encryption/configure-certmanager-integration.md + - Configure external domain encryption: serving/encryption/external-domain-tls.md + - Configure cluster-local domain encryption: serving/encryption/cluster-local-domain-tls.md + - Configure Knative system-internal encryption: serving/encryption/system-internal-tls.md + - Eventing configuration: + - Configure Broker defaults: eventing/configuration/broker-configuration.md + - Configure Channel defaults: eventing/configuration/channel-configuration.md + - Configure Apache Kafka Channel defaults: eventing/configuration/kafka-channel-configuration.md + - Configure Kafka Broker features: eventing/brokers/broker-types/kafka-broker/configuring-kafka-features.md + - Configure event source defaults: eventing/configuration/sources-configuration.md + - Configure Sugar Controller: eventing/configuration/sugar-configuration.md + - Configure KEDA Autoscaling of Knative Kafka Resources: eventing/configuration/keda-configuration.md + - Flagged features: + - Serving Features: serving/configuration/feature-flags.md + - Eventing Features: + - Configuring Features: eventing/features/README.md + - DeliverySpec.Timeout field: eventing/features/delivery-timeout.md + - DeliverySpec.RetryAfterMax field: eventing/features/delivery-retryafter.md + - New APIServerSource Filters: eventing/features/new-apiserversource-filters.md + - KReference.Group field: eventing/features/kreference-group.md + - Knative reference mapping: eventing/features/kreference-mapping.md + - EventType auto creation: eventing/features/eventtype-auto-creation.md + - Transport Encryption: eventing/features/transport-encryption.md + - Sender Identity: eventing/features/sender-identity.md + - Authorization: eventing/features/authorization.md + - Eventing with Istio: eventing/features/istio-integration.md + - Cross Namespace Event Links: eventing/features/cross-namespace-event-links.md + - Securing Knative: + - Security Disclosure: reference/security/README.md + - Threat Model: reference/security/threat-model.md + - Verifying Knative Images: reference/security/verifying-images.md + - Verifying Knative Binaries: reference/security/verifying-cli.md + # TODO: add some observability documentation + - Observability: + - Serving Request traces: serving/observability/accessing-traces.md + - Collecting Serving logs: serving/observability/logging/collecting-logs.md + - Collecting Eventing logs: eventing/observability/logging/collecting-logs.md + - Configuring Serving logging: serving/observability/logging/config-logging.md + - Configuring Serving Request logging: serving/observability/logging/request-logging.md + - Configuring Eventing logging: eventing/observability/logging/config-logging.md + - Accessing CloudEvent traces: eventing/observability/accessing-traces.md # Upgrading Knative - - Upgrading your installation: + - Upgrading and removing: - About upgrading Knative: install/upgrade/README.md - Checking your Knative version: install/upgrade/check-install-version.md - Upgrading with kubectl: install/upgrade/upgrade-installation.md - Upgrading with the Knative Operator: install/upgrade/upgrade-installation-with-operator.md - # Uninstall - - Uninstalling Knative: install/uninstall.md + - Uninstalling Knative: install/uninstall.md # Troubleshooting - Troubleshooting Knative: install/troubleshooting.md ############################################################################### @@ -331,11 +337,4 @@ nav: - Code sample overview: samples/README.md - Serving code samples: samples/serving.md - Eventing code samples: samples/eventing.md - # Reference docs - - Reference: - - Security: - - Security Disclosure: reference/security/README.md - - Threat Model: reference/security/threat-model.md - - Verifying Knative Images: reference/security/verifying-images.md - - Verifying Knative Binaries: reference/security/verifying-cli.md - - Release notes: reference/relnotes/README.md + diff --git a/docs/versioned/administer/README.md b/docs/versioned/administer/README.md new file mode 100644 index 00000000000..2fa2838d2bb --- /dev/null +++ b/docs/versioned/administer/README.md @@ -0,0 +1,13 @@ +--- +audience: administrator +components: + - eventing + - serving +function: concept +--- + +# Administration Overview + +The Knative [Serving](../serving/) and [Eventing](../ eventing/) components extend Kubernetes clusters to provide consistent serverless compute and event-delivery interfaces for developers. (Knative [Functions](../functions/) provides primarily client-side tools for more easily building containers.) This means that Serving and Eventing need to be installed on a Kubernetes cluster, and they extend the Kubernetes API using [Custom Resources Definitions](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/). + +Knative aims to integrate with Kubernetes, so existing tools for managing Kubernetes such as RBAC, quota, admission control, and monitoring should "just work". By using a plug-in model for underlying HTTP and event routing functionality, Knative aims to integrate with existing battle-tested software you may already have installed on your cluster, such as Istio, Contour, Kafka, NATS or RabbitMQ. Knative also provides lightweight implementations for these functions if you are looking to avoid pulling in large, complex systems. \ No newline at end of file diff --git a/docs/versioned/eventing/brokers/broker-types/kafka-broker/README.md b/docs/versioned/eventing/brokers/broker-types/kafka-broker/README.md index 97e11f29c2e..075fb3bfcb2 100644 --- a/docs/versioned/eventing/brokers/broker-types/kafka-broker/README.md +++ b/docs/versioned/eventing/brokers/broker-types/kafka-broker/README.md @@ -1,6 +1,5 @@ --- audience: developer -# And audience: administrator for install components: - eventing function: how-to @@ -23,43 +22,7 @@ The Knative Kafka Broker stores incoming CloudEvents as Kafka records, using the ## Prerequisites -1. You have installed Knative Eventing. -2. You have access to an Apache Kafka cluster. - -!!! tip - If you need to set up a Kafka cluster, you can do this by following the instructions on the [Strimzi Quickstart page](https://strimzi.io/quickstarts/). - -## Installation - -1. Install the Kafka controller by entering the following command: - - ```bash - kubectl apply --filename {{ artifact(org="knative-extensions", repo="eventing-kafka-broker", file="eventing-kafka-controller.yaml") }} - ``` - -1. Install the Kafka Broker data plane by entering the following command: - - ```bash - kubectl apply --filename {{ artifact(org="knative-extensions", repo="eventing-kafka-broker", file="eventing-kafka-broker.yaml") }} - ``` - -1. Verify that `kafka-controller`, `kafka-broker-receiver` and `kafka-broker-dispatcher` are running, -by entering the following command: - - ```bash - kubectl get deployments.apps -n knative-eventing - ``` - - Example output: - - ```{ .bash .no-copy } - NAME READY UP-TO-DATE AVAILABLE AGE - eventing-controller 1/1 1 1 10s - eventing-webhook 1/1 1 1 9s - kafka-controller 1/1 1 1 3s - kafka-broker-dispatcher 1/1 1 1 4s - kafka-broker-receiver 1/1 1 1 5s - ``` +These directions assume your cluster administrator has [installed the Knative Kafka broker](../../../../install/eventing/kafka-install.md). ## Create a Kafka Broker @@ -103,7 +66,7 @@ apiVersion: v1 kind: ConfigMap metadata: name: kafka-broker-config - namespace: knative-eventing + namespace: your-namespace data: # Number of topic partitions default.topic.partitions: "10" @@ -141,45 +104,6 @@ data: default.topic.config.retention.ms: "3600" ``` -## Set as default broker implementation - -To set the Kafka broker as the default implementation for all brokers in the Knative deployment, -you can apply global settings by modifying the `config-br-defaults` ConfigMap in the `knative-eventing` namespace. - -This allows you to avoid configuring individual or per-namespace settings for each broker, -such as `metadata.annotations.eventing.knative.dev/broker.class` or `spec.config`. - -The following YAML is an example of a `config-br-defaults` ConfigMap using Kafka broker as the default implementation. - -```yaml -apiVersion: v1 -kind: ConfigMap -metadata: - name: config-br-defaults - namespace: knative-eventing -data: - default-br-config: | - clusterDefault: - brokerClass: Kafka - apiVersion: v1 - kind: ConfigMap - name: kafka-broker-config - namespace: knative-eventing - namespaceDefaults: - namespace1: - brokerClass: Kafka - apiVersion: v1 - kind: ConfigMap - name: kafka-broker-config - namespace: knative-eventing - namespace2: - brokerClass: Kafka - apiVersion: v1 - kind: ConfigMap - name: kafka-broker-config - namespace: knative-eventing -``` - ## Security Apache Kafka supports different security features, Knative supports the followings: @@ -285,106 +209,6 @@ spec: !!! note When using an external topic, the Knative Kafka Broker does not own the topic and is not responsible for managing the topic. This includes the topic lifecycle or its general validity. Other restrictions for general access to the topic may apply. See the documentation about using [Access Control Lists (ACLs)](https://kafka.apache.org/documentation/#security_authz). -## Consumer Offsets Commit Interval - -Kafka consumers keep track of the last successfully sent events by committing offsets. - -Knative Kafka Broker commits the offset every `auto.commit.interval.ms` milliseconds. - -!!! note - To prevent negative impacts to performance, it is not recommended committing - offsets every time an event is successfully sent to a subscriber. - -The interval can be changed by changing the `config-kafka-broker-data-plane` `ConfigMap` -in the `knative-eventing` namespace by modifying the parameter `auto.commit.interval.ms` as follows: - -```yaml - -apiVersion: v1 -kind: ConfigMap -metadata: - name: config-kafka-broker-data-plane - namespace: knative-eventing -data: - # Some configurations omitted ... - config-kafka-broker-consumer.properties: | - # Some configurations omitted ... - - # Commit the offset every 5000 millisecods (5 seconds) - auto.commit.interval.ms=5000 -``` - -!!! note - Knative Kafka Broker guarantees at least once delivery, which means that your applications may - receive duplicate events. A higher commit interval means that there is a higher probability of - receiving duplicate events, because when a Consumer restarts, it restarts from the last - committed offset. - -## Kafka Producer and Consumer configurations - -Knative exposes all available Kafka producer and consumer configurations that can be modified to suit your workloads. - -You can change these configurations by modifying the `config-kafka-broker-data-plane` `ConfigMap` in -the `knative-eventing` namespace. - -Documentation for the settings available in this `ConfigMap` is available on the -[Apache Kafka website](https://kafka.apache.org/documentation/), -in particular, [Producer configurations](https://kafka.apache.org/documentation/#producerconfigs) -and [Consumer configurations](https://kafka.apache.org/documentation/#consumerconfigs). - -## Enable debug logging for data plane components - -The following YAML shows the default logging configuration for data plane components, that is created during the -installation step: - -```yaml -apiVersion: v1 -kind: ConfigMap -metadata: - name: kafka-config-logging - namespace: knative-eventing -data: - config.xml: | - - - - - - - - -``` - -To change the logging level to `DEBUG`, you must: - -1. Apply the following `kafka-config-logging` `ConfigMap` or replace `level="INFO"` with `level="DEBUG"` to the -`ConfigMap` `kafka-config-logging`: - - ```yaml - apiVersion: v1 - kind: ConfigMap - metadata: - name: kafka-config-logging - namespace: knative-eventing - data: - config.xml: | - - - - - - - - - ``` - -2. Restart the `kafka-broker-receiver` and the `kafka-broker-dispatcher`, by entering the following commands: - - ```bash - kubectl rollout restart deployment -n knative-eventing kafka-broker-receiver - kubectl rollout restart deployment -n knative-eventing kafka-broker-dispatcher - ``` - ## Configuring the order of delivered events When dispatching events, the Kafka broker can be configured to support different delivery ordering guarantees. diff --git a/docs/versioned/eventing/brokers/broker-types/rabbitmq-broker/README.md b/docs/versioned/eventing/brokers/broker-types/rabbitmq-broker/README.md index 6344245821c..be0f3bfc590 100644 --- a/docs/versioned/eventing/brokers/broker-types/rabbitmq-broker/README.md +++ b/docs/versioned/eventing/brokers/broker-types/rabbitmq-broker/README.md @@ -1,6 +1,5 @@ --- audience: developer -# And audience: administrator for install components: - eventing function: explanation @@ -12,34 +11,7 @@ This topic describes how to create a RabbitMQ Broker. ## Prerequisites -1. You have installed Knative Eventing. -1. You have installed [CertManager v1.5.4](https://github.com/jetstack/cert-manager/releases/tag/v1.5.4) - easiest integration with RabbitMQ Messaging Topology Operator -1. You have installed [RabbitMQ Messaging Topology Operator](https://github.com/rabbitmq/messaging-topology-operator) - our recommendation is [latest release](https://github.com/rabbitmq/messaging-topology-operator/releases/latest) with CertManager -1. You have access to a working RabbitMQ instance. You can create a RabbitMQ instance by using the [RabbitMQ Cluster Kubernetes Operator](https://github.com/rabbitmq/cluster-operator). For more information see the [RabbitMQ website](https://www.rabbitmq.com/kubernetes/operator/using-operator.html). - -## Install the RabbitMQ controller - -1. Install the RabbitMQ controller by running the command: - - ```bash - kubectl apply -f {{ artifact(org="knative-extensions", repo="eventing-rabbitmq", file="rabbitmq-broker.yaml") }} - ``` - -1. Verify that `rabbitmq-broker-controller` and `rabbitmq-broker-webhook` are running: - - ```bash - kubectl get deployments.apps -n knative-eventing - ``` - - Example output: - - ```{ .bash .no-copy } - NAME READY UP-TO-DATE AVAILABLE AGE - eventing-controller 1/1 1 1 10s - eventing-webhook 1/1 1 1 9s - rabbitmq-broker-controller 1/1 1 1 3s - rabbitmq-broker-webhook 1/1 1 1 4s - ``` +These directions assume your cluster administrator has [installed the Knative RabbitMQ broker](../../../../install/eventing/rabbitmq-install.md). ## Create a RabbitMQBrokerConfig object diff --git a/docs/versioned/install/eventing/kafka-install.md b/docs/versioned/install/eventing/kafka-install.md new file mode 100644 index 00000000000..52ffb231b3e --- /dev/null +++ b/docs/versioned/install/eventing/kafka-install.md @@ -0,0 +1,169 @@ +--- +audience: administrator +components: + - eventing +function: how-to +--- + +# Knative Broker for Apache Kafka + +The Knative Broker for Apache Kafka is an implementation of the Knative Broker API natively targeting Apache Kafka to reduce network hops and offering a better integration with Apache Kafka for the Broker and Trigger API model. + +Notable features are: + +- Control plane High Availability +- Horizontally scalable data plane +- [Extensively configurable](./configuring-kafka-features) +- Ordered delivery of events based on [CloudEvents partitioning extension](https://github.com/cloudevents/spec/blob/v1.0.2/cloudevents/extensions/partitioning.md) +- Support any Kafka version, see [compatibility matrix](https://cwiki.apache.org/confluence/display/KAFKA/Compatibility+Matrix) +- Supports 2 [data plane modes](#data-plane-isolation-vs-shared-data-plane): data plane isolation per-namespace or shared data plane + +The Knative Kafka Broker stores incoming CloudEvents as Kafka records, using the [binary content mode](https://github.com/cloudevents/spec/blob/v1.0.2/cloudevents/bindings/kafka-protocol-binding.md#32-binary-content-mode), because it is more efficient due to its optimizations for transport or routing, as well avoid JSON parsing. Using `binary content mode` means all CloudEvent attributes and extensions are mapped as [headers on the Kafka record](https://github.com/cloudevents/spec/blob/v1.0.2/cloudevents/bindings/kafka-protocol-binding.md#323-metadata-headers), while the `data` of the CloudEvent corresponds to the actual value of the Kafka record. This is another benefit of using `binary content mode` over `structured content mode` as it is less _obstructive_ and therefore compatible with systems that do not understand CloudEvents. + +## Prerequisites + +1. You have installed Knative Eventing. +2. You have access to an Apache Kafka cluster. + +!!! tip + If you need to set up a Kafka cluster, you can do this by following the instructions on the [Strimzi Quickstart page](https://strimzi.io/quickstarts/). + +## Installation + +1. Install the Kafka controller by entering the following command: + + ```bash + kubectl apply --filename {{ artifact(org="knative-extensions", repo="eventing-kafka-broker", file="eventing-kafka-controller.yaml") }} + ``` + +1. Install the Kafka Broker data plane by entering the following command: + + ```bash + kubectl apply --filename {{ artifact(org="knative-extensions", repo="eventing-kafka-broker", file="eventing-kafka-broker.yaml") }} + ``` + +1. Verify that `kafka-controller`, `kafka-broker-receiver` and `kafka-broker-dispatcher` are running, +by entering the following command: + + ```bash + kubectl get deployments.apps -n knative-eventing + ``` + + Example output: + + ```{ .bash .no-copy } + NAME READY UP-TO-DATE AVAILABLE AGE + eventing-controller 1/1 1 1 10s + eventing-webhook 1/1 1 1 9s + kafka-controller 1/1 1 1 3s + kafka-broker-dispatcher 1/1 1 1 4s + kafka-broker-receiver 1/1 1 1 5s + ``` + +## Consumer Offsets Commit Interval + +Kafka consumers keep track of the last successfully sent events by committing offsets. + +Knative Kafka Broker commits the offset every `auto.commit.interval.ms` milliseconds. + +!!! note + To prevent negative impacts to performance, it is not recommended committing + offsets every time an event is successfully sent to a subscriber. + +The interval can be changed by changing the `config-kafka-broker-data-plane` `ConfigMap` +in the `knative-eventing` namespace by modifying the parameter `auto.commit.interval.ms` as follows: + +```yaml + +apiVersion: v1 +kind: ConfigMap +metadata: + name: config-kafka-broker-data-plane + namespace: knative-eventing +data: + # Some configurations omitted ... + config-kafka-broker-consumer.properties: | + # Some configurations omitted ... + + # Commit the offset every 5000 millisecods (5 seconds) + auto.commit.interval.ms=5000 +``` + +!!! note + Knative Kafka Broker guarantees at least once delivery, which means that your applications may + receive duplicate events. A higher commit interval means that there is a higher probability of + receiving duplicate events, because when a Consumer restarts, it restarts from the last + committed offset. + +## Kafka Producer and Consumer configurations + +Knative exposes all available Kafka producer and consumer configurations that can be modified to suit your workloads. + +You can change these configurations by modifying the `config-kafka-broker-data-plane` `ConfigMap` in +the `knative-eventing` namespace. + +Documentation for the settings available in this `ConfigMap` is available on the +[Apache Kafka website](https://kafka.apache.org/documentation/), +in particular, [Producer configurations](https://kafka.apache.org/documentation/#producerconfigs) +and [Consumer configurations](https://kafka.apache.org/documentation/#consumerconfigs). + +## Enable debug logging for data plane components + +The following YAML shows the default logging configuration for data plane components, that is created during the +installation step: + +```yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: kafka-config-logging + namespace: knative-eventing +data: + config.xml: | + + + + + + + + +``` + +To change the logging level to `DEBUG`, you must: + +1. Apply the following `kafka-config-logging` `ConfigMap` or replace `level="INFO"` with `level="DEBUG"` to the +`ConfigMap` `kafka-config-logging`: + + ```yaml + apiVersion: v1 + kind: ConfigMap + metadata: + name: kafka-config-logging + namespace: knative-eventing + data: + config.xml: | + + + + + + + + + ``` + +2. Restart the `kafka-broker-receiver` and the `kafka-broker-dispatcher`, by entering the following commands: + + ```bash + kubectl rollout restart deployment -n knative-eventing kafka-broker-receiver + kubectl rollout restart deployment -n knative-eventing kafka-broker-dispatcher + ``` + +### Enabling and configuring autoscaling of triggers with KEDA + +To enable and configreu autoscaling of triggers referencing Kafka Brokers with KEDA, follow [the instructions here](../../eventing/configuration/keda-configuration.md). + +## Additional information + +- To report a bug or request a feature, open an issue in the [eventing-kafka-broker repository](https://github.com/knative-extensions/eventing-kafka-broker). diff --git a/docs/versioned/install/eventing/rabbitmq-install.md b/docs/versioned/install/eventing/rabbitmq-install.md new file mode 100644 index 00000000000..a27dfb256b1 --- /dev/null +++ b/docs/versioned/install/eventing/rabbitmq-install.md @@ -0,0 +1,46 @@ +--- +audience: administrator +components: + - eventing +function: explanation +--- + +# Creating a RabbitMQ Broker + +This topic describes how to install the RabbitMQ Broker onto your cluster. + +## Prerequisites + +1. You have installed Knative Eventing. +1. You have installed [CertManager v1.5.4](https://github.com/jetstack/cert-manager/releases/tag/v1.5.4) - easiest integration with RabbitMQ Messaging Topology Operator +1. You have installed [RabbitMQ Messaging Topology Operator](https://github.com/rabbitmq/messaging-topology-operator) - our recommendation is [latest release](https://github.com/rabbitmq/messaging-topology-operator/releases/latest) with CertManager +1. You have access to a working RabbitMQ instance. You can create a RabbitMQ instance by using the [RabbitMQ Cluster Kubernetes Operator](https://github.com/rabbitmq/cluster-operator). For more information see the [RabbitMQ website](https://www.rabbitmq.com/kubernetes/operator/using-operator.html). + +## Install the RabbitMQ controller + +1. Install the RabbitMQ controller by running the command: + + ```bash + kubectl apply -f {{ artifact(org="knative-extensions", repo="eventing-rabbitmq", file="rabbitmq-broker.yaml") }} + ``` + +1. Verify that `rabbitmq-broker-controller` and `rabbitmq-broker-webhook` are running: + + ```bash + kubectl get deployments.apps -n knative-eventing + ``` + + Example output: + + ```{ .bash .no-copy } + NAME READY UP-TO-DATE AVAILABLE AGE + eventing-controller 1/1 1 1 10s + eventing-webhook 1/1 1 1 9s + rabbitmq-broker-controller 1/1 1 1 3s + rabbitmq-broker-webhook 1/1 1 1 4s + ``` + +## Additional information + +- For more samples visit the [`eventing-rabbitmq` Github repository samples directory](https://github.com/knative-extensions/eventing-rabbitmq/tree/main/samples) +- To report a bug or request a feature, open an issue in the [`eventing-rabbitmq` Github repository](https://github.com/knative-extensions/eventing-rabbitmq). diff --git a/docs/versioned/serving/knative-kubernetes-services.md b/docs/versioned/serving/knative-kubernetes-services.md deleted file mode 100644 index 992c853f488..00000000000 --- a/docs/versioned/serving/knative-kubernetes-services.md +++ /dev/null @@ -1,90 +0,0 @@ ---- -audience: developer -components: - - serving -function: explanation ---- - -# Kubernetes services - -This guide describes the -[Kubernetes Services](https://kubernetes.io/docs/concepts/services-networking/service/) -that are active when running Knative Serving. - -## Before You Begin - -1. This guide assumes that you have [installed Knative Serving](../install/yaml-install/serving/install-serving-with-yaml.md). -2. Verify that you have the proper components in your cluster. To view the - services installed in your cluster, use the command: - - ```bash - $ kubectl get services -n knative-serving - ``` - - This returns an output similar to the following: - - ```sh - NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE - activator-service ClusterIP 10.96.61.11 80/TCP,81/TCP,9090/TCP 1h - autoscaler ClusterIP 10.104.217.223 8080/TCP,9090/TCP 1h - controller ClusterIP 10.101.39.220 9090/TCP 1h - webhook ClusterIP 10.107.144.50 443/TCP 1h - ``` - -3. To view the deployments in your cluster, use the following command: - - ```bash - $ kubectl get deployments -n knative-serving - ``` - - This returns an output similar to the following: - - ```{ .bash .no-copy } - NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE - activator 1 1 1 1 1h - autoscaler 1 1 1 1 1h - controller 1 1 1 1 1h - net-istio-controller 1 1 1 1 1h - webhook 1 1 1 1 1h - ``` - -These services and deployments are installed by the `serving.yaml` file during -install. The next section describes their function. - -## Components - -### Service: activator - -The activator is responsible for receiving & buffering requests for inactive -revisions and reporting metrics to the autoscaler. It also retries requests to a -revision after the autoscaler scales the revision based on the reported metrics. - -### Service: autoscaler - -The autoscaler receives request metrics and adjusts the number of pods required -to handle the load of traffic. - -### Service: controller - -The controller service reconciles all the public Knative objects and autoscaling -CRDs. When a user applies a Knative service to the Kubernetes API, this creates -the configuration and route. It will convert the configuration into revisions -and the revisions into deployments and Knative Pod Autoscalers (KPAs). - -### Service: webhook - -The webhook intercepts all Kubernetes API calls as well as all CRD insertions -and updates. It sets default values, rejects inconsistent and invalid objects, -and validates and mutates Kubernetes API calls. - -### Deployment: net-istio-controller - -The net-istio-controller deployment reconciles a cluster's ingress into an -[Istio virtual service](https://istio.io/docs/reference/config/networking/v1alpha3/virtual-service/). - -## What's Next - -- For a deeper look at the services and deployments involved in Knative Serving, - see the [specs](https://github.com/knative/specs/blob/main/specs/serving/overview.md) repository. -- For a high-level analysis of Knative Serving, see the [Knative Serving overview](README.md). -- For hands-on tutorials, see the [Knative Serving code samples](../samples/serving.md).