Docs: Add docs for using external secrets in Kafka Connect

documentation/book/assembly-deployment-configuration-kafka-connect-s2i.adoc

@@ -40,6 +40,8 @@ include::assembly-configuring-container-images.adoc[leveloffset=+1]
 
 include::assembly-scheduling.adoc[leveloffset=+1]
 
+include::assembly-kafka-connect-external-configuration.adoc[leveloffset=+1]
+
 include::ref-list-of-kafka-connect-s2i-resources.adoc[leveloffset=+1]
 
 include::proc-using-openshift-builds-create-image.adoc[leveloffset=+1]

documentation/book/assembly-deployment-configuration-kafka-connect.adoc

@@ -40,6 +40,8 @@ include::assembly-configuring-container-images.adoc[leveloffset=+1]
 
 include::assembly-scheduling.adoc[leveloffset=+1]
 
+include::assembly-kafka-connect-external-configuration.adoc[leveloffset=+1]
+
 include::ref-list-of-kafka-connect-resources.adoc[leveloffset=+1]
 
 // Restore the context to what it was before this assembly.

documentation/book/assembly-kafka-connect-external-configuration.adoc

@@ -0,0 +1,24 @@
+// This assembly is included in the following assemblies:
+//
+// assembly-deployment-configuration-kafka-connect.adoc
+// assembly-deployment-configuration-kafka-connect-s2i.adoc
+
+// Save the context of the assembly that is including this one.
+// This is necessary for including assemblies in assemblies.
+// See also the complementary step on the last line of this file.
+
+[id='assembly-kafka-connect-external-configuration-{context}']
+
+= Using external configuration and secrets
+
+Kafka Connect connectors have their own configuration.
+They are configured using the REST interface.
+The connector configuration is passed to Kafka Connect as part of the HTTP requests.
+In some cases, it might make sense to externalize the configuration of Kafka Connect connectors and keep it in the regular Kubernetes resources such as ConfigMaps or Secrets.
+This applies especially to confidential data such as usernames, passwords or certificates.
+
+include::con-kafka-connect-external-configuration.adoc[leveloffset=+1]
+
+include::proc-kafka-connect-mounting-secrets-as-environment-variables.adoc[leveloffset=+1]
+
+include::proc-kafka-connect-mounting-volumes.adoc[leveloffset=+1]

documentation/book/con-kafka-connect-external-configuration.adoc

@@ -0,0 +1,92 @@
+// This assembly is included in the following assemblies:
+//
+// assembly-kafka-connect-external-configuration.adoc
+
+[id='con-kafka-connect-external-configuration-{context}']
+
+= Externalizing Kafka Connect connector configuration
+
+{ProductName} lets you to mount ConfigMaps or Secrets into the Kafka Connect pods as either volumes or environment variables.
+The volumes and environment variables can be configured in the `externalConfiguration` property in `KafkaConnect.spec` and `KafkaConnectS2I.spec`
+
+== External configuration as environment variable
+
+The `env` property can be used to specify one or more environment variables.
+These variables can contain a value from a ConfigMap or from a Secret.
+The names of the environment variables configured by the user cannot start with `KAFKA_` or `STRIMZI_`.
+
+To mount a value from a Secret to an environment variable, you have to use the `valueFrom` property and the `secretKeyReference`
+
+.Example of environment variable set to a value from a Secret
+[source,yaml,subs="attributes+"]
+----
+apiVersion: {KafkaApiVersion}
+kind: KafkaConnect
+metadata:
+  name: my-connect
+spec:
+  # ...
+  externalConfiguration:
+    env:
+      - name: MY_ENVIRONMENT_VARIABLE
+        valueFrom:
+          secretKeyRef:
+            name: my-secret
+            key: my-key
+----
+
+A common use case for mounting Secrets into environment variables is for example when your connector needs to communicate with Amazon AWS and needs to read the `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables with credentials.
+
+To mount a value from a ConfigMap use `configMapKeyRef` in the `valueFrom` property.
+
+.Example of environment variable set to a value from a ConfigMap
+[source,yaml,subs="attributes+"]
+----
+apiVersion: {KafkaApiVersion}
+kind: KafkaConnect
+metadata:
+  name: my-connect
+spec:
+  # ...
+  externalConfiguration:
+    env:
+      - name: MY_ENVIRONMENT_VARIABLE
+        valueFrom:
+          configMapKeyRef:
+            name: my-config-map
+            key: my-key
+----
+
+== External configuration as volume
+
+The ConfigMaps and Secrets can be also mounted as volumes.
+Using volumes instead of environment variables migth be useful for example for:
+* Mounting truststores / keystores with TLS certificates
+* Mounting properties file which should used to configure Kafka Connect connectors
+
+The ConfigMaps and Secrets which should be mounted as volumes should be listed in the `volumes` property of `externalConfiguration`.
+Each volume has to specify a name in the `name` property and a reference to ConfigMap or Secret.
+
+.Example of volumes with external configuration
+[source,yaml,subs="attributes+"]
+----
+apiVersion: {KafkaApiVersion}
+kind: KafkaConnect
+metadata:
+  name: my-connect
+spec:
+  # ...
+  externalConfiguration:
+    volumes:
+      - name: connector1
+        configMap:
+          name: connector1-configuration
+    - name: connector1-certificates
+      secret:
+        secretName: connector1-certificates
+----
+
+The volumes will be mounted inside the Kafka Connect containers in the path `/opt/kafka/external-configuration/_<volume-name>_`.
+For example the files from the volume named `connector1` will be in the directory `/opt/kafka/external-configuration/connector1`.
+
+The `FileConfigProvider` has to be used to read the values from the mounted properties files in connector configurations.

documentation/book/proc-kafka-connect-mounting-secrets-as-environment-variables.adoc

@@ -0,0 +1,75 @@
+// This assembly is included in the following assemblies:
+//
+// assembly-kafka-connect-external-configuration.adoc
+
+[id='proc-kafka-connect-mounting-secrets-as-environment-variables-{context}']
+
+= Mounting Secrets as environment variables
+
+This procedure shows how to use an {ProductPlatformName} Secret and mount it as environment variable into Kafka Connect.
+
+.Prerequisites
+
+* A running Cluster Operator.
+
+.Procedure
+
+. Create a secret containing the information which should be mounted as environment variable.
+For example:
++
+[source,yaml,subs=attributes+]
+----
+apiVersion: v1
+kind: Secret
+metadata:
+  name: aws-creds
+type: Opaque
+data:
+  awsAccessKey: QUtJQVhYWFhYWFhYWFhYWFg=
+  awsAecretAccessKey: Ylhsd1lYTnpkMjl5WkE=
+----
+
+. Create or edit the Kafka Connect resource.
+Configure the `externalConfiguration` section of the `KafkaConnect` or `KafkaConnectS2I` custom resource to reference the secret:
+For example:
++
+[source,yaml,subs="attributes+"]
+----
+apiVersion: {KafkaApiVersion}
+kind: KafkaConnect
+metadata:
+  name: my-connect
+spec:
+  # ...
+  externalConfiguration:
+    env:
+      - name: AWS_ACCESS_KEY_ID
+        valueFrom:
+          secretKeyRef:
+            name: aws-creds
+            key: awsAccessKey
+      - name: AWS_SECRET_ACCESS_KEY
+        valueFrom:
+          secretKeyRef:
+            name: aws-creds
+            key: awsAecretAccessKey
+----
+
+. Apply the changes to your Kafka Connect deployment.
++
+ifdef::Kubernetes[]
+On {KubernetesName} this can be done using `kubectl apply`:
+[source,shell,subs=+quotes]
+kubectl apply -f _your-file_
++
+endif::Kubernetes[]
+On {OpenShiftName} this can be done using `oc apply`:
++
+[source,shell,subs=+quotes]
+oc apply -f _your-file_
+
+. Use the environment variables in your connectors.
+
+.Additional resources
+
+* For more information about external configuration in Kafka Connect, see xref:type-ExternalConfiguration-reference[].

documentation/book/proc-kafka-connect-mounting-volumes.adoc

@@ -0,0 +1,89 @@
+// This assembly is included in the following assemblies:
+//
+// assembly-kafka-connect-external-configuration.adoc
+
+[id='proc-kafka-connect-mounting-volumes-{context}']
+
+= Mounting Secrets as volumes
+
+This procedure shows how to use an {ProductPlatformName} Secret, mount it as volume into Kafka Connect, and use it to configure Kafka Connect connector.
+
+.Prerequisites
+
+* A running Cluster Operator.
+
+.Procedure
+
+. Create a secret containing a properties file with the configuration options we want to use in our connector configuration.
+For example:
++
+[source,yaml,subs=attributes+]
+----
+apiVersion: v1
+kind: Secret
+metadata:
+  name: mysecret
+type: Opaque
+stringData:
+  connector.properties: |-
+    dbUsername: my-user
+    dbPassword: my-password
+----
+
+. Create or edit the Kafka Connect resource.
+Configure the `FileConfigProvider` in the `config` section and the `externalConfiguration` section of the `KafkaConnect` or `KafkaConnectS2I` custom resource to reference the secret:
+For example:
++
+[source,yaml,subs="attributes+"]
+----
+apiVersion: {KafkaApiVersion}
+kind: KafkaConnect
+metadata:
+  name: my-connect
+spec:
+  # ...
+  config:
+    config.providers: file
+    config.providers.file.class: org.apache.kafka.common.config.provider.FileConfigProvider
+  #...
+  externalConfiguration:
+    volumes:
+      - name: connector-config
+        secret:
+          secretName: mysecret
+----
+
+. Apply the changes to your Kafka Connect deployment.
++
+ifdef::Kubernetes[]
+On {KubernetesName} this can be done using `kubectl apply`:
+[source,shell,subs=+quotes]
+kubectl apply -f _your-file_
++
+endif::Kubernetes[]
+On {OpenShiftName} this can be done using `oc apply`:
++
+[source,shell,subs=+quotes]
+oc apply -f _your-file_
+
+. Use the values from the mounted properties file in your JSON payload with connector configuration.
+For example:
++
+[source,json,subs="attributes+"]
+----
+{  
+   "name":"my-connector",
+   "config":{
+      "connector.class":"MyDbConnector",
+      "tasks.max":"3",
+      "database": "my-postgresql:5432"
+      "username":"${file:/opt/kafka/external-configuration/connector-config/connector.properties:dbUsername}",
+      "password":"${file:/opt/kafka/external-configuration/connector-config/connector.properties:dbPassword}",
+      # ...
+   }
+}
+----
+
+.Additional resources
+
+* For more information about external configuration in Kafka Connect, see xref:type-ExternalConfiguration-reference[].