Skip to content

Commit

Permalink
Merge pull request #37798 from libander/RHDEVDOCS-3121
Browse files Browse the repository at this point in the history
RHDEVDOCS-3121 - Relax Authorization Requirements for Log Forwarding
  • Loading branch information
JStickler committed Nov 15, 2021
2 parents 56fd01e + e259a2e commit 8cde50a
Showing 1 changed file with 36 additions and 10 deletions.
46 changes: 36 additions & 10 deletions modules/cluster-logging-collector-log-forwarding-about.adoc
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
[id="cluster-logging-collector-log-forwarding-about_{context}"]
= About forwarding logs to third-party systems

Forwarding cluster logs to external third-party systems requires a combination of _outputs_ and _pipelines_ specified in a `ClusterLogForwarder` custom resource (CR) to send logs to specific endpoints inside and outside of your {product-title} cluster. You can also use _inputs_ to forward the application logs associated with a specific project to an endpoint.
To send logs to specific endpoints inside and outside your {product-title} cluster, you specify a combination of _outputs_ and _pipelines_ in a `ClusterLogForwarder` custom resource (CR). You can also use _inputs_ to forward the application logs associated with a specific project to an endpoint. Authentication is provided by a Kubernetes _Secret_ object.

* An _output_ is the destination for log data that you define, or where you want the logs sent. An output can be one of the following types:
_output_:: The destination for log data that you define, or where you want the logs sent. An output can be one of the following types:
+
--
* `elasticsearch`. An external Elasticsearch instance. The `elasticsearch` output can use a TLS connection.
Expand All @@ -21,9 +21,7 @@ Forwarding cluster logs to external third-party systems requires a combination o
* `default`. The internal {product-title} Elasticsearch instance. You are not required to configure the default output. If you do configure a `default` output, you receive an error message because the `default` output is reserved for the Red Hat OpenShift Logging Operator.
--
+
If the output URL scheme requires TLS (HTTPS, TLS, or UDPS), then TLS server-side authentication is enabled. To also enable client authentication, the output must name a secret in the `openshift-logging` project. The secret must have keys of: *tls.crt*, *tls.key*, and *ca-bundle.crt* that point to the respective certificates that they represent.

* A _pipeline_ defines simple routing from one log type to one or more outputs, or which logs you want to send. The log types are one of the following:
_pipeline_:: Defines simple routing from one log type to one or more outputs, or which logs you want to send. The log types are one of the following:
+
--
* `application`. Container logs generated by user applications running in the cluster, except infrastructure container applications.
Expand All @@ -33,11 +31,16 @@ If the output URL scheme requires TLS (HTTPS, TLS, or UDPS), then TLS server-sid
* `audit`. Audit logs generated by the node audit system, `auditd`, Kubernetes API server, OpenShift API server, and OVN network.
--
+
You can add labels to outbound log messages by using `key:value` pairs in the pipeline. For example, you might add a label to messages that are forwarded to others data centers or label the logs by type. Labels that are added to objects are also forwarded with the log message.

* An _input_ forwards the application logs associated with a specific project to a pipeline.
You can add labels to outbound log messages by using `key:value` pairs in the pipeline. For example, you might add a label to messages that are forwarded to other data centers or label the logs by type. Labels that are added to objects are also forwarded with the log message.

_input_:: Forwards the application logs associated with a specific project to a pipeline.
+
--
In the pipeline, you define which log types to forward using an `inputRef` parameter and where to forward the logs to using an `outputRef` parameter.
--
+

_Secret_:: A `key:value map` that contains confidential data such as user credentials.

Note the following:

Expand All @@ -51,8 +54,6 @@ Note the following:

* The internal {product-title} Elasticsearch instance does not provide secure storage for audit logs. We recommend you ensure that the system to which you forward audit logs is compliant with your organizational and governmental regulations and is properly secured. OpenShift Logging does not comply with those regulations.

* You are responsible for creating and maintaining any additional configurations that external destinations might require, such as keys and secrets, service accounts, port openings, or global proxy configuration.

The following example forwards the audit logs to a secure external Elasticsearch instance, the infrastructure logs to an insecure external Elasticsearch instance, the application logs to a Kafka broker, and the application logs from the `my-apps-logs` project to the internal Elasticsearch instance.

.Sample log forwarding outputs and pipelines
Expand Down Expand Up @@ -150,3 +151,28 @@ spec:
== Fluentd log handling when the external log aggregator is unavailable

If your external logging aggregator becomes unavailable and cannot receive logs, Fluentd continues to collect logs and stores them in a buffer. When the log aggregator becomes available, log forwarding resumes, including the buffered logs. If the buffer fills completely, Fluentd stops collecting logs. {product-title} rotates the logs and deletes them. You cannot adjust the buffer size or add a persistent volume claim (PVC) to the Fluentd daemon set or pods.

[discrete]
== Supported Authorization Keys
Common key types are provided here. Some output types support additional specialized keys, documented with the output-specific configuration field. All secret keys are optional. Enable the security features you want by setting the relevant keys. You are responsible for creating and maintaining any additional configurations that external destinations might require, such as keys and secrets, service accounts, port openings, or global proxy configuration. Open Shift Logging will not attempt to verify a mismatch between authorization combinations.

Transport Layer Security (TLS):: Using a TLS URL ('http://...' or 'ssl://...') without a Secret enables basic TLS server-side authentication. Additional TLS features are enabled by including a Secret and setting the following optional fields:

* `tls.crt`: (string) File name containing a client certificate. Enables mutual authentication. Requires `tls.key`.

* `tls.key`: (string) File name containing the private key to unlock the client certificate. Requires `tls.crt`.

* `passphrase`: (string) Passphrase to decode an encoded TLS private key. Requires `tls.key`.

* `ca-bundle.crt`: (string) File name of a customer CA for server authentication.

Username and Password::
* `username`: (string) Authentication user name. Requires `password`.
* `password`: (string) Authentication password. Requires `username`.

Simple Authentication Security Layer (SASL)::
* `sasl.enable` (boolean) Explicitly enable or disable SASL.
If missing, SASL is automatically enabled when any of the other `sasl.` keys are set.
* `sasl.mechanisms`: (array) List of allowed SASL mechanism names.
If missing or empty, the system defaults are used.
* `sasl.allow-insecure`: (boolean) Allow mechanisms that send clear-text passwords. Defaults to false.

0 comments on commit 8cde50a

Please sign in to comment.