-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[enterprise-4.8]RHDEVDOCS-3226 Logging 5.2 docs aggregated branch
- Loading branch information
Showing
31 changed files
with
785 additions
and
403 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,27 +1,93 @@ | ||
| [id="cluster-logging-release-notes"] | ||
| = Release notes for Red Hat OpenShift Logging 5.1 | ||
| = Release notes for Red Hat OpenShift Logging 5.2 | ||
| include::modules/common-attributes.adoc[] | ||
| :context: cluster-logging-release-notes-v5x | ||
|
|
||
| toc::[] | ||
|
|
||
| [id="openshift-logging-about-this-release"] | ||
| == About this release | ||
|
|
||
| The following advisories are available for OpenShift Logging 5.1.x: | ||
|
|
||
| * link:https://access.redhat.com/errata/RHBA-2021:2885[RHBA-2021:2885 - Bug Fix Advisory. Openshift Logging Bug Fix Release 5.1.1] | ||
| * link:https://access.redhat.com/errata/RHBA-2021:2112[RHBA-2021:2112 - Bug Fix Advisory. OpenShift Logging Bug Fix Release 5.1.0] | ||
|
|
||
| [id="openshift-logging-supported-versions"] | ||
| == Supported versions | ||
|
|
||
| * OpenShift Logging version 5.1 runs on {product-title} versions 4.7 and 4.8. | ||
| * OpenShift Logging versions 5.0, 5.1, and 5.2 run on {product-title} versions 4.7 and 4.8. | ||
|
|
||
| [id="openshift-logging-inclusive-language"] | ||
| == Making open source more inclusive | ||
|
|
||
| Red Hat is committed to replacing problematic language in our code, documentation, and web properties. We are beginning with these four terms: master, slave, blacklist, and whitelist. Because of the enormity of this endeavor, these changes will be implemented gradually over several upcoming releases. For more details, see link:https://www.redhat.com/en/blog/making-open-source-more-inclusive-eradicating-problematic-language[Red Hat CTO Chris Wright’s message]. | ||
|
|
||
| // Release Notes by version | ||
|
|
||
| [id="cluster-logging-release-notes-5-2-0"] | ||
| == OpenShift Logging 5.2.0 | ||
|
|
||
| This release includes link:https://access.redhat.com/errata/RHBA-2021:3393[RHBA-2021:3393 OpenShift Logging Bug Fix Release 5.2.0]. | ||
|
|
||
| [id="openshift-logging-5-2-0-new-features-and-enhancements"] | ||
| === New features and enhancements | ||
|
|
||
| * With this update, you can forward log data to Amazon CloudWatch, which provides application and infrastructure monitoring. For more information, see xref:../logging/cluster-logging-external.html#cluster-logging-collector-log-forward-cloudwatch_cluster-logging-external[Forwarding logs to Amazon CloudWatch]. (link:https://issues.redhat.com/browse/LOG-1173[LOG-1173]) | ||
|
|
||
| * With this update, you can forward log data to Grafana Loki, a horizontally scalable, highly available, multi-tenant log aggregation system. For more information, see xref:../logging/cluster-logging-external.html#cluster-logging-collector-log-forward-loki_cluster-logging-external[Forwarding logs to Grafana Loki]. (link:https://issues.redhat.com/browse/LOG-684[LOG-684]) | ||
|
|
||
| * With this update, if you use the Fluentd forward protocol to forward log data over a TLS-encrypted connection, you can now use a password-encrypted private key file and specify the passphrase in the Cluster Log Forwarder configuration. For more information, see xref:../logging/cluster-logging-external.html#cluster-logging-collector-log-forward-fluentd_cluster-logging-external[Forwarding logs using the Fluentd forward protocol]. (link:https://issues.redhat.com/browse/LOG-1525[LOG-1525]) | ||
|
|
||
| * This enhancement enables you to use a username and password to authenticate a log forwarding connection to an external Elasticsearch instance. For example, if you cannot use mutual TLS (mTLS) because a third-party operates the Elasticsearch instance, you can use HTTP or HTTPS and set a secret that contains the username and password. For more information see xref:../logging/cluster-logging-external.adoc#cluster-logging-collector-log-forward-es_cluster-logging-external[Forwarding logs to an external Elasticsearch instance]. (link:https://issues.redhat.com/browse/LOG-1022[LOG-1022]) | ||
|
|
||
| * With this update, you can collect OVN network policy audit logs for forwarding to a logging server. For more information, see xref:../logging/cluster-logging-external.html#cluster-logging-collecting-ovn-audit-logs_cluster-logging-external[Collecting OVN network policy audit logs]. (link:https://issues.redhat.com/browse/LOG-1526[LOG-1526]) | ||
|
|
||
| * By default, the data model introduced in {product-title} 4.5 gave logs from different namespaces a single index in common. This change made it harder to see which namespaces produced the most logs. | ||
| + | ||
| The current release, OpenShift Logging 5.2, adds namespace metrics to the *Logging* dashboard in the {product-title} console. With these metrics, you can see which namespaces produce logs and how many logs each namespace produces for a given timestamp. | ||
| + | ||
| To see these metrics, open the *Administrator* perspective in the {product-title} web console, and navigate to *Monitoring* -> *Dashboards* -> *Logging/Elasticsearch*. (link:https://issues.redhat.com/browse/LOG-1680[LOG-1680]) | ||
|
|
||
| * The current release, OpenShift Logging 5.2, enables two new metrics: For a given timestamp or duration, you can see the total logs produced or logged by individual containers, and the total logs collected by the collector. These metrics are labeled by namespace, pod, and container name, so you can see how many logs each namespace and pod collects and produces. (link:https://issues.redhat.com/browse/LOG-1213[LOG-1213]) | ||
|
|
||
| [id="openshift-logging-5-2-0-bug-fixes"] | ||
| === Bug fixes | ||
|
|
||
| * link:https://issues.redhat.com/browse/LOG-1130[LOG-1130] "BZ#1927249 - fieldmanager.go:186 - SHOULD NOT HAPPEN - failed to update managedFields...duplicate entries for key 'name="POLICY_MAPPING'" | ||
| * link:https://issues.redhat.com/browse/LOG-1268[LOG-1268] "elasticsearch-im-{app,infra,audit} successfully run but fail to run their tasks." | ||
| * link:https://issues.redhat.com/browse/LOG-1271[LOG-1271] "Logs Produced Line Chart does not display podname and namespace labels" | ||
| * link:https://issues.redhat.com/browse/LOG-1273[LOG-1273] "The index management job status is always 'Completed' even when there has an error in the job log." | ||
| * link:https://issues.redhat.com/browse/LOG-1385[LOG-1385] "APIRemovedInNextReleaseInUse alert for priorityclasses" | ||
| * link:https://issues.redhat.com/browse/LOG-1420[LOG-1420] "Operators missing disconnected annotation" | ||
| * link:https://issues.redhat.com/browse/LOG-1440[LOG-1440] "BZ#1966561 - OLM bug workaround for workload partitioning (PR#1042)" | ||
| * link:https://issues.redhat.com/browse/LOG-1499[LOG-1499] "release-5.2 - Error 'com.fasterxml.jackson.core.JsonParseException: Invalid UTF-8 start byte 0x92' in Elasticsearch/Fluentd logs" | ||
| * link:https://issues.redhat.com/browse/LOG-1567[LOG-1567] "Use correct variable for nextIndex" | ||
| * link:https://issues.redhat.com/browse/LOG-1446[LOG-1446] "kibana-proxy CrashLoopBackoff with error Invalid configuration cookie_secret must be 16, 24, or 32 bytes to create an AES cipher" | ||
| * link:https://issues.redhat.com/browse/LOG-1625[LOG-1625] "ds/fluentd is not created due to: 'system:serviceaccount:openshift-logging:cluster-logging-operator' cannot create resource 'securitycontextconstraints' in API group 'security.openshift.io' at the cluster scope" | ||
| * link:https://issues.redhat.com/browse/LOG-1071[LOG-1071] "fluentd configuration posting all messages to its own log" | ||
| * link:https://issues.redhat.com/browse/LOG-1276[LOG-1276] "Update Elasticsearch/kibana to use opendistro security plugin 2.10.5.1" | ||
| * link:https://issues.redhat.com/browse/LOG-1353[LOG-1353] "No datapoints found on top 10 containers dashboard" | ||
| * link:https://issues.redhat.com/browse/LOG-1411[LOG-1411] "Underestimate queued_chunks_limit_size value with chunkLimitSize and totalLimitSize tuning parameters" | ||
| * link:https://issues.redhat.com/browse/LOG-1558[LOG-1558] "Update OD security dependency to resolve kibana index migration issue" | ||
| * link:https://issues.redhat.com/browse/LOG-1562[LOG-1562] "CVE-2021-32740 logging-fluentd-container: rubygem-addressable: ReDoS in templates - openshift-logging-5" | ||
| * link:https://issues.redhat.com/browse/LOG-1570[LOG-1570] "Bug 1981579: Fix built-in application behavior to collect all of logs" | ||
| * link:https://issues.redhat.com/browse/LOG-1589[LOG-1589] "There are lots of dockercfg secrets and service-account-token secrets for ES and Kibana in openshift-logging namespace after deploying EFK pods." | ||
| * link:https://issues.redhat.com/browse/LOG-1590[LOG-1590] "Vendored viaq/logerr dependency is missing a license file" | ||
| * link:https://issues.redhat.com/browse/LOG-1623[LOG-1623] "Metric`log_collected_bytes_total` is not exposed" | ||
| * link:https://issues.redhat.com/browse/LOG-1624[LOG-1624] "Index management cronjobs are using wrong image in CSV/elasticsearch-operator.5.2.0-1" | ||
| * link:https://issues.redhat.com/browse/LOG-1634[LOG-1634] "Logging 5.2 - The CSV version is not changed in new bundles." | ||
| * link:https://issues.redhat.com/browse/LOG-1647[LOG-1647] "Fluentd pods raise error `Prometheus::Client::LabelSetValidator::InvalidLabelSetError` when forward logs to external logStore" | ||
| * link:https://issues.redhat.com/browse/LOG-1657[LOG-1657] "Index management jobs failing with error while attemping to determine the active write alias no permissions" | ||
| * link:https://issues.redhat.com/browse/LOG-1681[LOG-1681] "Fluentd pod metric not able to scrape , Fluentd target down" | ||
| * link:https://issues.redhat.com/browse/LOG-1683[LOG-1683] "Loki output not present in CLO CRD" | ||
| * link:https://issues.redhat.com/browse/LOG-1702[LOG-1702] "Entry out of order when forward logs to loki" | ||
| * link:https://issues.redhat.com/browse/LOG-1714[LOG-1714] "Memory/CPU spike issues seen with Logging 5.2 on Power" | ||
| * link:https://issues.redhat.com/browse/LOG-1722[LOG-1722] "The value of card `Total Namespace Count` in Logging/Elasticsearch dashboard is not correct." | ||
| * link:https://issues.redhat.com/browse/LOG-1723[LOG-1723] "In fluentd config, flush_interval can't be set with flush_mode=immediate" | ||
|
|
||
| [id="openshift-logging-5-2-0-known-issues"] | ||
| === Known issues | ||
|
|
||
| * If you forward logs to an external Elasticsearch server and then change a configured value in the pipeline secret, such as the username and password, the fluentd forwarder loads the new secret but uses the old value to connect to an external Elasticsearch server. This issue happens because the Red Hat OpenShift Logging Operator does not currently monitor secrets for content changes. (link:https://issues.redhat.com/browse/LOG-1652[LOG-1652]) | ||
| + | ||
| As a workaround, if you change the secret, you can force the Fluentd pods to redeploy by entering: | ||
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ oc delete pod -l component=fluentd | ||
| ---- | ||
|
|
||
| include::modules/cluster-logging-release-notes-5.1.0.adoc[leveloffset=+1] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
88 changes: 88 additions & 0 deletions
88
modules/cluster-logging-collector-collecting-ovn-logs.adoc
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,88 @@ | ||
| [id="cluster-logging-collecting-ovn-audit-logs_{context}"] | ||
| = Collecting OVN network policy audit logs | ||
|
|
||
| You can collect the OVN network policy audit logs from the `/var/log/ovn/acl-audit-log.log` file on OVN-Kubernetes pods and forward them to logging servers. | ||
|
|
||
| .Prerequisites | ||
|
|
||
| * You are using {product-title} version 4.8 or later. | ||
| * You are using Cluster Logging version 5.2 or later. | ||
| * You have already set up a `ClusterLogForwarder` custom resource (CR) object. | ||
| * The {product-title} cluster is configured for OVN-Kubernetes network policy audit logging. See the following "Additional resources" section. | ||
|
|
||
| [NOTE] | ||
| ==== | ||
| Often, logging servers that store audit data must meet organizational and governmental requirements for compliance and security. | ||
| ==== | ||
|
|
||
| .Procedure | ||
|
|
||
| . Create or edit a YAML file that defines the `ClusterLogForwarder` CR object as described in other topics on forwarding logs to third-party systems. | ||
|
|
||
| . In the YAML file, add the `audit` log type to the `inputRefs` element in a pipeline. For example: | ||
| + | ||
| [source,yaml] | ||
| ---- | ||
| pipelines: | ||
| - name: audit-logs | ||
| inputRefs: | ||
| - audit <1> | ||
| outputRefs: | ||
| - secure-logging-server <2> | ||
| ---- | ||
| <1> Specify `audit` as one of the log types to input. | ||
| <2> Specify the output that connects to your logging server. | ||
|
|
||
| . Recreate the updated CR object: | ||
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ oc create -f <file-name>.yaml | ||
| ---- | ||
|
|
||
| .Verification | ||
|
|
||
| Verify that audit log entries from the nodes that you are monitoring are present among the log data gathered by the logging server. | ||
|
|
||
| Find an original audit log entry in `/var/log/ovn/acl-audit-log.log` and compare it with the corresponding log entry on the logging server. | ||
|
|
||
| For example, an original log entry in `/var/log/ovn/acl-audit-log.log` might look like this: | ||
|
|
||
| [source,txt] | ||
| ---- | ||
| 2021-07-06T08:26:58.687Z|00004|acl_log(ovn_pinctrl0)|INFO|name="verify-audit- | ||
| logging_deny-all", verdict=drop, severity=alert: | ||
| icmp,vlan_tci=0x0000,dl_src=0a:58:0a:81:02:12,dl_dst=0a:58:0a:81:02:14,nw_src=10 | ||
| .129.2.18,nw_dst=10.129.2.20,nw_tos=0,nw_ecn=0,nw_ttl=64,icmp_type=8,icmp_code=0 | ||
| ---- | ||
|
|
||
| And the corresponding OVN audit log entry you find on the logging server might look like this: | ||
|
|
||
| [source,json] | ||
| ---- | ||
| { | ||
| "@timestamp" : "2021-07-06T08:26:58..687000+00:00", | ||
| "hostname":"ip.abc.iternal", | ||
| "level":"info", | ||
| "message" : "2021-07-06T08:26:58.687Z|00004|acl_log(ovn_pinctrl0)|INFO|name=\"verify-audit-logging_deny-all\", verdict=drop, severity=alert: icmp,vlan_tci=0x0000,dl_src=0a:58:0a:81:02:12,dl_dst=0a:58:0a:81:02:14,nw_src=10.129.2.18,nw_dst=10.129.2.20,nw_tos=0,nw_ecn=0,nw_ttl=64,icmp_type=8,icmp_code=0" | ||
| } | ||
| ---- | ||
|
|
||
| Where: | ||
|
|
||
| * `@timestamp` is the timestamp of the log entry. | ||
| * `hostname` is the node from which the log originated. | ||
| * `level` is the log entry. | ||
| * `message` is the original audit log message. | ||
|
|
||
| [NOTE] | ||
| ==== | ||
| On an Elasticsearch server, look for log entries whose indices begin with `audit-00000`. | ||
| ==== | ||
|
|
||
| .Troubleshooting | ||
|
|
||
| . Verify that your {product-title} cluster meets all the prerequisites. | ||
| . Verify that you have completed the procedure. | ||
| . Verify that the nodes generating OVN logs are enabled and have `/var/log/ovn/acl-audit-log.log` files. | ||
| . Check the Fluentd pod logs for issues. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.