Logging 6x - Docs Outline / Planning #76623

libander · 2024-05-29T00:23:22Z

Version(s): 4.16+

Issue: https://issues.redhat.com/browse/OBSDOCS-1099

Link to docs previews:

https://76623--ocpdocs-pr.netlify.app/openshift-enterprise/latest/observability/logging/logging-6.0/log6x-about.html
https://76623--ocpdocs-pr.netlify.app/openshift-enterprise/latest/observability/logging/logging-6.0/log6x-clf.html
https://76623--ocpdocs-pr.netlify.app/openshift-enterprise/latest/observability/logging/logging-6.0/log6x-loki.html
https://76623--ocpdocs-pr.netlify.app/openshift-enterprise/latest/observability/logging/logging-6.0/log6x-start.html

Additional information: Not intended to merge, this PR is for planning & collaboration.

ocpdocs-previewbot · 2024-05-29T00:32:49Z

🤖 Fri Jun 28 15:54:19 - Prow CI generated the docs preview:

https://76623--ocpdocs-pr.netlify.app/
https://76623--ocpdocs-pr.netlify.app/openshift-dedicated/latest/observability/logging/performance_reliability/logging-content-filtering.html
https://76623--ocpdocs-pr.netlify.app/openshift-enterprise/latest/observability/logging/logging-6.0/log6x-about.html
https://76623--ocpdocs-pr.netlify.app/openshift-enterprise/latest/observability/logging/logging-6.0/log6x-clf.html
https://76623--ocpdocs-pr.netlify.app/openshift-enterprise/latest/observability/logging/logging-6.0/log6x-loki.html
https://76623--ocpdocs-pr.netlify.app/openshift-enterprise/latest/observability/logging/logging-6.0/log6x-start.html
https://76623--ocpdocs-pr.netlify.app/openshift-enterprise/latest/observability/logging/performance_reliability/logging-content-filtering.html
https://76623--ocpdocs-pr.netlify.app/openshift-rosa/latest/observability/logging/performance_reliability/logging-content-filtering.html

libander · 2024-06-12T03:11:02Z

/retest

libander · 2024-06-26T09:29:33Z

@r2d2rnd

modules/logging-elastic-to-loki-store.adoc

libander · 2024-06-28T11:41:53Z

observability/logging/logging-6.0/log6x-clf.adoc

+openshiftLabels:: Adds OpenShift-specific labels to log messages.
+
+// Needs to be validated by engineering for 6.0.
+include::modules/logging-audit-log-filtering.adoc[leveloffset=+1]


@anpingli / @jcantrill : need to know if this can be used for 6.0 & if not what changes are needed.

https://github.com/openshift/openshift-docs/blob/enterprise-4.16/modules/logging-audit-log-filtering.adoc

Created a #78477 for verification of the CLF content.

There isn't changes for kubeAPIAudit. But we need to update the example.

libander · 2024-06-28T11:42:05Z

observability/logging/logging-6.0/log6x-clf.adoc

+include::modules/logging-audit-log-filtering.adoc[leveloffset=+1]
+
+// Needs to be validated by engineering for 6.0.
+include::modules/logging-content-filter-drop-records.adoc[leveloffset=+1]


@anpingli / @jcantrill : need to know if this can be used for 6.0 & if not what changes are needed.

https://github.com/openshift/openshift-docs/blob/enterprise-4.16/modules/logging-content-filter-drop-records.adoc

Created a #78477 for verification of the CLF content.

We can reuse it. there are some bugs. we can summary the status before release.

libander · 2024-06-28T11:42:12Z

observability/logging/logging-6.0/log6x-clf.adoc

+include::modules/logging-content-filter-drop-records.adoc[leveloffset=+1]
+
+// Needs to be validated by engineering for 6.0.
+include::modules/logging-content-filter-prune-records.adoc[leveloffset=+1]


@anpingli / @jcantrill : need to know if this can be used for 6.0 & if not what changes are needed.

https://github.com/openshift/openshift-docs/blob/enterprise-4.16/modules/logging-content-filter-prune-records.adoc

Created a #78477 for verification of the CLF content.

yes. we can use it.

libander · 2024-06-28T11:42:56Z

observability/logging/logging-6.0/log6x-clf.adoc

+* Indicates management state (managed or unmanaged)
+
+// Needs engineering eval re applicability to 6.0
+include::modules/log-forwarding-implementations.adoc[leveloffset=+1]


@anpingli / @jcantrill : need to know if this can be used for 6.0 & if not what changes are needed.

What is "this"?.... is it the following verbiage of log6x-clf.adoc?

modules/log-forwarding-implementations.adoc

Created a PR for verification of the CLF content.

Will update in the verification PR.

libander · 2024-06-28T11:43:12Z

observability/logging/logging-6.0/log6x-clf.adoc

+
+[id="log6x-CLF-tuning_{context}"]
+== Output tuning
+//need to validate how much of this applies to 6.0 & edit as needed.


@anpingli / @jcantrill : need to know if this can be used for 6.0 & if not what changes are needed.

Tuning largely remains the same though it was moved down to the individual output types since not all things applied to everyone

We'll need the particulars of each for https://issues.redhat.com/browse/OBSDOCS-1169. Are they in the repo somewhere?

We have a Jira: OBSDOCS-1169 for Output tuning section discussion. I believe this discussion can be covered in PR , which is created for verification of the CLF content. So, that is the reason I have covered OBSDOCS-1170 and OBSDOCS-1169 in one PR. Please let me know if you have any other thoughts. Thanks!

Cc @jcantrill @anpingli @libander @briandooley

modules/logging-6x-v-5x.adoc

jcantrill · 2024-06-28T12:24:26Z

modules/logging-6x-v-5x.adoc

+
+* **Improved Query Performance:** Loki's indexing and query mechanisms are optimized for speed, enabling faster log retrieval and analysis.
+
+* **Streamlined Configuration:** The removal of Elasticsearch and Fluentd simplify setup and configuration.


It's arguable if this streamlines configuration. These are just technologies we no longer offer in 6.0

Simplifies? Going from 4 different config options to 1 is certainly...something.

modules/logging-6x-v-5x.adoc

observability/logging/logging-6.0/.logging-enhancement-proposals.adoc

jcantrill · 2024-06-28T12:32:51Z

observability/logging/logging-6.0/log6x-clf.adoc

+* Indicates management state (managed or unmanaged)
+
+// Needs engineering eval re applicability to 6.0
+include::modules/log-forwarding-implementations.adoc[leveloffset=+1]


What is "this"?.... is it the following verbiage of log6x-clf.adoc?

jcantrill · 2024-06-28T12:34:01Z

observability/logging/logging-6.0/log6x-clf.adoc

+
+[id="log6x-CLF-tuning_{context}"]
+== Output tuning
+//need to validate how much of this applies to 6.0 & edit as needed.


Tuning largely remains the same though it was moved down to the individual output types since not all things applied to everyone

observability/logging/logging-6.0/log6x-clf.adoc

openshift-ci · 2024-06-28T15:54:25Z

@libander: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/validate-portal	`aef31f5`	link	true	`/test validate-portal`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

acandelp · 2024-07-01T10:51:20Z

Hello Libby,

I want to add a comment only about the scheleton and in the code I will put some notes about my findings but as you commented, the priority is that it is well organized.

The following sections are under Performance and reliability tuning and also under Configuring log Forwarding.:
Filtering logs by content
Filtering logs by metadata

In the latest versions, they are under Performance and reliability.

From my perspective I think of two options.

Leave it as in the latest versions and also add Forwarding application logs from specific projects and Forwarding application logs from specific pods sections or,
Leave it under Configuring log Forwarding and create a new content inside this section adding these filters and Forwarding application logs from specific projects and Forwarding application logs from specific pods sections (Don't forget to add how to send the logs to an external third-party).

Configuring LokiStack storage section is fine but inside Log Storage section we found a section called Configuring the LokiStack log store and installing log storage. Maybe for the customer should be easy to have only one section with different contents that matches all the information from the three points.

These points may have been taken into consideration but I preferred to leave them reflected here. I will update and confirm that the review is progressing.

Thank you very much for your hard work on the documentation

anpingli · 2024-07-23T07:25:42Z

/assign @anpingli

r2d2rnd · 2024-08-21T21:39:19Z

modules/logging-content-filter-drop-records.adoc


-* You have installed the {clo}.
 * You have administrator permissions.
 * You have created a `ClusterLogForwarder` custom resource (CR).


This is not correct. You can create a clusterLogForwarder as the first time with all the needed and not needed to have it previously created.

What it's a requirement is to have create a serviceAccount to be used in the clusterLogForwarder linked to the correct clusterRoles depending on the log types allowed to this serviceAccount to collect.

All the details are already mentioned in the PR https://github.com/openshift/openshift-docs/pull/80384/files exactly here https://github.com/openshift/openshift-docs/pull/80384/files#diff-7312d97cd1d4f7bf7f053f75108d26c4898afe1c9442033332be3b09358f75b5 and here https://github.com/openshift/openshift-docs/pull/80384/files#diff-8a01457b83c79be59c3ecdcc18d761f057790cab1de118070de9608994e11bf2

@libander

r2d2rnd · 2024-08-21T21:43:15Z

modules/log6x-oc-explain.adoc

+
+= Using the `oc explain` command
+
+The `oc explain` command is an essential tool in the OpenShift Command-Line Interface (CLI) that provides detailed descriptions of the fields within Kubernetes resources, including Custom Resources (CRs) introduced by Operators. This command is invaluable for administrators and developers who are configuring or troubleshooting resources in an OpenShift cluster.


It should be useful an example in this section, for example, how to review the outputs and options supported for an specific output. This should be so much helpful as the most of the people doesn't know how to run it or what they need to write

oc explain <API>.spec....

@libander

oc explain lokistacks.loki.grafana.com
oc explain recordingrules.loki.grafana.com
oc explain rulerconfigs.loki.grafana.com
oc explain alertingrules.loki.grafana.com
oc explain lokistacks.loki.grafana.com.spec
oc explain lokistacks.loki.grafana.com.spec.storage
oc explain lokistacks.loki.grafana.com.spec.storage.schemas.FIELDS
oc explain lokistacks.loki.grafana.com.spec.size

openshift-bot · 2024-11-24T09:00:58Z

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

openshift-merge-robot · 2024-11-24T09:01:07Z

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

openshift-bot · 2024-12-25T00:30:32Z

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

libander · 2024-12-25T21:27:56Z

/close

openshift-ci · 2024-12-25T21:28:05Z

@libander: Closed this PR.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

openshift-ci bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label May 29, 2024

libander force-pushed the Logging-6.0-PoC branch from a58004e to 650434d Compare May 29, 2024 00:25

libander force-pushed the Logging-6.0-PoC branch 2 times, most recently from 1d4b517 to 443ce25 Compare May 29, 2024 12:22

libander force-pushed the Logging-6.0-PoC branch from 443ce25 to 1a1b4a3 Compare June 11, 2024 23:05

openshift-ci bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jun 11, 2024

libander force-pushed the Logging-6.0-PoC branch from 1a1b4a3 to 00200c6 Compare June 11, 2024 23:48

libander force-pushed the Logging-6.0-PoC branch from 00200c6 to 3c9cb59 Compare June 13, 2024 00:07

openshift-ci bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jun 13, 2024

libander force-pushed the Logging-6.0-PoC branch 4 times, most recently from bb1c386 to 73c4a2b Compare June 26, 2024 09:06

libander changed the title ~~Logging 6x - POC WIP~~ Logging 6x - Docs Outline / Planning Jun 26, 2024

libander force-pushed the Logging-6.0-PoC branch 2 times, most recently from e4d02d1 to f79cf15 Compare June 26, 2024 09:22

libander force-pushed the Logging-6.0-PoC branch from f79cf15 to 6da32ad Compare June 26, 2024 11:07

acandelp reviewed Jun 26, 2024

View reviewed changes

modules/logging-elastic-to-loki-store.adoc Show resolved Hide resolved

libander force-pushed the Logging-6.0-PoC branch from 6da32ad to bcd2b50 Compare June 26, 2024 11:18

acandelp reviewed Jun 26, 2024

View reviewed changes

modules/logging-elastic-to-loki-store.adoc Show resolved Hide resolved

libander force-pushed the Logging-6.0-PoC branch from bcd2b50 to aad87d1 Compare June 26, 2024 11:29

libander commented Jun 28, 2024

View reviewed changes

jcantrill reviewed Jun 28, 2024

View reviewed changes

libander force-pushed the Logging-6.0-PoC branch from aad87d1 to ca79a62 Compare June 28, 2024 13:23

Logging 6x - Docs Skeleton for Collab/Feedback.

aef31f5

libander force-pushed the Logging-6.0-PoC branch from ca79a62 to aef31f5 Compare June 28, 2024 15:46

r2d2rnd reviewed Aug 21, 2024

View reviewed changes

openshift-ci bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Nov 24, 2024

openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 24, 2024

openshift-ci bot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Dec 25, 2024

openshift-ci bot closed this Dec 25, 2024


		* Improved Query Performance: Loki's indexing and query mechanisms are optimized for speed, enabling faster log retrieval and analysis.

		* Streamlined Configuration: The removal of Elasticsearch and Fluentd simplify setup and configuration.


		= Using the `oc explain` command

		The `oc explain` command is an essential tool in the OpenShift Command-Line Interface (CLI) that provides detailed descriptions of the fields within Kubernetes resources, including Custom Resources (CRs) introduced by Operators. This command is invaluable for administrators and developers who are configuring or troubleshooting resources in an OpenShift cluster.

Logging 6x - Docs Outline / Planning #76623

Logging 6x - Docs Outline / Planning #76623

Uh oh!

Conversation

libander commented May 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ocpdocs-previewbot commented May 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

libander commented Jun 12, 2024

Uh oh!

libander commented Jun 26, 2024

Uh oh!

Uh oh!

Uh oh!

libander Jun 28, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

libander Jun 28, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

openshift-ci bot commented Jun 28, 2024

Uh oh!

acandelp commented Jul 1, 2024

Uh oh!

anpingli commented Jul 23, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

libander commented May 29, 2024 •

edited

Loading

ocpdocs-previewbot commented May 29, 2024 •

edited

Loading

libander Jun 28, 2024 •

edited

Loading

libander Jun 28, 2024 •

edited

Loading