[Hold review] OBSDOCS-1170 - Created a PR for verification of the CLF content #78477

smunje1 · 2024-07-04T07:04:17Z

OBSDOCS-1170 : log6x-clf.adoc - include verification / modifications
Aligned team: Observability - Logging
OCP version for cherry-picking:
JIRA issues: OBSDOCS-1170
Preview pages: https://78477--ocpdocs-pr.netlify.app/openshift-enterprise/latest/observability/logging/logging-6.0/log6x-clf.html
SME review requested: @jcantrill
QE review requested: @anpingli
Peer review requested:

ocpdocs-previewbot · 2024-07-04T07:10:43Z

🤖 Mon Aug 12 13:53:31 - Prow CI generated the docs preview:

smunje1 · 2024-07-04T08:17:33Z

observability/logging/logging-6.0/log6x-clf.adoc

+* Indicates management state (managed or unmanaged)
+
+// Needs engineering eval re applicability to 6.0
+include::modules/log-forwarding-implementations.adoc[leveloffset=+1]


@anpingli / @jcantrill : Could you help me understand if this module can be used for 6.0 & if not what changes are needed?

How can I comment on modules/log-forwarding-implementations.adoc?

As we discussed, there are no changes needed for modules/log-forwarding-implementations.adoc. Do you have anything else that you would like to suggest for this module?

https://github.com/openshift/openshift-docs/blob/a199424efd582d3fb7d40c267744e285bd359480/modules/log-forwarding-implementations.adoc
@smunje1 IMO ⬆️ is no longer applicable since Logging 6.0 has no concept of legacy vs multi. All deployments are multi and will be deployed under the same rules; there is no special handling

I am planning to remove the Log forwarding implementations, Legacy implementation sections, and rename Multi log forwarder feature -> Creating a log forwarder. I also modified Creating a log forwarder section.

Please let me know if you have other thoughts. Thanks!

smunje1 · 2024-07-04T08:18:52Z

observability/logging/logging-6.0/log6x-clf.adoc

+[id="log6x-CLF-tuning_{context}"]
+== Output tuning
+//need to validate how much of this applies to 6.0 & edit as needed.
+include::modules/logging-delivery-tuning.adoc[leveloffset=+1]


@anpingli / @jcantrill : Could you help me understand if this module can be used for 6.0 & if not what changes are needed?

Output tuning is functionally the same as in 5.9 though it has moved from common to outputs to each individual spec. This was done because not all outputs support the same options

AtLeastOnce is atLeastOnce in 6.-0
AtMostOnce is atMostOnce in 6.0

May be we need to update ClusterLogForwarder CR tuning options as follows?

apiVersion: "observability.openshift.io/v1" kind: ClusterLogForwarder metadata: # ... spec: outputs: tuning: delivery: atLeastOnce maxWrite: <integer> compression: none minRetryDuration: 1s maxRetryDuration: 1s # ...

apiVersion: "observability.openshift.io/v1" kind: ClusterLogForwarder metadata: # ... spec: outputs: - name : <output-name> type: <output_type> <output_type>: tuning: delivery: atLeastOnce maxWrite: <integer> compression: none minRetryDuration: 1s maxRetryDuration: 1s

please refer to https://github.com/openshift/enhancements/blob/master/enhancements/cluster-logging/logs-observability-openshift-io-apis.md#clusterlogforwarer-customresourcedefinition

This problem in the current documentation is reflected in documentation bug https://issues.redhat.com/browse/OBSDOCS-1049

Tagging @libander so that we can cover this as part of OBSDOCS-1169.

smunje1 · 2024-07-04T08:30:19Z

observability/logging/logging-6.0/log6x-clf.adoc

+openshiftLabels:: Adds OpenShift-specific labels to log messages.
+
+// All of these need to be validated by engineering for 6.0.
+include::modules/logging-audit-log-filtering.adoc[leveloffset=+1]


@anpingli / @jcantrill : Could you help me understand if this module can be used for 6.0 & if not what changes are needed?

https://github.com/openshift/openshift-docs/blob/enterprise-4.16/modules/logging-audit-log-filtering.adoc

This should be usable with the exception of needing to modify the CLF where it is contained

As discussed with @anpingli , there are no changes needed for Overview of API audit filter section. I just updated the apiVersion and metadata in Example audit policy.

@jcantrill Let me know if you have other thoughts.

smunje1 · 2024-07-04T08:31:51Z

observability/logging/logging-6.0/log6x-clf.adoc

+// All of these need to be validated by engineering for 6.0.
+include::modules/logging-audit-log-filtering.adoc[leveloffset=+1]
+
+include::modules/logging-content-filter-drop-records.adoc[leveloffset=+1]


@anpingli / @jcantrill : Could you help me understand if this module can be used for 6.0 & if not what changes are needed?

https://github.com/openshift/openshift-docs/blob/enterprise-4.16/modules/logging-content-filter-drop-records.adoc

This should be usable with the exception of needing to modify the CLF where it is contained

As discussed with @anpingli , there are no changes needed for Configuring content filters to drop unwanted log records section. I just updated the apiVersion in Example ClusterLogForwarder CR and Additional examples.

@jcantrill Let me know if you have other thoughts.

smunje1 · 2024-07-04T08:32:48Z

observability/logging/logging-6.0/log6x-clf.adoc

+
+include::modules/logging-content-filter-drop-records.adoc[leveloffset=+1]
+
+include::modules/logging-content-filter-prune-records.adoc[leveloffset=+1]


@anpingli / @jcantrill : Could you help me understand if this module can be used for 6.0 & if not what changes are needed?

https://github.com/openshift/openshift-docs/blob/enterprise-4.16/modules/logging-content-filter-prune-records.adoc

This should be usable with the exception of needing to modify the CLF where it is contained. Maybe it makes sense to remove the CLF envelope and only showcase the filter spec

As discussed with @anpingli , there are no changes needed for Configuring content filters to prune log records section. I just updated the apiVersion in Example ClusterLogForwarder CR.

@jcantrill Let me know if you have other thoughts.

jcantrill · 2024-07-09T11:46:52Z

observability/logging/logging-6.0/log6x-clf.adoc

+// This needs to be called out extensively.
+[source,yaml]
+----
+apiVersion: observability.openshift.io/v1


To what extent is this intend to represent a valid sample. As-is, this is not valid and does not properly describe a CLF?

@anpingli could you help with your input here? Thank you!

@anpingli Could you help verify if the following sample properly describe CLF and if it is valid ? If not, could you help with your input?

apiVersion: observability.openshift.io/v1 kind: ClusterLogForwarder metadata: name: collector namespace: openshift-logging spec: collector: nodeSelector: logcollect: run resources: {} outputs: - name: lokistack lokiStack: authentication: token: from: secret secret: key: token name: secret-to-lokistack labelKeys: - .hostname - .log_type - .kubernetes_container_name - .kubernetes_namespace_name - .kubernetes_pod_name target: name: lokistack-sample namespace: openshift-logging tls: ca: key: ca-bundle.crt secret: name: secret-to-lokistack type: lokiStack pipelines: - name: default-before inputRefs: - infrastructure - application outputRefs: - lokistack serviceAccount: name: logcollector

Can we use this ClusterLogForwarer CustomResourceDefinition and explain each filed?
https://github.com/openshift/enhancements/blob/master/enhancements/cluster-logging/logs-observability-openshift-io-apis.md#clusterlogforwarer-customresourcedefinition

@anpingli @@r2d2rnd @jcantrill Could you please confirm if the ClusterLogForwarer CustomResourceDefinition can be used as is in our documentation? If not, could you share the validated (working fine) CLF that we can use for our documentation?

This example should be modified. As bug opened, we should set the examples with limits for the collector and also recommending always setting them. The bug is https://issues.redhat.com/browse/OBSDOCS-856.

@anpingli @@r2d2rnd @jcantrill Could you please confirm if the ClusterLogForwarer CustomResourceDefinition can be used as is in our documentation? If not, could you share the validated (working fine) CLF that we can use for our documentation?

@smunje1 This is the enhancement document that was the initial proposal but the actual API has subtly evolved since this example. It is good source for an "overview" but the actual API is the final source of truth. This api reference is generated from the API and @libander has been including it in the docs. Additionally, we are working to provide samples that can be used. @cahartma can you please point to relevent GISTs for the time being

Thanks for the context.

Additionally, we are working to provide samples that can be used.

Can these samples be used as is i.e. Copy + Paste the samples that will be provided for our documentation?

jcantrill · 2024-07-09T11:48:22Z

observability/logging/logging-6.0/log6x-clf.adoc

+[id="log6x-CLF-tuning_{context}"]
+== Output tuning
+//need to validate how much of this applies to 6.0 & edit as needed.
+include::modules/logging-delivery-tuning.adoc[leveloffset=+1]


Output tuning is functionally the same as in 5.9 though it has moved from common to outputs to each individual spec. This was done because not all outputs support the same options

jcantrill · 2024-07-09T11:49:38Z

observability/logging/logging-6.0/log6x-clf.adoc

+The following filters are available:
+// Should be an include for each, create: modules/log6<filtername>.adoc - Associated JIRA?
+detectMultilineException:: Detects and handles multiline exceptions.
+parse:: Parses log messages using Vector's parsing language.


Parses log messages into the format defined by the filter

Tagging @libander so that we can cover this as part of OBSDOCS-1169.

jcantrill · 2024-07-09T11:50:21Z

observability/logging/logging-6.0/log6x-clf.adoc

+[id="log6x-CLF-filters_{context}"]
+== Filtering logs
+
+The following filters are available:


The complete list https://github.com/openshift/cluster-logging-operator/blob/master/api/observability/v1/filter_types.go#L24-L29

openShiftLabels,detectMultilineException, parse are the properties under filters in 6.0.

Tagging @libander so that we can cover this as part of OBSDOCS-1169.

jcantrill · 2024-07-09T11:51:50Z

observability/logging/logging-6.0/log6x-clf.adoc

+openshiftLabels:: Adds OpenShift-specific labels to log messages.
+
+// All of these need to be validated by engineering for 6.0.
+include::modules/logging-audit-log-filtering.adoc[leveloffset=+1]


This should be usable with the exception of needing to modify the CLF where it is contained

jcantrill · 2024-07-09T11:52:27Z

observability/logging/logging-6.0/log6x-clf.adoc

+// All of these need to be validated by engineering for 6.0.
+include::modules/logging-audit-log-filtering.adoc[leveloffset=+1]
+
+include::modules/logging-content-filter-drop-records.adoc[leveloffset=+1]


This should be usable with the exception of needing to modify the CLF where it is contained

jcantrill · 2024-07-09T11:53:20Z

observability/logging/logging-6.0/log6x-clf.adoc

+
+include::modules/logging-content-filter-drop-records.adoc[leveloffset=+1]
+
+include::modules/logging-content-filter-prune-records.adoc[leveloffset=+1]


This should be usable with the exception of needing to modify the CLF where it is contained. Maybe it makes sense to remove the CLF envelope and only showcase the filter spec

jcantrill · 2024-07-09T11:54:41Z

observability/logging/logging-6.0/log6x-clf.adoc

+[id="log6x-CLF-samples_{context}"]
+== Use case samples
+// Should be an include, create: modules/log6x-use-multi-instance.adoc - Associated JIRA?
+* Multiple-Instance Mode


For 6.0 we should be able to drop references to "multi log forwarder" since the intent is there is no special handling or special named instances

jcantrill · 2024-07-09T11:55:53Z

observability/logging/logging-6.0/log6x-clf.adoc

+* Multiple-Instance Mode
+
+// Should be an include, create: modules/log6x-use-metric-export.adoc - Associated JIRA?
+* Log File Metric Exporter as a Separate Deployment


This is still a feature. If users wish to see the metrics produced by this component then they will need to deploy it

anpingli · 2024-07-11T13:05:59Z

@smunje1 QE will keep updating https://docs.google.com/document/d/1qysR-ph64d8jq5sdHSsy4V5pQ_qOpBOdq2bsP3T-Jog/edit. If you refer the example in it. Please let me know if you want to provide/verify some content.

anpingli · 2024-07-23T02:48:51Z

/assign @anpingli

anpingli · 2024-07-23T07:59:28Z

observability/logging/logging-6.0/log6x-clf.adoc

+
+include::modules/logging-input-spec-filter-namespace-container.adoc[leveloffset=+1]
+
+include::modules/logging-multiline-except.adoc[leveloffset=+1]


detectMultilineException is an item under filter in 6.0

Maybe the example can be updated like?

apiVersion: "observability.openshift.io/v1" kind: ClusterLogForwarder metadata: name: <log_forwarder_name> namespace: <log_forwarder_namespace> spec: filters: - name: <name> type: detectMultilineException pipelines: - inputRefs: - <input-name> name: <pipeline-name> filterRefs: - <filter-name> outputRefs: - <output-name>

anpingli · 2024-07-23T08:03:56Z

observability/logging/logging-6.0/log6x-clf.adoc

+toc::[]
+
+The `ClusterLogForwarder` CR serves as a single point of configuration for log forwarding, making it easier to manage and maintain log collection and forwarding rules.
+


we can place pods on specific node or limit resources using .spec.collector.
For example:

spec: collector: nodeSelector: kubernetes.io/os: "linux" resources: {}

Do we need to add the above information in Configuring log forwarding section?
(Need clarity because this is just the introductory section.)

The above information will be covered in Multi log forwarder CLF and it's callouts.

Multi log forwarder CLF

For 6.0 we should reconsider the use of "multi log forwarder CLF" as starting in 6.0. CLF becomes like any other custom resource on the cluster in that there can always be "multiple". There is no need to distinguish it separately from our legacy usecases

smunje1 · 2024-08-09T05:52:50Z

As commented in Jira: OBSDOCS-1169, this PR is now only for OBSDOCS-1170. So, I am changing the PR title and it`s description.

smunje1 · 2024-08-12T07:43:27Z

observability/logging/log_storage/about-log-storage.adoc

-Loki is a horizontally scalable, highly available, multi-tenant log aggregation system offered as an alternative to Elasticsearch as a log store for the {logging}.
-
-Elasticsearch indexes incoming log records completely during ingestion. Loki only indexes a few fixed labels during ingestion and defers more complex parsing until after the logs have been stored. This means Loki can collect logs more quickly.
+include::snippets/logging-loki-statement-snip.adoc[]


Please ignore this file (Not sure how it got added in this commit). I will remove this file in next commit.

smunje1 · 2024-08-12T07:44:11Z

snippets/logging-loki-statement-snip.adoc

+
+Loki is a horizontally scalable, highly available, multi-tenant log aggregation system offered as a GA log store for {logging} {for} that can be visualized with the OpenShift {ObservabilityShortName} UI. The Loki configuration provided by OpenShift {logging-uc} is a short-term log store designed to enable users to perform fast troubleshooting with the collected logs. For that purpose, the {logging} {for} configuration of Loki has short-term storage, and is optimized for very recent queries. For long-term storage or queries over a long time period, users should look to log stores external to their cluster.
+
+Elasticsearch indexes incoming log records completely during ingestion. Loki indexes only a few fixed labels during ingestion and defers more complex parsing until after the logs have been stored. This means Loki can collect logs more quickly.


Please ignore this file (Not sure how it got added in this commit). I will remove this file in next commit.

smunje1 · 2024-08-12T07:44:21Z

snippets/logging6x-loki-statement-snip.adoc

+//
+:_mod-docs-content-type: SNIPPET
+
+Loki is a horizontally scalable, highly available, multi-tenant log aggregation system offered as a GA log store for {logging} {for} that can be visualized with the OpenShift {ObservabilityShortName} UI. The Loki configuration provided by OpenShift {logging-uc} is a short-term log store designed to enable users to perform fast troubleshooting with the collected logs. For that purpose, the {logging} {for} configuration of Loki has short-term storage, and is optimized for very recent queries. For long-term storage or queries over a long time period, users should look to log stores external to their cluster.


Please ignore this file (Not sure how it got added in this commit). I will remove this file in next commit.

OBSDOCS-1166-Loki Statement of Purpose

smunje1 · 2024-08-12T13:52:20Z

I am planning to create a new PR and later close this PR as I am getting some problems with Git for this PR.

openshift-ci · 2024-08-12T14:00:07Z

@smunje1: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

smunje1 · 2024-08-13T07:40:42Z

I have created a new PR and closing this PR as mentioned here.

Cc @jcantrill @anpingli @r2d2rnd

openshift-ci bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jul 4, 2024

smunje1 mentioned this pull request Jul 4, 2024

Logging 6x - Docs Outline / Planning #76623

Closed

smunje1 changed the title ~~OBSDOCS-1170 - Created a PR for verification of the CLF content~~ OBSDOCS-1170 and OBSDOCS-1169 - Created a PR for verification of the CLF content Jul 4, 2024

smunje1 commented Jul 4, 2024

View reviewed changes

jcantrill reviewed Jul 9, 2024

View reviewed changes

openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 16, 2024

openshift-ci bot assigned anpingli Jul 23, 2024

anpingli reviewed Jul 23, 2024

View reviewed changes

smunje1 changed the title ~~OBSDOCS-1170 and OBSDOCS-1169 - Created a PR for verification of the CLF content~~ OBSDOCS-1170 - Created a PR for verification of the CLF content Aug 9, 2024

smunje1 force-pushed the OBSDOCS-1170 branch from a199424 to 8eacedb Compare August 12, 2024 06:33

openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 12, 2024

openshift-ci bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Aug 12, 2024

smunje1 commented Aug 12, 2024

View reviewed changes

smunje1 changed the title ~~OBSDOCS-1170 - Created a PR for verification of the CLF content~~ [Hold review] OBSDOCS-1170 - Created a PR for verification of the CLF content Aug 12, 2024

smunje1 force-pushed the OBSDOCS-1170 branch from 8eacedb to 308aaee Compare August 12, 2024 10:49

openshift-ci bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Aug 12, 2024

smunje1 force-pushed the OBSDOCS-1170 branch from 308aaee to c4a9670 Compare August 12, 2024 11:03

openshift-ci bot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Aug 12, 2024

smunje1 reopened this Aug 12, 2024

openshift-ci bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Aug 12, 2024

smunje1 force-pushed the OBSDOCS-1170 branch from e283995 to a199424 Compare August 12, 2024 11:59

openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 12, 2024

openshift-ci bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Aug 12, 2024

smunje1 force-pushed the OBSDOCS-1170 branch from a199424 to 8eacedb Compare August 12, 2024 12:02

openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 12, 2024

openshift-ci bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Aug 12, 2024

smunje1 closed this Aug 12, 2024

smunje1 force-pushed the OBSDOCS-1170 branch from 8eacedb to 867f048 Compare August 12, 2024 12:06

openshift-ci bot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Aug 12, 2024

smunje1 reopened this Aug 12, 2024

smunje1 force-pushed the OBSDOCS-1170 branch from 0a095e9 to 8eacedb Compare August 12, 2024 12:24

Merge pull request openshift#78822 from shreyasiddhartha/OBSDOCS-1166

543a9ea

OBSDOCS-1166-Loki Statement of Purpose

smunje1 force-pushed the OBSDOCS-1170 branch from 8eacedb to 543a9ea Compare August 12, 2024 13:40

smunje1 closed this Aug 13, 2024

smunje1 mentioned this pull request Aug 13, 2024

OBSDOCS -1170 (New) and OBSDOCS-1104 #80384

Closed


		include::modules/logging-content-filter-drop-records.adoc[leveloffset=+1]

		include::modules/logging-content-filter-prune-records.adoc[leveloffset=+1]


		include::modules/logging-input-spec-filter-namespace-container.adoc[leveloffset=+1]

		include::modules/logging-multiline-except.adoc[leveloffset=+1]

		toc::[]

		The `ClusterLogForwarder` CR serves as a single point of configuration for log forwarding, making it easier to manage and maintain log collection and forwarding rules.


		Loki is a horizontally scalable, highly available, multi-tenant log aggregation system offered as a GA log store for {logging} {for} that can be visualized with the OpenShift {ObservabilityShortName} UI. The Loki configuration provided by OpenShift {logging-uc} is a short-term log store designed to enable users to perform fast troubleshooting with the collected logs. For that purpose, the {logging} {for} configuration of Loki has short-term storage, and is optimized for very recent queries. For long-term storage or queries over a long time period, users should look to log stores external to their cluster.

		Elasticsearch indexes incoming log records completely during ingestion. Loki indexes only a few fixed labels during ingestion and defers more complex parsing until after the logs have been stored. This means Loki can collect logs more quickly. No newline at end of file

[Hold review] OBSDOCS-1170 - Created a PR for verification of the CLF content #78477

[Hold review] OBSDOCS-1170 - Created a PR for verification of the CLF content #78477

Uh oh!

Conversation

smunje1 commented Jul 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ocpdocs-previewbot commented Jul 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

smunje1 Aug 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

smunje1 Jul 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anpingli Jul 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

smunje1 Aug 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

smunje1 Aug 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

smunje1 Aug 9, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

smunje1 Jul 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

smunje1 commented Jul 4, 2024 •

edited

Loading

ocpdocs-previewbot commented Jul 4, 2024 •

edited

Loading

smunje1 Aug 9, 2024 •

edited

Loading

smunje1 Jul 30, 2024 •

edited

Loading

anpingli Jul 30, 2024 •

edited

Loading

smunje1 Aug 9, 2024 •

edited

Loading

smunje1 Aug 9, 2024 •

edited

Loading

smunje1 Aug 9, 2024 •

edited

Loading

smunje1 Jul 18, 2024 •

edited

Loading

jcantrill Aug 8, 2024 •

edited

Loading

smunje1 Aug 9, 2024 •

edited

Loading

anpingli Jul 23, 2024 •

edited

Loading