[enterprise-4.11] RHDEVDOCS-3401 Document how to enable Prometheus query logs in OpenShift Monitoring

modules/monitoring-setting-query-log-file-for-prometheus.adoc

@@ -0,0 +1,132 @@
+// Module included in the following assemblies:
+//
+// * monitoring/configuring-the-monitoring-stack.adoc
+
+[id="setting-query-log-file-for-prometheus_{context}"]
+:_content-type: PROCEDURE
+= Enabling the query log file for Prometheus
+
+[role="_abstract"]
+You can configure Prometheus to write all queries that have been run by the engine to a log file. You can do so for default platform monitoring and for user-defined workload monitoring.
+
+[IMPORTANT]
+====
+Because log rotation is not supported, only enable this feature temporarily when you need to troubleshoot an issue. After you finish troubleshooting, disable query logging by reverting the changes you made to the `ConfigMap` object to enable the feature.
+====
+
+.Prerequisites
+
+* You have installed the OpenShift CLI (`oc`).
+* *If you are enabling the query log file feature for Prometheus in the `openshift-monitoring` project*:
+** You have access to the cluster as a user with the `cluster-admin` role.
+** You have created the `cluster-monitoring-config` `ConfigMap` object.
+* *If you are enabling the query log file feature for Prometheus in the `openshift-user-workload-monitoring` project*:
+** You have access to the cluster as a user with the `cluster-admin` role, or as a user with the `user-workload-monitoring-config-edit` role in the `openshift-user-workload-monitoring` project.
+** You have created the `user-workload-monitoring-config` `ConfigMap` object.
+
+.Procedure
+
+** *To set the query log file for Prometheus in the `openshift-monitoring` project*:
+. Edit the `cluster-monitoring-config` `ConfigMap` object in the `openshift-monitoring` project:
++
+[source,terminal]
+----
+$ oc -n openshift-monitoring edit configmap cluster-monitoring-config
+----
++
+. Add `queryLogFile: <path>` for `prometheusK8s` under `data/config.yaml`:
++
+[source,yaml]
+----
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: cluster-monitoring-config
+  namespace: openshift-monitoring
+data:
+  config.yaml: |
+    prometheusK8s:
+      queryLogFile: <path> <1>
+----
+<1> The full path to the file in which queries will be logged.
++
+. Save the file to apply the changes.
++
+[WARNING]
+====
+When you save changes to a monitoring config map, pods and other resources in the related project might be redeployed. The running monitoring processes in that project might also be restarted.
+====
++
+. Verify that the pods for the component are running. The following sample command lists the status of pods in the `openshift-monitoring` project:
++
+[source,terminal]
+----
+$ oc -n openshift-monitoring get pods
+----
++
+. Read the query log:
++
+[source,terminal]
+----
+$ oc -n openshift-monitoring exec prometheus-k8s-0 -- cat <path>
+----
++
+[IMPORTANT]
+====
+Revert the setting in the config map after you have examined the logged query information.
+====
+
+** *To set the query log file for Prometheus in the `openshift-user-workload-monitoring` project*:
+. Edit the `user-workload-monitoring-config` `ConfigMap` object in the `openshift-user-workload-monitoring` project:
++
+[source,terminal]
+----
+$ oc -n openshift-user-workload-monitoring edit configmap user-workload-monitoring-config
+----
++
+. Add `queryLogFile: <path>` for `prometheus` under `data/config.yaml`:
++
+[source,yaml]
+----
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: user-workload-monitoring-config
+  namespace: openshift-user-workload-monitoring
+data:
+  config.yaml: |
+    prometheus:
+      queryLogFile: <path> <1>
+----
+<1> The full path to the file in which queries will be logged.
++
+. Save the file to apply the changes.
++
+[NOTE]
+====
+Configurations applied to the `user-workload-monitoring-config` `ConfigMap` object are not activated unless a cluster administrator has enabled monitoring for user-defined projects.
+====
++
+[WARNING]
+====
+When you save changes to a monitoring config map, pods and other resources in the related project might be redeployed. The running monitoring processes in that project might also be restarted.
+====
++
+. Verify that the pods for the component are running. The following example command lists the status of pods in the `openshift-user-workload-monitoring` project:
++
+[source,terminal]
+----
+$ oc -n openshift-user-workload-monitoring get pods
+----
++
+. Read the query log:
++
+[source,terminal]
+----
+$ oc -n openshift-user-workload-monitoring exec prometheus-user-workload-0 -- cat <path>
+----
++
+[IMPORTANT]
+====
+Revert the setting in the config map after you have examined the logged query information.
+====

monitoring/configuring-the-monitoring-stack.adoc

@@ -149,11 +149,14 @@ include::modules/monitoring-attaching-additional-labels-to-your-time-series-and-
 // Setting log levels for monitoring components
 include::modules/monitoring-setting-log-levels-for-monitoring-components.adoc[leveloffset=+1]
 
+// Setting query log for Prometheus
+include::modules/monitoring-setting-query-log-file-for-prometheus.adoc[leveloffset=+1]
+
 [role="_additional-resources"]
 .Additional resources
 
 * See xref:../monitoring/configuring-the-monitoring-stack.adoc#preparing-to-configure-the-monitoring-stack[Preparing to configure the monitoring stack] for steps to create monitoring config maps
-* xref:../monitoring/enabling-monitoring-for-user-defined-projects.adoc#enabling-monitoring-for-user-defined-projects[Enabling monitoring for user-defined projects]
+* See xref:../monitoring/enabling-monitoring-for-user-defined-projects.adoc#enabling-monitoring-for-user-defined-projects[Enabling monitoring for user-defined projects] for steps to enable user-defined monitoring.
 
 // Disabling the default Grafana deployment
 include::modules/monitoring-disabling-grafana.adoc[leveloffset=+1]