Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
206 changes: 206 additions & 0 deletions modules/monitoring-scrape-targets-in-multiple-namespaces.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,206 @@
// Module included in the following assemblies:
//
// * observability/cluster-observability-operator/configuring-the-cluster-observability-operator-to-monitor-a-service.adoc

:_mod-docs-content-type: PROCEDURE
[id="monitoring-scrape-targets-in-multiple-namespaces_{context}"]
= Scrape targets in multiple namespaces

To scrape targets in multiple namespaces, set the namespace and resource selector in the `MonitoringStack` object.

.Prerequisites

* You have access to the cluster as a user with the `cluster-admin` cluster role or as a user with administrative permissions for the namespace.
* You have installed the {coo-full}.

.Procedure

. Deploy the following namespace object and `MonitoringStack` YAML file:
+
.Example `MonitoringStack`
[source,yaml]
----
apiVersion: v1
kind: Namespace
metadata:
name: ns1-coo
labels:
monitoring.rhobs/stack: multi-ns
---
apiVersion: monitoring.rhobs/v1alpha1
kind: MonitoringStack
metadata:
name: example-coo-monitoring-stack
namespace: ns1-coo
spec:
logLevel: debug
retention: 1d
resourceSelector:
matchLabels:
k8s-app: prometheus-coo-example-monitor
namespaceSelector:
matchLabels:
monitoring.rhobs/stack: multi-ns
----

. Deploy a sample application in the namespace `ns1-coo`, with an alert that is always firing:
+
[source,yaml]
----
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: prometheus-coo-example-app
name: prometheus-coo-example-app
namespace: ns1-coo
spec:
replicas: 1
selector:
matchLabels:
app: prometheus-coo-example-app
template:
metadata:
labels:
app: prometheus-coo-example-app
spec:
containers:
- image: ghcr.io/rhobs/prometheus-example-app:0.4.2
imagePullPolicy: IfNotPresent
name: prometheus-coo-example-app
---
apiVersion: v1
kind: Service
metadata:
labels:
app: prometheus-coo-example-app
name: prometheus-coo-example-app
namespace: ns1-coo
spec:
ports:
- port: 8080
protocol: TCP
targetPort: 8080
name: web
selector:
app: prometheus-coo-example-app
type: ClusterIP
---
apiVersion: monitoring.rhobs/v1
kind: ServiceMonitor
metadata:
labels:
k8s-app: prometheus-coo-example-monitor
name: prometheus-coo-example-monitor
namespace: ns1-coo
spec:
endpoints:
- interval: 30s
port: web
scheme: http
selector:
matchLabels:
app: prometheus-coo-example-app
---
apiVersion: monitoring.rhobs/v1
kind: PrometheusRule
metadata:
name: example-alert
namespace: ns1-coo
labels:
k8s-app: prometheus-coo-example-monitor
spec:
groups:
- name: example
rules:
- alert: VersionAlert
for: 1m
expr: version{job="prometheus-coo-example-app"} > 0
labels:
severity: warning
----

. Deploy the same example application in another namespace labeled with `monitoring.rhobs/stack: multi-ns`:
+
[source,yaml]
----
apiVersion: v1
kind: Namespace
metadata:
name: ns2-coo
labels:
monitoring.rhobs/stack: multi-ns
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: prometheus-coo-example-app
name: prometheus-coo-example-app
namespace: ns2-coo
spec:
replicas: 1
selector:
matchLabels:
app: prometheus-coo-example-app
template:
metadata:
labels:
app: prometheus-coo-example-app
spec:
containers:
- image: ghcr.io/rhobs/prometheus-example-app:0.4.2
imagePullPolicy: IfNotPresent
name: prometheus-coo-example-app
---
apiVersion: v1
kind: Service
metadata:
labels:
app: prometheus-coo-example-app
name: prometheus-coo-example-app
namespace: ns2-coo
spec:
ports:
- port: 8080
protocol: TCP
targetPort: 8080
name: web
selector:
app: prometheus-coo-example-app
type: ClusterIP
---
apiVersion: monitoring.rhobs/v1
kind: ServiceMonitor
metadata:
labels:
k8s-app: prometheus-coo-example-monitor
name: prometheus-coo-example-monitor
namespace: ns2-coo
spec:
endpoints:
- interval: 30s
port: web
scheme: http
selector:
matchLabels:
app: prometheus-coo-example-app
----

.Verification

. Verify that the Prometheus instance adds new targets and that the alert are firing. Use a port-forward command to expose the Prometheus or the Alertmanager user interface that has been deployed by the `Monitoringstack` instance.
+
.Prometheus
[source,terminal]
----
$ oc port-forward -n ns1-coo pod/prometheus-example-coo-monitoring-stack-0 9090
----
+
.Alertmanager
[source,terminal]
----
$ oc port-forward -n ns1-coo pod/alertmanager-example-coo-monitoring-stack-0 9093
----

. Verify that the targets are being scraped and that the alerts are firing by browsing to `http://localhost:9090/targets` or `http://localhost:9093/#/alerts`.
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ include::modules/monitoring-specifying-how-a-service-is-monitored-by-cluster-obs
// Create a MonitoringStack object to discover the service monitor
include::modules/monitoring-creating-a-monitoringstack-object-for-cluster-observability-operator.adoc[leveloffset=+1]

// Validate a MonitoringStack
// Validate a MonitoringStack
include::modules/monitoring-validating-a-monitoringstack-for-cluster-observability-operator.adoc[leveloffset=+1]

// Scrape targets in multiple namespaces
include::modules/monitoring-scrape-targets-in-multiple-namespaces.adoc[leveloffset=+1]