Add service monitor for simple configuration of prometheus metrics

[duartesaraiva98]

The idea behind this pull request is to add a serviceMonitor configuration to the helm chart so that it can be used in combination with the prometheus operator.

The configuration allows for simple provision of the correct metric and offset endpoints.

Conceptually, the split was made between metrics and offsets. The options are:

• get all metrics OR specific metrics for schemas and clusters
• get all offsets OR specific offsets for topics and clusters
• get group offsets OR not

Configuration examples

All metrics + Offsets

serviceMonitor:
  enabled: true
  metrics:
    clusters: []
    schemas: []
  offsets:
    groups: true
    topics: []
    clusters: []

Will generate:

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: release-name-kpow
spec:
  endpoints:
    - interval: 15s
       port: 3000
       path: /metrics/v1
    - interval: 15s
       port: 3000
       path: /offsets/v1
    - interval: 15s
       port: 3000
       path: /group-offsets/v1

Specific Metrics

serviceMonitor:
  enabled: true
  metrics:
    clusters: 
     - cluster-1
    schemas:
    - schema-1
  offsets:
    groups: true
    topics: []
    clusters: []

Will generate:

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: release-name-kpow
spec:
  endpoints:
    - interval: 15s
       port: 3000
       path: /metrics/v1/cluster/cluster-1   
    - interval: 15s
       port: 3000
       path: /metrics/v1/schema/schema-1
    - interval: 15s
       port: 3000
       path: /offsets/v1
    - interval: 15s
       port: 3000
       path: /group-offsets/v1

Specific Offsets

serviceMonitor:
  enabled: true
  metrics:
    clusters: []
    schemas: []
  offsets:
    groups: true
    topics: 
      - topic-1
    clusters: 
      - cluster-1

Will generate:

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: release-name-kpow
spec:
  endpoints:
    - interval: 15s
       port: 3000
       path: /metrics/v1
    - interval: 15s
       port: 3000
       path: /offsets/v1/topic/topic-1
    - interval: 15s
       port: 3000
       path: /offsets/v1/cluster/cluster-1
    - interval: 15s
       port: 3000
       path: /group-offsets/v1

Without group offsets

serviceMonitor:
  enabled: true
  metrics:
    clusters: []
    schemas: []
  offsets:
    groups: false
    topics: []
    clusters: []

Will generate:

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: release-name-kpow
spec:
  endpoints:
    - interval: 15s
       port: 3000
       path: /metrics/v1
    - interval: 15s
       port: 3000
       path: /offsets/v1

[d-t-w]

Hello @duartesaraiva98, thanks for your contribution!

I will review this next week and come back to you with any questions or comments, have a nice weekend.

[duartesaraiva98]

Hi @d-t-w ! Any update on this pull request?

[d-t-w]

Hi @duartesaraiva98 we are paused on this one for a short while as we need to investigate:

1. Should this be a part of OpenShift specific helm charts (many of our users are not on OS).
2. Should we implement this after we release v2 of our metrics endpoints.

We have some upcoming work on metrics that might tie in with a slightly different implementation.