Skip to content

Latest commit

 

History

History
64 lines (52 loc) · 2.99 KB

File metadata and controls

64 lines (52 loc) · 2.99 KB

Service Discovery Demo for Prometheus on okd

Prerequisites

Default Minishift installation.

Setup

$ oc apply -f s2i-java11.yml
$ oc apply -f demo.yml
$ oc apply -f grafana
$ oc apply -f prometheus
$ oc start-build demo --from-dir=demo
Prometheus » Targets

Key Ideas

Create a service account (prometheus) and bind a dedicated role (e.g. podview) which is only allowed to list pods within the project. This service account must be used by the prometheus pod which is configured by the deployment config variable spec.template.spec.serviceAccount.

Pods with a Prometheus metric endpoint can be configured by utilizing Kubernetes annotations such that one Prometheus scrape job is sufficient to setup the pod as target (1, 2).

The annotations are used together with Prometheus relabel config mechanism:

relabel_configs:
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]              # (1)
  regex: true
  action: keep
- source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]   # (2)
  regex: (.+)(?::\d+);(\d+)
  replacement: $1:$2
  target_label: __address__
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]                # (3)
  regex: ((?:/[^/ ]+)+)
  target_label: __metrics_path__
- source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_scheme] # (4)
  regex: https?
  target_label: __scheme__
  1. keep only targets annotated w/ prometheus.io/scrape: "true"

  2. if prometheus.io/port: "…​" annotation is present use the given port (otherwise use pods exposed port)

  3. if prometheus.io/path: …​ is present use the given path (otherwise /prometheus)

  4. if one of prometheus.io/scheme: http|https is present, use the given scheme (default is http)

A deployment configuration for a pod with a common Prometheus enabled Spring Boot application has these annotations:

template:
  metadata:
    annotations:
      prometheus.io/scrape: "true"
      prometheus.io/path: /actuator/prometheus
      prometheus.io/port: "8080"