Skip to content
Repository for all things related to Prometheus on OpenShift
Branch: master
Clone or download
wkulhanek Merge pull request #23 from omilun/patch-1
Add some comments in setup.sh
Latest commit bc0f0a1 May 16, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
alertmanager Removed Dockerfile.atomic, updated for Prometheus 2.8.0 Mar 18, 2019
node-exporter add some comments May 15, 2019
prometheus-image Removed Dockerfile.atomic, updated for Prometheus 2.8.0 Mar 18, 2019
README.adoc Fixed docs for SCC Oct 6, 2017
install_everything.yml Playbook to install everything at once Nov 8, 2017
prometheus.yaml Fixed container_cpu_usage_percent_by_host record in recording.rules Mar 27, 2018
setup.sh
setup_infranodes.yml Removed PVC Oct 6, 2017

README.adoc

Prometheus on OpenShift

This repository contains definitions and tools to run Prometheus and its associated ecosystem on Red Hat OpenShift.

Components

The following components are available:

Project Organization

A new project called prometheus will be created to contain the entire ecosystem.

Execute the following command to create the project:

oc new-project prometheus --display-name="Prometheus Monitoring"

Make sure that there is not a default node selector on the project:

oc annotate namespace prometheus openshift.io/node-selector=""

Deploy Prometheus

Starting with OpenShift 3.6 the OpenShift routers expose a metrics endpoint on port 1936. For Prometheus to be able to monitor the routers this port needs to be open.

Additionally Prometheus does not work with remote volumes (NFS, EBS, …​) but needs local disk storage as well. This means we need to create a directory on (one of) the infranodes. The Prometheus template includes a Node Selector prometheus-host=true - so we need to set the correct label on the infranode(s) as well.

Run the following Ansible playbook to configure the infranodes:

ansible-playbook -i /etc/ansible/hosts ./setup_infranodes.yml

The router also requires basic authentication to be allowed to scrape the metrics. Find the router password by executing the following command:

oc set env dc router -n default --list|grep STATS_PASSWORD|awk -F"=" '{print $2}'

An OpenShift template has been provided to streamline the deployment to OpenShift.

Execute the following command to instantiate the Prometheus template using the previously retrieved router password as a parameter:

oc new-app -f prometheus.yaml --param ROUTER_PASSWORD=<Router Password>

Since Prometheus needs to use a local disk to write its metrics add the privileged SCC to the prometheus service account:

oc adm policy add-scc-to-user privileged system:serviceaccount:prometheus:prometheus

Make sure your Prometheus pod is running (on an Infranode):

oc get pod -o wide

Next Steps

Please refer to the following to enhance the functionality of Prometheus

Cleanup

Delete the project and the cluster-reader binding (which gets created by the template but doesn’t get deleted as part of the project):

oc delete project prometheus
oc delete clusterrolebinding prometheus-cluster-reader
oc adm policy remove-scc-from-user privileged prometheus

You will also need to clean up the directory /var/lib/prometheus-data on the Infranode(s) and remove the label prometheus-host=true from the Infranode(s).

You can’t perform that action at this time.