Prometheus

Prometheus is an open-source systems monitoring and alerting toolkit that collects and stores its metrics as time series data. This sample app shows how to ingest data from Prometheus into Deephaven.

This repository was built to work with Deephaven Community Core v0.13.0. No guarantee of forwards or backwards compatibility is given.

How it works

Deephaven application mode

This app runs using Deephaven's application mode.

Components

• Dockerfile - The dockerfile for the application. This extends the default Deephaven image to add dependencies. See our guide, How to install Python packages, for more information.
• docker-compose.yml - The Docker Compose file for the application. This is mostly the same as the Deephaven docker-compose file with modifications to run Prometheus, application mode, and the custom dependencies.
• prometheusDhMetricsStart.sh - A simple helper script to launch the application.
• app.d/app.app - The Deephaven application mode app file.
• app.d/requirements.txt - Python dependencies for the application.
• app.d/prometheus.py - The Python script that pulls the data from Prometheus and stores it into Deephaven.

High level overview

This app pulls data from Prometheus's API through HTTP requests. The API responses are deserialized, and the desired values are extracted and stored into a Deephaven table.

Once data is collected and tables are created, various Deephaven queries are then performed on the tables.

This app writes to Deephaven tables both statically and dynamically.

Dependencies

• The Deephaven-core dependencies are required for this project.
• If you want to use a different Prometheus instance than the default instance, you will need to install Prometheus at your desired location.

Launch

Before launching, you can modify the PROMETHEUS_QUERIES and BASE_URL values in prometheus.py to see the results of different queries, and to point the application at different Prometheus instances.

Once you are set, simply run the following to launch the app:

sh prometheusDhMetricsStart.sh

Go to http://localhost:10000/ide to view the tables in the top right Panels tab!

Ngrok

:::note

If you are running Prometheus locally and seeing errors like:

HTTPConnectionPool(host='localhost', port=9090): Max retries exceeded with url: /api/v1/query?query=up (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f4619929a90>: Failed to establish a new connection: [Errno 111] Connection refused'))

you may need to use Ngrok to make HTTP requests to your Prometheus instance.

:::

Install Ngrok on the machine that is running your Prometheus instance, then run the following in a new terminal window on that same machine:

ngrok http 9090

Use the URL on the terminal that forwards to http://localhost:9090 to construct the BASE_URL value. Edit this value in prometheus.py and re-launch the application. For example:

BASE_URL = "{base}/api/v1/query".format(base="http://c818-2603-6081-2300-2640-50c5-4e0a-6c65-498d.ngrok.io")