From 064240cee8032a8140aeb6b5a19128d13c9a9662 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Sinclert=20P=C3=A9rez?= Date: Tue, 3 Mar 2020 16:49:03 -0500 Subject: [PATCH 1/2] Improve multiproc documentation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Sinclert Pérez --- README.md | 57 ++++++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 42 insertions(+), 15 deletions(-) diff --git a/README.md b/README.md index 586f5315..91798341 100644 --- a/README.md +++ b/README.md @@ -489,13 +489,43 @@ This comes with a number of limitations: There's several steps to getting this working: -**One**: Gunicorn deployment +**1. Gunicorn deployment**: The `prometheus_multiproc_dir` environment variable must be set to a directory that the client library can use for metrics. This directory must be wiped between Gunicorn runs (before startup is recommended). -Put the following in the config file: +This environment variable must be set **from a start-up shell script**, +and not directly from Python (otherwise it will not propagate to child processes). + +**2. Metrics collector**: + +The application must initialize a new metrics `CollectorRegistry`, +and save it within the multi-process collector. + +```python +from prometheus_client import multiprocess +from prometheus_client import generate_latest, CollectorRegistry, CONTENT_TYPE_LATEST + +# Expose metrics. +def app(environ, start_response): + registry = CollectorRegistry() + multiprocess.MultiProcessCollector(registry) + data = generate_latest(registry) + status = '200 OK' + response_headers = [ + ('Content-type', CONTENT_TYPE_LATEST), + ('Content-Length', str(len(data))) + ] + start_response(status, response_headers) + return iter([data]) +``` + +**3. Metrics mode and configuration (Gauge)**: + +When `Gauge` metrics are used, additional tuning needs to be performed. +First of all, the `gunicorn` configuration file need to include the following function: + ```python from prometheus_client import multiprocess @@ -503,7 +533,16 @@ def child_exit(server, worker): multiprocess.mark_process_dead(worker.pid) ``` -**Two**: Inside the application +Second of, metrics instrumentation. Gauges have several modes they can run in, +which can be selected with the `multiprocess_mode` parameter. + +- 'all': Default. Return a timeseries per process alive or dead. +- 'liveall': Return a timeseries per process that is still alive. +- 'livesum': Return a single timeseries that is the sum of the values of alive processes. +- 'max': Return a single timeseries that is the maximum of the values of all processes, alive or dead. +- 'min': Return a single timeseries that is the minimum of the values of all processes, alive or dead. + +Complete example: ```python from prometheus_client import multiprocess from prometheus_client import generate_latest, CollectorRegistry, CONTENT_TYPE_LATEST, Gauge @@ -527,18 +566,6 @@ def app(environ, start_response): return iter([data]) ``` -**Three**: Instrumentation - -Counters, Summarys and Histograms work as normal. - -Gauges have several modes they can run in, which can be selected with the -`multiprocess_mode` parameter. - -- 'all': Default. Return a timeseries per process alive or dead. -- 'liveall': Return a timeseries per process that is still alive. -- 'livesum': Return a single timeseries that is the sum of the values of alive processes. -- 'max': Return a single timeseries that is the maximum of the values of all processes, alive or dead. -- 'min': Return a single timeseries that is the minimum of the values of all processes, alive or dead. ## Parser From ba2432ca8a908deebe0acfabbce317217a4ddeb9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Sinclert=20P=C3=A9rez?= Date: Wed, 4 Mar 2020 10:19:50 -0500 Subject: [PATCH 2/2] Apply suggestions to multiproc docs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Sinclert Pérez --- README.md | 40 ++++++++++++---------------------------- 1 file changed, 12 insertions(+), 28 deletions(-) diff --git a/README.md b/README.md index 91798341..3e4a8bb0 100644 --- a/README.md +++ b/README.md @@ -495,13 +495,13 @@ The `prometheus_multiproc_dir` environment variable must be set to a directory that the client library can use for metrics. This directory must be wiped between Gunicorn runs (before startup is recommended). -This environment variable must be set **from a start-up shell script**, -and not directly from Python (otherwise it will not propagate to child processes). +This environment variable should be set from a start-up shell script, +and not directly from Python (otherwise it may not propagate to child processes). **2. Metrics collector**: -The application must initialize a new metrics `CollectorRegistry`, -and save it within the multi-process collector. +The application must initialize a new `CollectorRegistry`, +and store the multi-process collector inside. ```python from prometheus_client import multiprocess @@ -521,10 +521,9 @@ def app(environ, start_response): return iter([data]) ``` -**3. Metrics mode and configuration (Gauge)**: +**3. Gunicorn configuration**: -When `Gauge` metrics are used, additional tuning needs to be performed. -First of all, the `gunicorn` configuration file need to include the following function: +The `gunicorn` configuration file needs to include the following function: ```python from prometheus_client import multiprocess @@ -533,8 +532,10 @@ def child_exit(server, worker): multiprocess.mark_process_dead(worker.pid) ``` -Second of, metrics instrumentation. Gauges have several modes they can run in, -which can be selected with the `multiprocess_mode` parameter. +**4. Metrics tuning (Gauge)**: + +When `Gauge` metrics are used, additional tuning needs to be performed. +Gauges have several modes they can run in, which can be selected with the `multiprocess_mode` parameter. - 'all': Default. Return a timeseries per process alive or dead. - 'liveall': Return a timeseries per process that is still alive. @@ -542,28 +543,11 @@ which can be selected with the `multiprocess_mode` parameter. - 'max': Return a single timeseries that is the maximum of the values of all processes, alive or dead. - 'min': Return a single timeseries that is the minimum of the values of all processes, alive or dead. -Complete example: ```python -from prometheus_client import multiprocess -from prometheus_client import generate_latest, CollectorRegistry, CONTENT_TYPE_LATEST, Gauge +from prometheus_client import Gauge -# Example gauge. +# Example gauge IN_PROGRESS = Gauge("inprogress_requests", "help", multiprocess_mode='livesum') - - -# Expose metrics. -@IN_PROGRESS.track_inprogress() -def app(environ, start_response): - registry = CollectorRegistry() - multiprocess.MultiProcessCollector(registry) - data = generate_latest(registry) - status = '200 OK' - response_headers = [ - ('Content-type', CONTENT_TYPE_LATEST), - ('Content-Length', str(len(data))) - ] - start_response(status, response_headers) - return iter([data]) ```